Content
Our amazing users have already built more than 10,000 crawlers for lots of websites, but except for the community forum there has been nowhere for people to share their crawlers, exchange know-how and reuse existing solutions to avoid reinventing the wheel. Today, we are launching exactly that kind of place — the community crawlers page.
We hope that by connecting the Apifier community we will save people a lot of valuable time and help them build better crawlers. For new users, learning by example is often the best way to get started. Right now, the community page already contains crawlers for some of the most demanded sites, such as Amazon, Google, Yelp, Booking.com, as well as a generic crawler for SEO analysis, which a lot of people have asked for.
Sharing a crawler only takes a minute. Simply go to the crawler configuration page and click “Publish”.
Then you’ll need to fill in a short form. We understand that not everyone wants to associate their name with their crawler, so we’ve also added the option to publish crawlers anonymously.
By sharing a crawler you can start building your portfolio and could eventually land a contract from business people who cannot or don’t want to code (hey, nobody’s perfect). We’ll be launching more features like this very soon, so stay tuned!
To reward contributions to the community, we’re going to lift limits for everyone who shares a meaningful crawler. For every shared and approved crawler, we will give you one extra parallel crawling or web scraping process and 10,000 pages per month.
Our to-do list is already full of ideas on how to improve the community page, such as making it possible for you to ask the creators of crawlers questions, add GitHub integration or enabling people to tell others more about themselves and their skills. We’d love to hear your thoughts — just open a discussion on the community forum or let us know at info@apifier.com
Happy sharing!
