Apify Blog

How to scrape any website (for beginners)

Find out how to use Apify to scrape web data fast and easily. Step-by-step example of collecting Stranger Things news from CNN. Use our guide to web scraping for beginners to find out how to use Apify to scrape web data fast and easily. Includes illustrated step-by-step example of collecting Strang

9 min read Jul 4, 2019

Instagram

How to scrape Instagram posts, comments, and photos

Jarda Hejlek

Extract public Instagram profile data, comments and photos using the new jaroslavhejlek/instagram-scraper actor on Apify. Extract public Instagram profile data, comments and photos using the new jaroslavhejlek/instagram-scraper actor on Apify. Go beyond what the official Instagram API allows.

4 min read May 20, 2019

Web Scraping

What is the difference between web scraping and crawling?

David Barton

Crawling isn’t always the same as scraping. We explain how scraping is different and how it gives you a lot more control and freedom. Web crawling isn’t always the same as web scraping. We explain how scraping is different and how it gives you a lot more control and freedom - especially when you us

4 min read Mar 22, 2019

Booking

Scrape hotel data from Booking.com

Administrator

Apify’s Booking.com scraper lets you collect comprehensive data about hotels in machine-readable format.

4 min read Mar 7, 2019

Google

Unofficial Google Search API from Apify

Marek Trunkát

Extract organic results, ads, product ads and related queries from Google Search result pages. Supports custom country and language. New Apify actor to scrape Google Search result pages (SERPs) and extract a list of organic and paid results, ads, snap packs and more. Supports custom country, langua

5 min read Feb 25, 2019

API

Import data easily with Apify’s new Google Sheets actor

Lukáš Křivka

Integrate your Apify actors and Google spreadsheets to upload lists of URLs, save crawled data for later processing and lots more Google Sheets is the most widely used tool for handling Excel-like spreadsheets on the web. Now you can unleash the full potential of Apify actors and Google spreadsheet

10 min read Jan 30, 2019

Monitoring

How to set up a content change watchdog for any website in 5 minutes

Jakub Balada

Use an Apify actor to regularly check specific content on any website and send you an email notification whenever it changes. Use a simple Apify actor to quickly create a custom watchdog for any website. Monitor specific CSS and receive an email notification whenever the content changes.

4 min read Jan 9, 2019

API

Kickstarter search actor — create your own Kickstarter API

Jarda Hejlek

Search Kickstarter quicker, easier and with machine-readable results

5 min read Dec 21, 2018

VPN

Tunneling arbitrary protocols over HTTP proxy with static IP address in Node.js

Jarda Hejlek

How to connect from a Node.js application running on a server with a random IP address to an external service protected by a firewall. Learn how to connect from a Node.js application running on a server with a random IP address to an external service protected by a firewall that only allows access

5 min read May 2, 2018

Web Scraping

Web scraping in 2018 — forget HTML, use XHRs, metadata or JavaScript variables

Jakub Balada

Learn modern ways to extract data from web pages using XHRs, Schema.org Microdata, JSON Linked Data, JavaScript variables and more. Web scraping in 2018 is not only about processing HTML. Learn modern ways to extract data from web pages using XHRs, Schema.org Microdata, JSON Linked Data, JavaScript

7 min read Feb 8, 2018

Proxy

How to make headless Chrome and Puppeteer use a proxy server with authentication

Jan Čurn

We released an open-source package called proxy-chain on NPM to enable running headless Chrome and Puppeteer over proxy with… We have released a new open-source package called proxy-chain on NPM to enable running headless Chrome and Puppeteer over a proxy server that requires authentication.

3 min read Dec 29, 2017

Automation

How to never miss a beat on ever changing websites

Marek Trunkát

The web is constantly changing. Posts in discussions are being deleted, online stores are changing prices on hourly bases and news sites…

2 min read Nov 9, 2017

Tutorial