Step-by-step guide to scraping Facebook Pages without the official API. Extract and download Facebook posts, comments, reviews, and photos.
Read on for our detailed step-by-step guide to scraping Facebook Pages without using the official API. Extract and download Facebook posts, comments, reviews, photos, and any other publicly available information with Apify's Facebook Pages Scraper.
Facebook is more than just a social platform. It’s also a powerful business platform and one of its strongest features is that it directly connects consumers to brands.
Because of this, Facebook is home to the biggest brands in the world, meaning you can find trends on posts and comments on those pages, and gauge consumer interest. You can also extract detailed data, such as:
- geolocation coordinates
- like counts
- company website
- local traffic data
- available payment methods
Getting data with the Facebook API
Some of this data is available through their official API, but Facebook’s rules and rate limits are strictly enforced: you won’t be able to extract a lot of information in a short period of time and you can easily get your API key blocked.
That’s where Apify’s Facebook Pages Scraper comes in 😉
Apify’s readymade Facebook Pages Scraper will create an unofficial Facebook API that will let you do a lot more. You will be able to scrape any publicly available information from any page, without running up against Facebook’s limits. You can also fine-tune your scraping runs to filter the results by minimum and maximum dates, set the maximum number of results you want to receive, set your preferred language, and get your data in a matter of minutes.
Once you have run the scraper, you can download your data as HTML table, JSON, CSV, Excel, XML, and RSS feed. You can then use it however you want, with no limitations.
Facebook pages are usually public and don’t require a login to allow you to fetch information. This gives our Facebook Scraper the freedom it needs to scrape pages with no limits, although much (all, as of 2021 😞) of the time you will need to use Apify Proxy to get the most out of our Facebook scraping tool.
Update: in 2021, Facebook changed the rules so that you need to use residential proxies if you want to scrape Facebook posts. You can still get some results (such as reviews, about, and some other content) with just datacenter proxies, but we strongly recommend using residential proxies for all Facebook scraping. Luckily, our free plan now includes a free trial of residential proxies so you can fully test the scraper.
Is it legal to scrape Facebook Pages?
In principle, yes, scraping is legal. To keep it that way, you should make sure only to crawl publicly available websites and not scrape copyrighted content or accumulate personal data without having a legal basis for doing so. Legal rules must be followed, such as the European GDPR and DSM directive or American Fair Use doctrine and CCPA regulations. Read more on the legality of web scraping at our blog post on the subject.
How to scrape Facebook Pages with Apify’s Facebook Scraper
Facebook consists of user profiles and pages. Facebook Pages are used by brands, companies, and organizations to provide information about their product or service. A typical Facebook Page looks like this:
If you want to scrape a Facebook URL that you think is a page, you should first check that it is an actual “page” by appending /pg/ before the username. When it’s a page, it will redirect you to the page itself. If it’s just a profile, it will give you a “not found” page (Facebook’s broken thumb). Once you know you’re dealing with a page, you know what to expect from the URL and can start to scrape.
The next step is to check for the correct layout: Facebook is a huge platform available in a lot of countries, and sometimes they will serve a different layout of the page depending on your IP geographical location, your browser user agent, and how long it takes to load.
So you’ve determined that the URL is a Facebook page and you’ve set the country to be stable using Apify Proxy. Now you can start scraping. And here’s our step-by-step tutorial to doing exactly that 🐣
How to scrape Facebook: a step-by-step guide to scraping Facebook Pages
Let’s go! We’ll use the Apify platform and the readymade Facebook Scraper. The scraper, or crawler, will enable you to extract and download posts, comments, reviews, and any other public data from Facebook Pages.
Because Facebook changed its policies in 2021 😢 you’ll need residential proxies to scrape Facebook. Just email firstname.lastname@example.org and say that you need a free trial of residential proxies for scraping Facebook!
1. Go to Apify and sign in.
If you don’t already have an account, you can sign up by using your email, Google account, or GitHub account.
2. Now you’ll find yourself in Apify Console. This is full of useful tools, but we’re only interested in finding the Facebook scraper for now, so click on the Store button. That’s where you can find Apify’s ready-made scraping tools. We call them actors.
3. Once you’re on Apify Store, search for Facebook.
4. On the Facebook Pages Scraper, click Try for free and it will automatically redirect you back to your Apify Console.
5. On your Apify Console, a Task will be created. Now you can start specifying how you want to scrape data from Facebook.
6. Let’s visit Facebook and see what data you might like to scrape and download.
7. Find something you’re interested in and copy the URL. For example, we might decide to scrape the Apify Facebook page, so we would copy https://www.facebook.com/apifytech
8. Go back to the Apify task and click “+ Add URL”. Paste your Facebook URL into the field that appears.
Tip 1: To speed up scraping Facebook, set low numbers for the following parameters — max posts, max comments, max reviews.
9. Click on the Save & Run button and wait for the scraping to finish.
10. Your task will change status to show that it is Running. Once the status changes to Succeeded, you can click on the Dataset tab to see the results.
11. The Dataset tab contains your data in lots of useful formats, including HTML table, JSON, CSV, Excel, XML, and RSS feed. You can open them by clicking on View or Download. You can then share the data, or upload it anywhere you like. Use it in spreadsheets, other programs or apps, or your own projects!
So now you can scrape data from Facebook. It’s as easy as that. Have fun 🥳