How to scrape Instagram posts, comments, and photos

Jarda Hejlek
Jarda Hejlek
Table of Contents

How to use our unofficial Instagram API to extract Instagram profile data, comments, hashtags, likes, locations, and photos. New for 2022 - dedicated, streamlined miniscrapers!

The official Instagram API allows you to programmatically access your own comments and posts on Instagram. However, the Instagram API doesn’t allow you to get a list of posts made by other people, comments, and photos on the posts, or get a list of posts with a particular hashtag.

To extract that kind of information, you'll need to use our Instagram Scraper. The scraper is free to use and will let you download all of this public data from Instagram:

  • Profiles
  • Hashtags
  • Locations
  • Comments
  • Likes

Scraping publicly available data is legal, but you need to be careful not to extract content that is protected by copyright or contains personal information. So, after scraping Instagram, double-check your output for data that would go against GDPR, CCPA, or could be considered intellectual property.

Read more about the legality of web scraping in our blog post on the subject.

Just want to scrape Instagram posts, hashtags, profiles, or comments?

New, streamlined Instagram scrapers for 2022!

Now you can use our ultra-fast and easy dedicated scrapers if you want to scrape specific Instagram data. These Instagram scrapers have fewer settings to change and they'll give you fast, clean results ⬇️

Just enter one or more Instagram usernames or URLs and click to scrape.

Just want to scrape Instagram profiles? Try Instagram Profile Scraper

Where to find the data

The easiest way to search and access content on Instagram is by using the mobile app or the website:

To scrape Instagram search and access content on the Instagram app or website
Medium on Instagram

Do you need a log-in to access Instagram? Some of Instagram is accessible even without logging in, but not as much as in the past. Once you log in, you can browse through hashtags, profiles, locations, and posts. This is very encouraging because if you can do something manually in a web browser, you can automate it on Apify 😉

Just want to scrape Instagram posts? Try Instagram Post Scraper

Data available publicly without login

If you check the website in an incognito browser window, you’ll quickly find that there are some features that you can access freely and some that are either blocked or require you to log in. Here is what you can find without a login (Instagram keeps changing the rules, so some of this might have changed):

You can search for profiles, hashtags, and places, and Instagram will return the top 100 posts.

Scrape Instagram profiles, posts, hashtags, and comments with Instagram scrapers
The search feature on Instagram.com

There is even a nice internal API endpoint that can be used to get the results in JSON format:

https://www.instagram.com/web/search/topsearch/?context=blended&query=avengers

The context query parameter serves as the filter and it can contain a location, user, or hashtag. The only limitation is that the endpoint returns just 100 results. If you need more, you need to enter a more detailed filter.

Want to scrape only Instagram hashtags? Try Instagram Hashtag Scraper

2. Posts from Profiles/Hashtags/Locations

When you open any public Instagram page that contains posts (e.g. profile, hashtag or location), Instagram will return an HTML page with the first few posts preloaded (probably using React server-side rendering). Then, when you scroll down the page, Instagram will continue loading more posts using an XHR request to Instagram’s GraphQL endpoint. The endpoint is protected with a token, so it’s not really possible to access it directly and we need to infinitely scroll the page. However, we can automate infinite scrolling nicely using headless Chrome with Puppeteer.

Scrape Instagram posts from profiles, hashtags, and locations
Posts visible on Instagram’s own profile page

Our tests haven’t found a limit to how many posts can be loaded using infinite scroll. There probably is one, but even up to a thousand posts were loaded during testing.

3. Comments on posts

Every Instagram post has publicly visible comments and shows a Load more comments button if there are more comments that can be shown.

Scrape Instagram comments with Instagram Comments Scraper
Comments on Medium.com’s Instagram post - “Medication” by @misscaseyjo

Clicking on the button fires off an XHR request to Instagram’s GraphQL endpoint. Again, we can easily automate this using Puppeteer’s page.click() function and then extract the content of the comments from the web page.

Want to scrape only Instagram comments? Try Instagram Comments Scraper

Data available only after login

Unfortunately, certain content can be only accessed if you’re logged in using your Instagram account, for example:

  • List of followers
  • List of people a user follows

Although it would be possible to automatically log in to Instagram in order to access this data, this approach is risky since it can lead to the banning of your account by Instagram. Sure, you could create a fake Instagram account and use that instead, but that’s beyond the scope of this article.

Over time, Instagram has been increasingly limiting the data you can access without login, so you’ll need to test to see what you can scrape.

Using an Apify actor to scrape the data

Our Instagram Scraper is one of many actors available in Apify Store. Apify actors are cloud programs that accept input, perform their job, and generate some output. They can be run manually in the app, using the API or scheduler.

The actor is written in Node.js and uses Apify SDK. On input, it takes an Instagram query or a list of direct profile URLs, then it searches the query and scrapes page details, posts, or comments from the results and direct URLs. All of the resulting data is stored in a structured form into a dataset, from which you can download it in formats such as JSON, XML, Excel, CSV, etc. The source code for Instagram Scraper is available on GitHub — pull requests and ideas for improvement are more than welcome!

Our Instagram scraper is free to use, although you will need to use residential proxies on Apify Proxy if you run it on the Apify platform. This is because Instagram changed the rules in 2021 and now you always need to use a residential proxy for scraping 😖

Luckily, every new Apify account includes a small free trial of residential proxies, so you should be able to test Instagram Scraper or any of the miniscrapers 🤩

Before you get started scraping Instagram, note that we do not consider scraping vast amounts of personal data ethical and discourage anyone from doing so.

How to scrape Instagram with Instagram Scraper

So you want to use our Instagram Scraper to extract data from Instagram? This step-by-step guide should get you started in just a few minutes.

Step 1. Go to Apify Store for Instagram Scraper

When you are on the actor's page, click the Try for free button.

If you already have an Apify account, you will be redirected to Apify Console, which is your workspace to run tasks for your scrapers. If you don't have an Apify account, you can sign up using your email account, Google, or GitHub.

How to scrape Instagram Step 1. Go to Instagram Scraper on Apify Store and click Try for free

How to scrape Instagram. Sign up to Apify for free

Step 2. Insert your search query or URLs in the Input field

Fill in the input fields, such as the Instagram URLs you want to scrape or what kind of data you want to extract.

You can add multiple URLs by clicking the +Add button.

Insert the maximum number of items you want to scrape in the Max items field.

How to scrape Instagram Step 2. Insert your search query or URLs in the Input field

Step 3. Run the scraper to collect your data

Once you are all set, click the Start button.

Notice that your task will change its status to Running, so wait for the scraper's run to finish. It will be just a minute before you see the status switch to Succeeded.

How to scrape Instagram Step 3. Run the scraper to collect your data

Step 4. Download your data

Move to the Dataset / Storage tab for the results of your scraping. This tab contains your data in lots of versatile formats: HTML table, JSON, CSV, Excel, XML, and RSS feed.

You can open them by clicking on View in another tab, Preview, or Download. You can then share the data, or upload it anywhere you like. Use it in spreadsheets, other programs or apps, or your own projects.

How to scrape Instagram Step 4. Download your data

And that's all you need to get started scraping Instagram. If you run into any problems or need some advice, just email support@apify.com and we'll try to help. You can also join the conversation on our Discord server, where the Apify team regularly discuss solutions to tricky scraping problems.

If you need to scrape Instagram at scale or need end-to-end service, you can request a custom solution.



Great! Next, complete checkout for full access to Apify
Welcome back! You've successfully signed in
You've successfully subscribed to Apify
Success! Your account is fully activated, you now have access to all content
Success! Your billing info has been updated
Your billing was not updated