Content
We're Apify. We've created 2,000+ data extraction tools and unofficial APIs for popular websites, including Instagram. Check us out.
The limitations of the Instagram APIs brought about the need to find alternatives to get around these limitations such as Instagram Scraper. This scraper is one of the first web scrapers developed and published on Apify Store. It was developed as an unofficial Instagram API to help extract data from Instagram.
30K+ users and a few behind-login policies from Instagram later, it's still going strong, enabling users to extract data from profiles, posts, comments and media. And the best part: you don't have to have deep technical knowledge to be able to use this scraper. You won't have to use a terminal, line up queries, get authorization tokens, and all that jazz. So let's learn how to use it together.
💿 Why not use the official Instagram API for scraping?
Instagram does provide two free public APIs that can be used for various purposes, including web scraping and data extraction. But would those APIs be enough for scraping Instagram?
Not really. The official public Instagram API comes in two forms: Instagram Basic Display API and Instagram Graph API. Both of them allow you to programmatically access posts on Instagram. However, the extent and scope of it varies depending on the API type.
Instagram Basic Display API
The secret lies in its name. Instagram Basic Display API is, well, basic. You can use it to scrape Instagram profiles but the data output will be limited to very basic IG post data such as username, account type, caption, media type, media URL, permalink, timestamp, and thumbnail. You cannot use Instagram Basic Display API to scrape comments, stories, reels, promoted posts, and IGTV content. Needless to say, accessing Basic Display API has a few prerequisites: you should have/submit a Facebook developer account, an Instagram account, and a public website of your own.
Instagram Graph API
You can use the Instagram Graph API to find mentions, get basic data on other businesses such as the number of followers, likes, number of comments, or get hashtagged photos. You cannot use Instagram Graph API to get regular Instagram user data; it is created specifically for Instagram business accounts and creators. Besides, accessing this API requires an Instagram Business or Creator account, a Facebook page connected to this account, a Facebook Developer account, and a registered Facebook App with basic settings.
You don't have to jump through any of these loops if you're using Instagram Scraper. And you don't have to apply any technical skills unless you want to.
🛠 How does this Instagram web scraper work?
It's built to take a list of Instagram profile URLs, and then search for and scrape page details, posts, or comments from those URLs. By a similar logic, it can scrape Instagram feed guided by one or multiple search queries.
All of the resulting data gets stored in a structured form as a dataset, from which you can download it in formats such as JSON, XML, Excel, CSV, etc. Or extract it using API Endpoints or a Client which turns this scraper into a full-fledged Instagram API of its own.
📝 What data can I extract from Instagram?
It is free to use and will let you download public data from Instagram, namely:
- Profiles
- Hashtags (top 50 results)
- Comments (up to 50 per post)
- User and place searches (via Google Search)
- Likes
To see why we have such precise limitations on hashtags and comments, see the ❓FAQ section below. Here's a more detailed list of what your regular Instagram dataset could look like:
| 🆔 Profile ID, username, full name, and URL | 📝 Bio and external URLs | 📸 Profile picture URLs |
| 📊 Followers and follows count | 📺 IGTV video and highlight reel count | ✅ Verified status |
| 📝 Post ID, caption and hashtags | 🔗 Post URL and shortcode | 🗨️ Comments and likes count |
| ⏰ Timestamp | 👥 Mentions in the post | 🖼️ Image URL |
| 🔵 Business account and category | 🔒 Private account | 👍 Post likes count |
| 📅 Joined recently status | 📡 Has channel status | 👥 Related profiles |
🎞 How to scrape Instagram 101
Let's learn how to use Instagram Scraper to extract data from Instagram profiles and their posts. This step-by-step guide should get you started in just a few minutes.
Step 1. Go to Instagram Scraper
Click the Try for free button to sign up for a free Apify account and start using Instagram Scraper. The free plan gives you $5 of monthly credit, which you can use with this or any other tool on Apify Store.
You can sign up using your email account, Google, or GitHub. After this, you'll be redirected to Apify Console, which is your workspace to run tasks for your scrapers, such as scraping profiles, posts, hashtags, and so on.
Step 2. Add Instagram URLs or search queries
Head over to the Instagram website and pick the profiles you want to scrape. Copy and paste each profile URL into the input. You can add as many profiles as you want using the + Add button, or upload a whole list of them at once using the Bulk edit button nearby.
Then choose Details from the drop-down list of what you want to scrape. Now insert the maximum number of items you want to scrape in the Max items field. We're going to put 200.
You can also choose other parameters of this scraper such as Add metadata, Get tagged posts, Add time frame. You can also set it up to scrape comments and posts, or scrape Instagram feed using search queries. We don't have to set these for now, feel free to explore these in your next run.
Step 3. Click Start ▶️ to run the web scraper
Once you are all set, click the Start button. Your task will change its status to Running 🏃🏻♀️, so just wait for the scraper's run to finish. It will be a minute before you see the status switch to Succeeded ✅ — 10 seconds in our case.
Each username we scraped will come up as a separate result, which is why our output says 3 results. At the end of the run, you will see a preview table with Output results.
Step 4. Export your Instagram data
Now you can move to the Storage tab to view all the results of your scraping. This tab contains your data in lots of versatile formats: HTML table, JSON, CSV, Excel, XML, and RSS feed.
You can open them by clicking on View in another tab, Preview, or Download. You can then share the data, or upload it anywhere you like. You can also clean and preprocess the scraped data before downloading it by removing any irrelevant information.
Step 4. Export your Instagram data
Here, in our example, we only want username, latest posts, follows count, related profiles, and some other details such as bio, profile pic, and URL in our dataset. In order to export only that data, we’ll choose the relevant tags from the list of Selected fields before downloading them.
Step 5. Bonus: integrate your data, export it using API, or set up a scraping schedule
The previous four steps are really all you need to get started for scraping Instagram. But the Apify platform has much more to offer such as scheduling scraping, exporting data using an API or connecting it with integration platforms such as Zapier or Make. Make sure to check out those opportunities as well!
💬 Just want to extract Instagram posts, comments or photos?
You can use the Instagram data scraper from this tutorial to scrape data from Instagram in multiple different ways. However, if you want to scrape specific Instagram data, let's say comments only, you might like our dedicated Instagram web scrapers more.
Compared to the original IG Scraper, there are fewer settings to change and you'll get fast, precise results. As a rule, all you have to do is paste one or more Instagram URLs (or usernames) and click to scrape.
Quick tutorial on how to use one of the Instagram web scrapers
❓ FAQ
What about Instagram data available without login?
There isn't much Instagram data that's publicly available without login.
Search. You used to be able to search for profiles, hashtags, and places, and Instagram would return the top 100 posts. Not anymore. The search feature on Instagram.com is not available directly unless you log in or think of workarounds such as Google Search. Which is why the number of scraped results might turn out in a different order than compared to native Instagram search.
Profiles, hashtags, and locations. When you open any public Instagram page that contains posts (e.g. profile, hashtag or location), Instagram will show you a page with the first few posts preloaded. However, you can not see further than just a couple of posts — they simply won't be displayed. With an exception for profile which still has the See more posts button that has to be clicked every time. And even that will get you so far.
Comments. Every Instagram post used to have publicly visible comments and a Load more comments button for infinite scrolling. That data is not available without login anymore.
Our scrapers do not support logging into Instagram, so there are some limits on what you can and cannot scrape. Our scraper is capable of getting around those limitations (such as using proxies or Google Search results) but only to an extent.
Can I get Instagram data available only after login?
In theory, yes. But this is not what our scraper can do. If you check the Instagram website in an incognito browser window, you’ll quickly find that there's very little data that you can view and access freely without logging in. Most Instagram data is either blocked or requires you to log in, for example: list of followers or list of people a user follows.
Although it would be possible to automatically log in to Instagram to access this data, this approach is risky since it can lead to the banning of your account by Instagram. Sure, you could create a fake Instagram account and use that instead, but that’s beyond the scope of this article and against Instagram's terms and conditions. We don't recommend that you do this and none of our scrapers support it.
Over time, Instagram has been increasingly limiting the data you can access without login, so you’ll need to test to see what you can scrape without logging in.
Can I create my own Instagram scraper on your platform?
Yes, you can build your own Instagram scraper on the Apify platform in Python or JavaScript — either just for deployment or for full production — and take advantage of built-in proxies, infrastructure, scheduling, and storage. If the platform seems too limiting, you can also make good use of our free web scraping library.
Do you need proxies to scrape Instagram?
These days, you generally need proxies to have a successful and generally reliable scraping process. Social media websites often use blocking technologies to prevent scraping. We recommend using residential proxies as the most reliable way to go about scraping anything from Meta in 2024. Fortunately, our free plan offers a free trial of residential proxies, so you can fully test this scraper tool on getting data from Instagram, Facebook or Threads.
Can I use this scraper with API?
This Instagram Scraper is one of many Actors available in Apify Store. As with the other Actors, you can use this scraper directly on the platform with a few clicks, or using an API or scheduler. So basically, you can consider it your own personal Instagram API. You can call this Instagram web scraper using an API Client in Python, JavaScript, and curl. You can also use available API Endpoints to get the scraped Instagram data to where you need it.
Is it legal to scrape Instagram?
Scraping publicly available data is legal, but you need to be careful not to extract content that is protected by copyright or contains personal information. So, after scraping Instagram, double-check your output for data that would go against GDPR, CCPA, or could be considered intellectual property. Read more about the legality of web scraping in our blog post on the subject.
Can I scrape both Instagram users and Threads users?
Since Instagram and Threads share userbase, you can scrape both Threads users and Instagram users since they share the same usernames and profiles. By using scraping techniques, you can extract data from both platforms simultaneously and get insights into user profiles and their activities on both Meta platforms. You may want to check out our Threads Profile Scraper as well.
Natasha Lekh
