Understanding job market trends begins with data, and Indeed is one of the richest sources for job-related information. However, while Indeed offers some APIs, they don’t provide the full picture of job listings or metadata that many companies need.
Scraping the Indeed website directly can quickly give you the information your teams rely on.
In this article, we’ll explore how to scrape Indeed effectively using a web scraping platform designed to make the process simple, scalable, and reliable.
Can I scrape jobs using Indeed API?
Not really. There are several Indeed APIs available. In the past, Indeed used to have an API group designed to scrape Indeed job listings, it was called Publisher Jobs API (Get Job API and Job Search API). However, these APIs have been deprecated. The Job Search API allowed users to access data like job titles, company names, locations, posting times, and job descriptions. Due to these changes, users have been looking for alternatives to Indeed APIs, such as this Indeed web scraper 🔗
Can I scrape applicant info using Indeed API?
To an extent. The available Indeed APIs are useful for tasks like integrating Indeed data into applicant tracking systems, monitoring applicant conversions, or managing interview schedules. They mainly cater to the hiring audience of the platform. Unfortunately, they have a number of limitations and are not suitable for job searching purposes.
Sadly, limitations on website APIs have become very common these days. Websites with huge user bases rarely leave their APIs available and open, even if they did in the past. This is where scraping comes in. Because of the rising trend in API limits, over time we've built hundreds of ready-to-use tools for scraping and automation to keep the public data available for automated extraction. If you’re interested in keeping track of job listings on Indeed, you can try using Indeed Scraper 🔗 to automate the process of collecting and extracting the data.
How does Indeed scraper work?
Indeed Scraper 🔗 works in a way similar to manually searching through the Indeed website and extracting information from each page. But it's way faster!
It starts by going over to the Indeed website and navigating to the specified location. Then, it enters your search query into the search bar and proceeds to scroll down until it reaches the final page. The scraper queues up all the job listings it finds and efficiently copies and organizes all the visible data into a well-structured document. All you have to do is download it.
How can I use the data extracted from Indeed?
| For job seekers | For recruiters |
|---|---|
| 🎖 Extract job listings by titles, benefits, skills, and certifications | 👥 Collect candidate data and automate the applicant management process |
| ⏱ Extract Indeed job openings with relevant positions the moment they appear | 💰 Compare salaries in the industry to determine fair market rates for similar roles |
| 🏡 Find job positions with specific requirements (part-time, WFH, no experience, etc.) | 🔍 Research to support investment decisions for future office space needs |
| 💶 Get salary estimates on job positions posted on Indeed | 📊 Analyze competitors, track hiring trends for skills and technology gaps |
| 📇 Analyze hiring trends, forecast skills, and job role demands | 😊 Assess employee sentiment |
Step-by-step guide to Indeed scraping
Prefer video? You can watch this instead
Step 1. Choose Indeed Scraper
First, go to Indeed Scraper on Apify Store and click the Try for free button.
If you're not signed in to the Apify platform, clicking the button will take you to the signup page. You can sign up using your email account, Google, or GitHub and try the Indeed scraper for free.
Step 2. Find Indeed jobs to scrape
You can now access the scraper’s page in Apify Console. This is where you can set up, run, and download data from scraping. Now let's fill in the required input fields, mainly:
- Positions: e.g. let's look for data analyst jobs on Indeed
- Country: for our example, we’re going with the US
- Location: we’re using San Francisco
- Max items: let's get 100 job listings.
We will also use the Scrape company details toggle to get the hiring companies' info such as name, rating, and number of reviews.
Step 3. Start automating data collection
Once you're all set, click the Start button. You'll notice that your task will change its status to Running. It will be just a minute before you see the status switch to Succeeded.
Step 4. Download Indeed job listings
When the status changes to Succeeded, you can check the results in the Storage tab or by clicking the Export results button.
We've managed to scrape 100 data analysis job postings together with information on the companies that offer these positions. Now we can preview and download this data as Excel, HTML table, JSON, CSV, or XML. We can also share the scraped Indeed data directly via an API.
Step 5. Bonus: Schedule your runs
You can set up the whole scraping job to run automatically every day. First, you have to save your scraping parameters (specific location, job position, number of listings) as a task. This step will make sure the scraper always has the right input which you now don't have to type in yourself. Then click Actions -> Schedule.
Now set up the timing conditions for your scraping task: how frequently it should scrape Indeed, at what time, and in which time zone. All you have left to do is click Create. Last but not least, you can also set up a notification to get an email every time your scheduled Indeed scraping has been completed.
Step 5. Set up job scraping every hour, day, week, or month
FAQ
Can I scrape data from LinkedIn?
Yes. If you don’t want to scrape Indeed job postings and rather get data from another job-hunting platform, you can use some of the other available job listing scrapers. You can scrape reviews on Glassdoor, scrape data from LinkedIn, or freelancers’ offers on Upwork or Fiverr. Just browse our 30+ available job listing scrapers.
Is it legal to scrape Indeed data?
Web scraping is legal. However, different rules set by regulations such as GDPR or CCPA may apply based on where you are in the world. So be careful when scraping personal data (users, resumes, and other sensitive information) and avoid websites that are not publicly available or replicating copyrighted content. Read our web scraping legal article to learn more about the subject.
Can I scrape Indeed with Python?
Yes. To carry out Indeed scraping using Python, you can use Apify API. It gives you programmatic access to the Apify platform: any datasets with scraped data, performance metrics, results, versions, and more. You can access the API using Python via apify-client PyPI package.
Can I scrape Indeed company reviews?
In theory, you can scrape anything available on the website. So Indeed reviews could be scraped. Indeed Scraper doesn't have that functionality yet, but you can request it using the Issues tab. Another way would be to send a request to one of our community devs on Discord; they'll make a custom scraper for Indeed reviews for a fee.
Last but not least, you can build this Indeed Reviews scraper yourself - by using scraper templates or the open-source scraping library Crawlee.
Can I use AI to scrape Indeed?
AI is currently unable to scrape websites directly, but it can help generate code for scraping Indeed if you prompt it with the target elements you want to scrape. Note that the code may not be functional, and website structure and design changes may impact the targeted elements and attributes.
