Content
Indeed helps people find their dream jobs. Or at least the ones that meet their standards. Indeed has a few of its own APIs, but they don't cover the job listings available on the site. The good news is that because of its clear categories and customizable parameters, the Indeed website is perfect for web scraping. So, let's give scraping Indeed a try using a specialized scraping tool.
Can I scrape jobs using Indeed API?
Not really. There are several Indeed APIs available. In the past, Indeed used to have an API group designed to scrape Indeed job listings, it was called Publisher Jobs API (Get Job API and Job Search API). However, these APIs have been deprecated. The Job Search API allowed users to access data like job titles, company names, locations, posting times, and job descriptions. Due to these changes, users have been looking for alternatives to Indeed APIs, such as this Indeed web scraper 🔗
Can I scrape applicant info using Indeed API?
To an extent. The available Indeed APIs are useful for tasks like integrating Indeed data into applicant tracking systems, monitoring applicant conversions, or managing interview schedules. They mainly cater to the hiring audience of the platform. Unfortunately, they have a number of limitations and are not suitable for job searching purposes.
Sadly, limitations on website APIs have become very common these days. Websites with huge user bases rarely leave their APIs available and open, even if they did in the past. This is where scraping comes in. Because of the rising trend in API limits, over time we've built hundreds of ready-to-use tools for scraping and automation to keep the public data available for automated extraction. If you’re interested in keeping track of job listings on Indeed, you can try using Indeed Scraper 🔗 to automate the process of collecting and extracting the data.
🗂 How does Indeed scraper work?
Indeed Scraper 🔗 works in a way similar to manually searching through the Indeed website and extracting information from each page. But it's way faster!
It starts by going over to the Indeed website and navigating to the specified location. Then, it enters your search query into the search bar and proceeds to scroll down until it reaches the final page. The scraper queues up all the job listings it finds and efficiently copies and organizes all the visible data into a well-structured document. All you have to do is download it.
🖇 How can I use the data extracted from Indeed?
| For job seekers | For recruiters |
|---|---|
| 🎖 Extract job listings by titles, benefits, skills, and certifications | 👥 Collect candidate data and automate the applicant management process |
| ⏱ Extract Indeed job openings with relevant positions the moment they appear | 💰 Compare salaries in the industry to determine fair market rates for similar roles |
| 🏡 Find job positions with specific requirements (part-time, WFH, no experience, etc.) | 🔍 Research to support investment decisions for future office space needs |
| 💶 Get salary estimates on job positions posted on Indeed | 📊 Analyze competitors, track hiring trends for skills and technology gaps |
| 📇 Analyze hiring trends, forecast skills, and job role demands | 😊 Assess employee sentiment |
📇 Step-by-step guide to Indeed scraping
Step 1. Find Indeed Scraper
First, go to Indeed Scraper's page, and click the Try for free button.
If you're not signed in to the Apify platform, clicking the button will take you to the signup page. You can sign up using your email account, Google, or GitHub and try the Indeed scraper for free.
Step 2. Find Indeed jobs to scrape
You can now access the scraper’s page in Apify Console. This is where you can set up, run, and download data from scraping. Now let's fill in the required input fields, mainly:
- Positions: e.g. let's look for data analyst jobs on Indeed
- Country: for our example, we’re going with the US
- Location: we’re using San Francisco
- Max items: let's get 100 job listings.
We will also use the Scrape company details toggle to get the hiring companies' info such as name, rating, and number of reviews.
Step 3. Start scraping Indeed
Once you are all set, click the Start button. You will notice that your task will change its status to Running 🏃♂️. It will be just a minute before you see the status switch to Succeeded ✅
Step 4. Download Indeed job listings
When the status changes to Succeeded ✅, you can check the results in the Storage tab.
We've managed to scrape 100 data analysis job postings together with information on the companies that offer these positions. Now we can preview and download this data as Excel, HTML table, JSON, CSV, or XML. We can also share the scraped Indeed data directly via an API.
Step 5. Bonus: set up job scraping Indeed every hour, day, week, or month
You can set up the whole scraping job to run automatically every day. First, you have to save your scraping parameters (specific location, job position, number of listings) as a task. This step will make sure the scraper always has the right input which you now don't have to type in yourself. Then click Actions -> Schedule.
Now set up the timing conditions for your scraping task: how frequently it should scrape Indeed, at what time, and in which time zone. All you have left to do is click Create. Last but not least, you can also set up a notification to get an email every time your scheduled Indeed scraping has been completed.
Step 5. Set up job scraping every hour, day, week, or month
❓ FAQ
Can I scrape data from LinkedIn?
Yes. If you don’t want to scrape Indeed job postings and rather get data from another job-hunting platform, you can use some of the other available job listing scrapers. You can scrape reviews on Glassdoor, scrape data from LinkedIn, or freelancers’ offers on Upwork or Fiverr. Just browse our 30+ available job listing scrapers.
Mariachiara Faraon
Is it legal to scrape Indeed data?
Web scraping is legal. However, different rules set by regulations such as GDPR or CCPA may apply based on where you are in the world. So be careful when scraping personal data (users, resumes, and other sensitive information) and avoid websites that are not publicly available or replicating copyrighted content. Read our web scraping legal article to learn more about the subject.
Can I scrape Indeed with Python?
Yes. To carry out Indeed scraping using Python, you can use Apify API. It gives you programmatic access to the Apify platform: any datasets with scraped data, performance metrics, results, versions, and more. You can access the API using Python via apify-client PyPI package.
Can I scrape Indeed company reviews?
In theory, you can scrape anything available on the website. So Indeed reviews could be scraped. Indeed Scraper doesn't have that functionality yet, but you can request it using the Issues tab. Another way would be to send a request to one of our community devs on Discord; they'll make a custom scraper for Indeed reviews for a fee.
Last but not least, you can build this Indeed Reviews scraper yourself - by using scraper templates or the open-source scraping library Crawlee.
Can I use AI to scrape Indeed?
AI is currently unable to scrape websites directly, but it can help generate code for scraping Indeed if you prompt it with the target elements you want to scrape. Note that the code may not be functional, and website structure and design changes may impact the targeted elements and attributes.
