How to scrape Indeed jobs and company profiles

Indeed's APIs don't cover the job listings available on the site. The good news is that because of its clear categories and customizable parameters, the Indeed website is perfect for web scraping.


Indeed helps people find their dream jobs. Or at least the ones that meet their standards. Indeed has a few of its own APIs, but they don't cover the job listings available on the site. The good news is that because of its clear categories and customizable parameters, the Indeed website is perfect for web scraping. So, let's give scraping Indeed a try using a specialized scraping tool.

Can I scrape jobs using Indeed API?

Not really. There are several Indeed APIs available. In the past, Indeed used to have an API group designed to scrape Indeed job listings, it was called Publisher Jobs API (Get Job API and Job Search API). However, these APIs have been deprecated. The Job Search API allowed users to access data like job titles, company names, locations, posting times, and job descriptions. Due to these changes, users have been looking for alternatives to Indeed APIs, such as this Indeed web scraper πŸ”—

Can I scrape applicant info using Indeed API?

To an extent. The available Indeed APIs are useful for tasks like integrating Indeed data into applicant tracking systems, monitoring applicant conversions, or managing interview schedules. They mainly cater to the hiring audience of the platform. Unfortunately, they have a number of limitations and are not suitable for job searching purposes.

Sadly, limitations on website APIs have become very common these days. Websites with huge user bases rarely leave their APIs available and open, even if they did in the past. This is where scraping comes in. Because of the rising trend in API limits, over time we've built hundreds of ready-to-use tools for scraping and automation to keep the public data available for automated extraction. If you’re interested in keeping track of job listings on Indeed, you can try using Indeed Scraper πŸ”— to automate the process of collecting and extracting the data.

πŸ—‚ How does Indeed scraper work?

Indeed Scraper πŸ”— works in a way similar to manually searching through the Indeed website and extracting information from each page. But it's way faster!

It starts by going over to the Indeed website and navigating to the specified location. Then, it enters your search query into the search bar and proceeds to scroll down until it reaches the final page. The scraper queues up all the job listings it finds and efficiently copies and organizes all the visible data into a well-structured document. All you have to do is download it.

πŸ–‡ How can I use the data extracted from Indeed?

For job seekers For recruiters
πŸŽ– Extract job listings by titles, benefits, skills, and certifications πŸ‘₯ Collect candidate data and automate the applicant management process
⏱ Extract Indeed job openings with relevant positions the moment they appear πŸ’° Compare salaries in the industry to determine fair market rates for similar roles
🏑 Find job positions with specific requirements (part-time, WFH, no experience, etc.) πŸ” Research to support investment decisions for future office space needs
πŸ’Ά Get salary estimates on job positions posted on Indeed πŸ“Š Analyze competitors, track hiring trends for skills and technology gaps
πŸ“‡ Analyze hiring trends, forecast skills, and job role demands 😊 Assess employee sentiment

πŸ“‡ Step-by-step guide to Indeed scraping

Step 1. Find Indeed Scraper

First, go to Indeed Scraper's page, and click the Try for free button.

indeed salary web scrape.png
Step 1. Find Indeed Scraper

If you're not signed in to the Apify platform, clicking the button will take you to the signup page. You can sign up using your email account, Google, or GitHub and try the Indeed scraper for free.

indeed job openings.png
Sign in to the Apify platform. Don't worry, we don't bite 😁

Step 2. Find Indeed jobs to scrape

You can now access the scraper’s page in Apify Console. This is where you can set up, run, and download data from scraping. Now let's fill in the required input fields, mainly:

  • Positions: e.g. let's look for data analyst jobs on Indeed
  • Country: for our example, we’re going with the US
  • Location: we’re using San Francisco
  • Max items: let's get 100 job listings.

We will also use the Scrape company details toggle to get the hiring companies' info such as name, rating, and number of reviews.

indeed job scraping.png
Step 2. Find Indeed jobs to scrape

Step 3. Start scraping Indeed

Once you are all set, click the Start button. You will notice that your task will change its status to Running πŸƒβ€β™‚οΈ. It will be just a minute before you see the status switch to Succeeded βœ…

scrape indeed job postings.png
Step 3. Start scraping Indeed

Step 4. Download Indeed job listings

When the status changes to Succeeded βœ…, you can check the results in the Storage tab.

We've managed to scrape 100 data analysis job postings together with information on the companies that offer these positions. Now we can preview and download this data as Excel, HTML table, JSON, CSV, or XML. We can also share the scraped Indeed data directly via an API.

scrape indeed python.png
Step 4. Download Indeed job listings

Step 5. Bonus: set up job scraping Indeed every hour, day, week, or month

You can set up the whole scraping job to run automatically every day. First, you have to save your scraping parameters (specific location, job position, number of listings) as a task. This step will make sure the scraper always has the right input which you now don't have to type in yourself. Then click Actions -> Schedule.

Now set up the timing conditions for your scraping task: how frequently it should scrape Indeed, at what time, and in which time zone. All you have left to do is click Create. Last but not least, you can also set up a notification to get an email every time your scheduled Indeed scraping has been completed.


Can I scrape data from LinkedIn?

Yes. If you don’t want to scrape Indeed job postings and rather get data from another job-hunting platform, you can use some of the other available job listing scrapers. You can scrape reviews on Glassdoor, scrape data from LinkedIn, or freelancers’ offers on Upwork or Fiverr. Just browse our 30+ available job listing scrapers.

πŸ” Google Jobs Scraper πŸšͺ Glassdoor Scraper
πŸš€ Scraper πŸ’Ό Upwork Scraper
πŸ‘Ή Monster Job Search Scraper 🎨 Fiverr Scraper
πŸ‘” LinkedIn Company URL - Mass Profile Finder πŸ’Ό LinkedIn Company Ads
πŸ“° LinkedIn Posts Scraper πŸ™οΈ Craigslist Scraper
How to scrape LinkedIn companies and profiles
Learn how to scrape LinkedIn for URLs with these scrapers.

How to scrape LinkedIn for company and personal page URLs

Web scraping is legal. However, different rules set by regulations such as GDPR or CCPA may apply based on where you are in the world. So be careful when scraping personal data (users, resumes, and other sensitive information) and avoid websites that are not publicly available or replicating copyrighted content. Read our web scraping legal article to learn more about the subject.

Can I scrape Indeed with Python?

Yes. To carry out Indeed scraping using Python, you can use Apify API. It gives you programmatic access to the Apify platform: any datasets with scraped data, performance metrics, results, versions, and more. You can access the API using Python via apify-client PyPI package.

Can I scrape Indeed company reviews?

In theory, you can scrape anything available on the website. So Indeed reviews could be scraped. Indeed Scraper doesn't have that functionality yet, but you can request it using the Issues tab. Another way would be to send a request to one of our community devs on Discord; they'll make a custom scraper for Indeed reviews for a fee.

Last but not least, you can build this Indeed Reviews scraper yourself - by using scraper templates or the open-source scraping library Crawlee.

Can I use AI to scrape Indeed?

AI is currently unable to scrape websites directly, but it can help generate code for scraping Indeed if you prompt it with the target elements you want to scrape. Note that the code may not be functional, and website structure and design changes may impact the targeted elements and attributes.

Natasha Lekh
Natasha Lekh
Crafting content that charms both readers and Google’s algorithms: readmes, blogs, and SEO secrets.

Get started now

Step up your web scraping and automation