Apify Blog: Web scraping and automation stories

What is large-scale scraping and how does it work?

When scraping very large or complex websites, a normal scraping workflow won't get the job done. Read on to find out how to overcome the challenges of large-scale scraping.

4 min read May 16, 2022

Amazon

How an Amazon price tracker can help you compete with big business

Theo Vasilis

We show you how to automate Amazon price checks to help you stay ahead of the competition.

5 min read May 12, 2022

SEO

Powerful tool to check broken links on any website

Natasha Lekh

Read on how and why using a free broken links checker can keep your SEO score healthy.

7 min read May 10, 2022

Web Scraping

How web scraping is bridging language barriers

Theo Vasilis

When a research team set out to build a machine translation module for Syrian and Moroccan migrants and refugees, it faced a significant challenge: how do you get enough data for dialects used only in casual conversation? This is how web scraping became the solution.

4 min read May 9, 2022

Engineering

One week in Apify engineering

Kateřina Hroníková

From a Financial Controller to Engineer: read about Kačka's coding journey and what her regular week in the Apify Engineering team looks like.

9 min read May 6, 2022

Actors

Actor build life cycle is being changed

Marek Trunkát

On June 5, we start removing actor builds that are untagged and have not been used for 180 days, and on August 5 we start removing untagged actor builds not used for 90 days.

2 min read May 5, 2022

Life in Apify

How a volcano eruption got me a job at Apify 🌋

Paulo Cesar

Many roads can lead to a career at Apify. Think you don’t have the right background or experience? Think again.

8 min read May 3, 2022

Web Scraping

Is web scraping legal?

Ondra Urban

Web scraping is legal if you scrape data publicly available on the internet. But you should avoid scraping personal data or intellectual property. We cover the confusion surrounding the legality of web scraping and give you tips for compliant and ethical scrapers.

16 min read Apr 28, 2022

Web Scraping

Your step-by-step guide to scraping Amazon

David Barton

Scraping Amazon is a fast and efficient method of extracting useful data in the e-commerce industry. Our simple step-by-step guide will show you how to get ahead of the competition with a powerful Amazon Product Scraper.

3 min read Apr 28, 2022

Tutorial

How to scrape Tripadvisor data with an unofficial API

Theo Vasilis

Using the right API will help you scrape Tripadvisor at scale. You don't need to be a computer wizard to do it. Our step-by-step guide will show you how to extract data from Tripadvisor with a ready-to-use web scraping tool that anyone can use.

4 min read Apr 21, 2022

Research

5 surprising ways to use software for academic research

Theo Vasilis

What are the latest technological trends in the field of academia? What software do academics use? What are the best digital research tools and online research methods for students and scholars? We tackled these questions, and the answers will surprise you.

6 min read Apr 19, 2022

Life in Apify

A Series of Unfortunate Events or What it Takes to Organize an Apify Teambuilding

Karolína Jánošíková, Natasha Lekh, Mariachiara Faraon, Theo Vasilis

Apifiers vs. the bad luck - who will win? Not to spoil the good fun, but nothing can stop an Apifier when teambuilding is at stake. Well, almost nothing ;)

5 min read Apr 7, 2022