11 best open-source web crawlers and scrapers in 2024
Free software libraries, packages, and SDKs for web crawling? Or is it a web scraper that you need?
Category
Web
Explore articles on web crawling, testing, proxies, and more. Learn from web scraping and automation experts on the Apify blog.
11 Posts
How to inspect elements in Chrome, Safari, Firefox, and more
This guide will equip you with the knowledge you need to use DevTools in multiple browsers.
How to create an RSS feed from a website (step-by-step guide)
What if a website you want to integrate doesn't provide an RSS feed? We have a solution!
How to run Puppeteer in a Docker container
A guide to making automated browser tasks and tests run consistently across different environments.
API for dummies
Types and functions of APIs and their role in web development, data collection, and automation.
How to take website screenshots automatically
A simple step-by-step guide to capturing screenshots or videos of websites in JPEG, MP4, and GIF.
Playwright testing: how to write and run E2E tests properly
E2E tests can be time-consuming and boring. What if we told you there’s a better way?
What is a web ripper? Top 5 website ripper tools (but we’re hiding something)
If you wish to migrate your website to a new server, keep a copy of your site as a backup, or use web data for educational purposes, these powerful site rippers and website ripper tools might be what you need.
What is an online aggregator and how to create your own
What is a content aggregator? What you need to know PLUS how to build one from scratch for social media, news, video, stats, and any other web content.
Are website terms of use enforced?
For contracts that are concluded on the internet, there are four categories to help with assessing their enforceability. Read on to find out what they are and if they can be enforced.
How do proxies affect digital freedom?
Proxies and VPNs can affect internet transparency in many ways. They can unlock geo-restricted content, prevent discrimination, and grant digital freedom under oppressive regimes. Learn more about online openness and censorship.
