How to set up a content change watchdog for any website in 5 minutes

Natasha Lekh
Natasha Lekh
Table of Contents

Use our actor to regularly check specific content on any website and receive an email notification whenever it changes.

Some websites, typically in the area of e-commerce, provide a feature called a watchdog that you can use to get a notification when something changes (usually a product price). This feature is really handy -  you don’t need to remember to regularly open the same page to see if something changed. Our Content Checker lets you set up such a watchdog for any website and get an email notification whenever the content you want to monitor changes.

Just think about how often you’ve checked the same page every evening to see if tickets are already available or on sale, a product price is below what you want to pay, your competitor changed their homepage, or there’s a new version of a library your work depends on. This is the issue that Content Checker is created to solve and we’ll show you how to set up this watchdog to keep you in the loop before everyone else.

Applications of a watchdog program

Content Checker is one of the most popular actors on the Apify platform. The reason for this is simple: it allows you to monitor specific content on any web page and sends a notification with before and after screenshots whenever that content changes. This allows automating monitoring anything from prices to availability which can be useful both for business and personal use.

This makes the Content Checker a truly universal solution because you can use it to:

Basically, the scraper enables you to automate tracking changes in any content that you want to keep an eye on. It can be literally any parameter that gets updated - that’s the best part of the idea behind web automation. You don’t have to spend time doing things manually if you can get a bot to do it!

How Content Checker works

The logic behind the website checker is simple as well: it searches for specific text in a chosen CSS selector and compares the current results to the original state. If there is any change, Content Checker launches another actor to send an email or a Slack notification and also makes and sends screenshots before and after the change occurred on the website.

The caveat here is that this Apify actor is a bit more on the technical side than others from our Store. It can only scrape fixed selectors, and you have to know some basic JavaScript to be able to fine-tune it for your needs. But don’t let this discourage you - we’ve got a step-by-step tutorial to help you out!

Step-by-step guide on how to set up your own website watchdog program

1. First, go to the Content Checker page on Apify Store. Scroll down or explore tabs to get familiar with its parameters and possibilities. The Readme and Input tabs will tell you everything you should know about how to set up this scraper to fit your unique needs. Then click the Try for free button.


2. Sign in using your email account, Google, or GitHub.


3. You’ll be redirected to the Apify Console with Content Checker ready to be set up. Apify Console is the main tab where we’ll be working from now on.


4. But let’s set aside the scraper for a little while. Now we need to find a website to scrape. It could be any web page of your choice, but the (slightly) tricky JavaScript part is that you have to know which specific part of the website you want to scrape and which command to use for that. Some popular website elements (selectors) deal with the price changing, update when a new entry shows up, such as new clothes, technology, news, etc.

5. In this example, we’ll be scraping a website full of event announcements. We want to scrape it from time to time and get notified every time there’s a new jazz concert announcement in our area (New York). So we go to the upper Searcher bar, set up the City, Genres, Sort by sections and click GO. It is crucial to have your search sorted by Newly Announced since we want only the latest, freshest concert announcements.


6. Now just copy the URL that you’ve got from your search. Go to the Apify Console tab with the Content Checker page open, and paste the copied link into the first field - URL to check.


7. The next step is finding the right selector on the concert website. In our case, we need the very first box that gets refreshed with a new entry when a new concert is added. So right-click on the page, choose Inspect and target the first box. You’ll see what it’s called in the code. In our example, this element is called event_row. This is all we need to know from this inspection. Back to Apify Console.


8. Now type into the .event_row:nth-of-type(1) into both selector fields. This combination will let the Content Checker know which selector to look for and check when scraping that concert web page. You can read more about the nth-of-type (1) command in our SDK library. Combined with the URL we put above, with this command, we're simply telling the scraper to only watch for the first box that appears at the top of the concert’s webpage - only the newly announced concerts.


9. Fill in your email address in the corresponding box, add the message you want to receive when getting an email from your scraper and click the Save and run button. If the actor shows the status Succeeded after the run - it’s all ready.


The interesting thing is that most likely, your scraping run will show you zero results. However, collecting results is not the point of this scraper - its purpose is only to notify you if a change occurred. So the moment changes on the website happen, you will receive a notification, and get handy before and after screenshots in your email inbox.



10. Last but not least - you can schedule your scraper to check the website every now and then on a regular basis. You can set that up in the Schedules tab. You can tell Content Checker to check on your chosen selector hourly, daily, and so on - and can even customize the schedule based on different time zones.


And that’s it -  you’re all set. The pattern of the website checker will run every day, scrape the content of your selector, compare it with the previous run and send you a notification if the content changes (along with the previous and current content and screenshots).

Our APIs - both the free and the paid ones - can also be used in conjunction with each other, so if you prefer getting Slack notifications rather than emails - you can pair the Content Checker scraper with a Slack notification one.

We have another scraper that can be customized in a similar way -  Web Scraper. It is even more adjustable and flexible than Content Checker, so if you feel like challenging yourself with scraper customization, here’s a detailed tutorial of how to set it up according to your unique scraping needs.

And that’s everything you need to know to get started using this actor. Be sure to check out other actors in Apify Store - they're just as great, although most are definitely easier to adapt. If you come up with any interesting ways to use Content Checker, we’ll be happy to hear about it from you on Twitter :)

Great! Next, complete checkout for full access to Apify
Welcome back! You've successfully signed in
You've successfully subscribed to Apify
Success! Your account is fully activated, you now have access to all content
Success! Your billing info has been updated
Your billing was not updated