How you can web scrape social media to understand the data behind likes and follows. Do followers always translate into revenue? If not, why not?
New technologies have given rise to new marketing practices. Social media has taken on a crucial role in marketing strategy, with everything that the rise of a new approach entails.
Paid advertisement on its own isn’t enough to bring in revenue for brands without a comprehensive strategy to bring in a stable flow of organic followers. New content and content creators sprout every day on every platform, and the competition is fierce. This case study deals with Marc Rutten, a content creator who used data to provide added value to his clients.
Marc vs. the algorithm
Marc had already managed to turn his passion for sports cars into a profitable creative business. But passion alone doesn’t pay the bills, so Marc dug deep into the works of social media platforms and the forces that drive traffic: algorithms.
Simply put, algorithms are sets of rules that determine how data behaves. In the context of social media, this data is user-generated content. How data behaves depends on how users behave, which implies that, as no two users behave exactly alike, no two social media feeds will look exactly the same.
This is done to create the best user experience possible, where users get to see the content they are actually interested in first. The user’s past behavior combined with the content’s and the creator’s existing popularity and adherence to recent trends come together to determine its visibility – which will strongly influence its future popularity.
There are, however, ways to “trick” the algorithm into liking your content just a tiny bit more. For instance, on YouTube, the algorithms sometimes apparently favor longer videos. Instagram has gone from being a photo-dominated platform to a video and reel hub. TikTok is topic-oriented. Hashtags and collaborations contribute to visibility. Staying on top of the latest trends (which can change hundreds of times each year) is crucial – although not always enough.
This makes it hard to start from scratch, and that’s why many creators and retailers resort to buying followers and likes and obsess over their numbers. The problem is that these numbers on their own don’t actually generate any revenue for brands – whether they are the brand’s direct followers or the followers of a content creator or influencer the brand has selected as a brand ambassador.
That’s the kind of situation Marc found himself facing: brands with high follower and like counts wondering why these numbers didn’t match revenue growth. And that’s when he started web scraping.
When social growth and revenue don’t match
So how could it be that these brands’ social media numbers were going up while their sales were going down?
A simple explanation is that their followers – and their followers’ followers – were not interested enough in their products to buy them. They were tapping the "Follow" button with no intention of taking any further step toward a purchase. But how can you gauge follower interest based on numbers?
Besides good ol’ marketing research and customer outreach, there are telltale signs.
When growth is organic, it simply rises steadily and consistently. When it’s not, it rises and drops drastically – although it’s hard to spot. If you look at it once a day, you won’t notice the ups and downs and only see the numbers going up. Only observing the numbers at regular and short intervals can provide actual insights.
Chasing empty numbers is easier than changing your whole marketing strategy, dropping old contacts, and creating new collaborations, but it’s the only way to keep growing in today’s market. Big companies that refuse to evolve may survive for a while longer, but it’s not a sustainable strategy in the long term.
Extracting data 24 times a day manually, however, is downright impossible – especially when the follower count is in the hundreds of thousands, if not millions. Enter Apify’s Instagram Scraper.
Apify’s Instagram Scraper to the rescue
First, Marc hired a developer to automate the data extraction, but this wasn’t yielding many results – web scraping is a demanding and complex process that requires expertise and maintenance. So he did what we all do in time of need: he turned to Google. After searching and comparing web scraping services, he concluded Apify was the best choice for him because:
- Quick results: he couldn’t wait long to get proof and prepare a marketing strategy for the manufacturers who had hired him.
- Great price/quality ratio: it’s useless to go for a cheaper solution if it doesn’t work and then requires you to spend more money to fix the problem or find another one.
- Easy communication: from the moment he got in touch with Apify, the dev he worked with nailed down his needs and always responded fast whenever he needed information or updates.
Instagram Scraper allowed Marc to extract the data he needed every hour automatically, thanks to the convenient scheduling function on the Apify platform. He then downloaded the results into an easy-to-read spreadsheet, identifying unusual spikes in growth.
What’s next? Data needs to lead to change
After collecting and analyzing the data, Marc presents the results to brands, showing them where the glitch is that prevents revenue from matching social media numbers.
And sometimes this is where the ball stops rolling.
The reason for this is that the new type of marketing that social media brought on would require a dramatic reshuffle of strategy and resources, and many well-known companies with high revenue are tempted to rely on their already existing fame for as long as they can.
These old giants keep creating content for themselves instead of creating content tailored for the right audience because they already have a following and underestimate the impact of new marketing. They struggle to grow but are not open to change. It’s easier for them to use old formats and old contacts. Meanwhile, start-ups and influencers grow because they understand the game.