Apify Marketplace and 2020 Nobel Prize in Economics — what’s in common?

We calculated the standard deviation of bids placed by devs on Apify Marketplace and were surprised to find price convergence over time.


Supply and demand on Apify Marketplace

Since the launch of Apify Marketplace, the purpose has always been to provide the most cost-effective and frictionless data extraction experience possible. To this end, our team has been hard at work optimizing the process of ordering web scraping and web automation bots. Unlike your generic freelancing platform, Apify is committed to maintaining a pool of expert developers. These certified developers bid on projects and the prospective customer may then choose the offer that suits them best. For everything related to the bids, we decided to let market forces do their thing — unhindered. Over time, we noticed that several projects ended up receiving offers which were either equal or very close to each other. This is despite the fact that:

1. There was no attempt to educate the developers on how to price their bids.

2. The number is left floating up to the smallest decimal.

3. Developers are blinded to each other’s bids. They are only informed when their own bid has been accepted or rejected.

Inspired by the 2020 Nobel prize in Economics, we set out to analyze the bids that the developers placed for all projects since the launch of Apify Marketplace earlier this year. For every project which received two bids or more, we calculated the standard deviation of all the bids and plotted that value versus time. Over the course of a few months, the standard deviation of the bids more than halved (see graph below). That is, bids spontaneously converged towards the market price. And while this phenomenon is bound to emerge in any environment where market forces are unconstrained, the strength of the effect and the speed with which we were able to identify it were surprising. The points on the x-axis (standard deviation equals zero) represent the projects with equal offers. This spontaneous convergence of the bids towards the market price increases consumer surplus. And thanks to the latter, the reliability of the platform, and the world-class service customers receive, about 15% of project requests come from users who already use Apify Marketplace (one of our customers executed 34 projects in a little under 4 months!).

Plot showing that bids are converging over time
Standard deviation of bids versus time

As a moderated online marketplace, we intervene in each project to ensure smooth communication and high data quality. Because of this, our satisfaction rate is close to 100%. Since the beginning of the year, we have processed close to 1,000 project requests. All Apify Marketplace developers are carefully vetted, bound by strict non-disclosure agreements and undergo a thorough onboarding process.

The way Apify Marketplace is able to position itself as the most competitive solution in this space is by leveraging the entire Apify ecosystem in service of anyone looking for custom scraping/automation solutions. Apify provides an SDK that supports state-of-the-art scraping and automation technologies. This SDK is regularly updated to provide tools for bypassing the latest bot mitigation measures. Once developed, the solution is automatically deployed onto Apify’s scalable cloud platform. As the name of the company suggests, interacting with the solution is then possible through dedicated API endpoints (or through a simple interface from a web browser).

Screenshots showing the equal distribution of bids on projects posted on Apify Marketplace.
Screenshots of bids on Apify Marketplace’s projects

If you’re looking to extract value from the web and/or automate tedious tasks, don’t wait! Reach out to us today!

P.S. Shout-out to Francisco Villarreal for helping out with the analysis.

Sapiens Malti
Sapiens Malti
Technical consultant at Apify.

Get started now

Step up your web scraping and automation