This is the first part in a new series on Web Scraping Case Law that aims to give you the most up-to-date information on how courts worldwide deal with web scraping cases, the judgments they deliver, and how the law on web scraping affects your company.
The legal landscape around web scraping is not always clear and often relies on case law to clarify the interpretation of laws and regulations regarding web scraping. In this new Web Scraping Case Law series, we aim to provide an overview of the most important legal cases for web scraping.
In this first installment, we’ll look at a recent high-profile data scraping case: hiQ Labs, Inc. v. LinkedIn Corp.
Apify COO and ex-lawyer Ondra Urban discusses the hiQ v. LinkedIn case
What was hiQ Labs?
hiQ Labs Inc. was a ‘people analytics’ company that offered workforce insights to businesses through its two products: “Keeper”, which analyzed and predicted the retention risk for the employees and indicated which employees were at greatest risk of being recruited away, and “Skill Mapper” that aggregated and summarized the breadth and depth of the skills possessed by an employer’s workforce by analyzing all of the skills its employees listed in LinkedIn public profile data. hiQ Labs primarily relied on LinkedIn for its data by scraping public LinkedIn profiles using automated web scraping tools.
LinkedIn’s user agreement, which each user needs to accept while creating a new profile, prohibits web scraping. LinkedIn also employs various technical mechanisms to block data scrapers.
Latest Developments
Since the initial rulings in the hiQ Labs v. LinkedIn Corp. case, there have been further legal developments that provide additional clarity on web scraping practices:
District court's judgement of October 27, 2022
- Contractual Obligations: The court held that hiQ breached provisions of the contract between hiQ and LinkedIn that prohibited scraping and the creation of fake accounts. This highlights the importance of carefully reviewing and adhering to website terms of service when engaging in web scraping activities.
- LinkedIn's Actions Deemed Lawful: The court decided that LinkedIn's actions, which severely damaged hiQ's client relationships and eventually led to the closure of their business, were not unlawful. LinkedIn's actions were considered justified as they were taken to protect its rights in anticipation of litigation.
- Sanctions for Not Preserving Evidence: The court awarded sanctions for failure to preserve evidence according to the law, emphasizing the importance of proper evidence handling in legal disputes involving web scraping.
Impact on web scraping
- Previous Landmark Decision Remains Valid: The Ninth Circuit's opinion from April 18, 2022, which confirmed that one cannot be criminally liable for scraping publicly available data under the Computer Fraud and Abuse Act (CFAA), remains intact. This continues to be a significant precedent for the legality of web scraping publicly accessible information.
- Contractual Considerations: The latest judgment serves as a reminder that, while scraping public data may not be criminally punishable, it can still violate contractual agreements. Web scrapers should be particularly cautious when accessing data behind logins or where they have agreed to terms of service that prohibit scraping.
- Ongoing Legal Discussions: The court's decision not to deliberate further on the legality and constitutionality of selectively preventing access to public platforms suggests that these aspects of web scraping law remain open for future legal interpretation and potential challenges.
These latest developments underscore the complex and evolving nature of web scraping law. While the practice of scraping publicly available data has gained some legal clarity, the importance of considering contractual obligations and the potential for civil liabilities remains critical for organizations engaged in web scraping activities.
Preliminary injunction
LinkedIn started developing a new tool called “Talent Insights”, very similar to hiQ’s “Skill Mapper”. Having been aware of hiQ’s business for some time, LinkedIn recognized it as a direct competitor. Shortly after the start of the development of “Talent Insights”, LinkedIn Corp. sent hiQ Labs a cease-and-desist letter asserting that hiQ’s unauthorized scraping of LinkedIn’s profiles violates the law, restricted hiQ’s profile and access to the LinkedIn website, and threatened to sue.
In response to the cease and desist letter, hiQ Labs demanded that LinkedIn recognize its right to access public LinkedIn pages and stop blocking hiQ’s access. LinkedIn refused. Consequently, hiQ filed a lawsuit against LinkedIn, seeking a preliminary injunction – temporary and quick relief before the court decides on the whole dispute. As a result of the preliminary injunction granted to hiQ by the court, LinkedIn Corp. was ordered to reinstate hiQ’s access to public profiles and avoid restricting its access in any way.
Despite the preliminary injunction that was in place to ensure hiQ may continue its business activities, the uncertainty lingering over its business led to losing funding, clients, and employees, and eventually closing down its operations.
Litigation process
From a procedural standpoint, the litigation process might seem a bit untraditional. The longest part eventually leading to the US Supreme Court and bringing the important precedents was the proceedings about the preliminary injunction. HiQ sought a preliminary injunction to prevent LinkedIn from denying hiQ Labs access to publicly available data on public LinkedIn users’ profiles. The courts granted and upheld the injunction until hiQ’s business operations ceased. Following the cessation of hiQ’s business activities, there was no more need for the protective preliminary injunction as there was no longer any ongoing business activity requiring its protection. The court thus revoked the injunction.
After that followed a summary judgment proceedings. Summary judgment usually constitutes an initial phase of litigation where the most clear legal questions are answered. In the hiQ v LinkedIn case, some legal questions were already conclusively answered during the preliminary injunction phase, some were indeed answered summarily, and the rest remained open.
Ordinarily, the next (and typically longest) phase would be the main proceedings. However, this phase never occurred in the hiQ Labs v. LinkedIn Corp. case, as the parties settled the rest of the dispute out of court.
Criminal liability: Computer Fraud and Abuse Act
Before this important precedent was created, it was a common claim that web scraping violates the Computer Fraud and Abuse Act (CFAA), which was originally explicitly designed to combat federal computer fraud. CFAA is an anti-hacking act that criminalizes accessing protected computers without having the authorization to do so. The interpretation of “without authorization” or “exceeding authorization” has been the subject of debate ever since.
During the hiQ Labs, Inc. v. LinkedIn Corp proceedings, the Supreme Court in another case – Van Buren v United States – upheld the narrow interpretation of the law and held that the CFAA’s “exceeds authorized access” provision covers those who obtain information from computer networks or databases to which their computer access does not extend. It does not cover those who, like Van Buren, have improper motives for obtaining information that is otherwise available to them. Following the Van Buren decision, the Supreme Court, upon LinkedIn’s petition, ordered that the Court of Appeal (Ninth Circuit) reconsider its decision in light of this new judgment. This has allowed the Ninth Circuit to provide the conclusive precedent that scraping publicly available data is incapable of violating the Computer Fraud and Abuse Act.
The Ninth Circuit built upon the Supreme Court’s argumentation from Van Buren, where it used the “gates-up-or-down inquiry” for access to a protected computer - meaning that if authorization is required and has been given, the gates are up; if authorization is required and has not been given, the gates are down. The Ninth Circuit pointed out that a defining feature of public websites is their lack of limitations on access; therefore, using the gate analogy - there were no gates to lift or lower in the first place. In other words, where there is no authorization required in the first place, there is nothing to withdraw from later. The CFAA concept of “without authorization” simply does not apply to public websites.
Contractual liability: LinkedIn’s user agreement
In the summary judgment, the district court decided that, notwithstanding the above, hiQ was not allowed to scrape LinkedIn, not by law, but by contract. hiQ Labs concluded a contract with LinkedIn Corp. by accepting LinkedIn’s user agreement. The court held that hiQ breached provisions of the terms of use (which hiQ accepted and created a valid contract between the parties), prohibiting scraping and the creation of fake accounts.
Unfair competition
In the summary judgment decision, the court also held that LinkedIn’s actions, which severely damaged hiQ’s client relationships and eventually crushed their business, were not unlawful. Normally, such actions might be considered an unfair competition, which is prohibited by law. In LinkedIn’s case, those actions were lawful as they fell within the scope of California law governing litigation privilege. Such privilege justifies otherwise unlawful actions provided that they are taken to protect one’s own rights in anticipation of litigation.
HiQ and LinkedIn settle out of court
While the court’s judgments are published, an out-of-court settlement is made between the parties and the contents of it are typically confidential. It is, therefore, not publicly known how hiQ Labs and LinkedIn Corp. settled the dispute. Most likely, they built upon the court decisions and opinions presented in the judgments and agreed on the consequences. This is not an unusual approach, as court proceedings are very lengthy and costly.
Publicly available data not criminally punishable
The most important outcome for web scraping is the Ninth Circuit judgment ruling out the possibility of scraping publicly available data being criminally punishable.
Less groundbreaking, but a good reminder, is the contractual effects of user agreements. When web scraping, you need to consider whether you are subject to a binding user agreement. That often (though not always - read more about the enforceability of terms and conditions) happens when one signs in to a website and is asked to read and accept the website’s terms and conditions. The terms can include limitations or even prohibit web scraping altogether. It is essential to be extra vigilant when scraping behind a login.
Final thoughts on hiQ Labs v. LinkedIn Corp.
The hiQ Labs v. LinkedIn Corp. case offers some critical insights into the evolving legal landscape around web scraping. It sets a precedent that scraping publicly available data does not fall under computer fraud, as laid out by the CFAA. This is significant for anyone involved in web scraping. If you’re thinking about scraping a public website, the gates are essentially up, legally speaking.
However, it’s crucial not to overlook the contractual obligations that may come into play. LinkedIn was able to assert its rights through its user agreement, which hiQ had to accept to use the service, thereby rendering their actions unauthorized data scraping. So, while you might be in the clear from a criminal standpoint, make sure you’re not stepping on any contractual landmines.
Additionally, it’s worth noting that actions taken in anticipation of litigation, like what LinkedIn did to hiQ, can be legally justified. This reminds us that while the law provides a framework, each case is unique and can be influenced by various legal mechanisms.
Finally, the case ended in an out-of-court settlement, which, while not publicly disclosed, likely considered the various legal complexities that were brought to light during the court proceedings.
The hiQ v. LinkedIn case is a landmark in web scraping case law. It brings clarity to some extent but also leaves room for interpretation and further legal developments. If you’re a business relying on web scraping for data, keeping an eye on cases like this can provide valuable takeaways for navigating the complex and often murky waters of web scraping law.
We’ll return with another installment in our Web Scraping Case Law series soon.
