Google Places API limits (and how to overcome them using geolocation)

Need to scrape more than 120 places on Google Maps in a large city? What about hundreds of thousands or even millions of places across the whole country or maybe even a continent? Is there a tool able to do that? All is possible with the right Google Maps API, and that's not the official one for sure.

This guide to using Google Maps Scraper 🔗 and Google Maps Data Extractor 🔗 will show you how to overcome the limits of extracting data from Google Maps.

No time for reading today? Video guide to the rescue:

⛔ What are Google Maps limitations for scraping?

Let’s take New York City’s restaurants as an example to showcase how the Google Maps website applies this limit. When browsing the Google Maps website as a user, you might notice a peculiar thing – no matter how many places there are in a city, at some point, you will run out of places that are displayed. The helpful sidebar on the left will say You’ve reached the end of the list.

If you count how many items actually are on this list, you will always end up with 120 at most. The same thing will happen if you try to search even a bigger area (a state, for example) – 120 places displayed max.

But there are definitely more than 120 restaurants in New York City or New York State. This display limitation of Google Maps is there for a reason (better UX), but this is not the limitation you’re expecting when web scraping that data. So how do we overcome it?

To overcome the Google Maps limits of 120 places, we created Google Maps Scraper 🔗. This tool can double as both scraper and an unofficial Google Maps API or geolocation API. It can find or scrape hundreds of thousands of places at a time, leaving no stone unturned. Let’s explore how it works.

🌏 How to overcome Google Places API limits

As it has become increasingly popular, Apify’s Google Maps API has evolved to answer the demand for quality Google Maps scraping in many ways. We now have at least 4 different ways to scrape the same location: by creating Custom geolocation or using Location parameters. But that came with a price: it takes a bit of a learning curve to apply each of these methods. So let's learn them.

1. Create a custom area by using pairs of coordinates 📡

It is a more complex but also the most autonomous way to use Google Maps Scrapers. Defining the area by geolocation will give you full control over the map area, no matter how big or small. Here are a few interesting shapes you can create using the Custom geolocation section of the Google Maps Scraper:

⭕️ Circle

If you don’t care about scraping places in the outskirts, circle is your best option.

This is where we dive into a bit of the geometry-meets-JavaScript part. A circle feature is very useful for scraping places in specific, typically circular and dense areas such as the city center.

First, define the focus of your circle on Google Maps (longitude and latitude):

… then add your circle coordinates into Custom search area field. Don’t forget to pick the kilometer radius of how far your circle should reach.

💠 Polygon

Polygon is perfect for scraping places from custom or unusual shapes (a district, a peninsula or an island, for example). It can have as many points as you need (triangle, square, pentagon, hexagon, etc.)

Do not be intimidated – a polygon uses the same principle for interacting with Google Maps geolocation as the circle. However, it means you will have to add more than one coordinate pair.

To form a free polygon, just pick a few different points on the map (at least three) and take note of their coordinates. You can find any pair of coordinates by clicking anywhere on the map:

You can add as many points as you want to define your area in the most accurate shape possible. We're gonna go with the pentagon shape. Now copypaste each coordinate pair in a sequence into the Custom search area. This is what creating a pentagon for part of Manhattan would look like:

Now click Start ▷ to begin extracting places data. Of course, we could add even more points to this shape to make it more accurate for the island, but you get the gist. You can choose to scrape Google Maps using polygons if your priority is speed, consuming fewer platform credits, or scraping a specific area with unusual geometry.

💠💠 Multipolygon

It is best used for areas that are difficult to fit into one polygon or areas that are quite far away geographically but still have to be scraped together (for instance, an island together with the mainland).

The most complicated and most flexible in custom geolocation is Multipolygon. It is not a scraping option for everyone but it can definitely solve a few headaches.

For instance, this multipolygon would include two polygons since that seems to be the only way to encompass all of Florida. Don't doubt it, even though you as a user see no Google Maps pins displayed on this map, the Google Maps Scraper will find them all.

So you do the same thing as with the polygons but twice or more times. Here's how you paste the coordinates of a multipolygon (by combining two or more polygon shapes) to Google Maps Scraper. The three rules above on how to insert the coordinates apply to multipolygons as well.

2. Choose the location using regular toponomy📍

This method of Google places scraping is a better choice if you don't need to customize the area you're scraping. The area is already defined by its name – a city, an region, a zipcode area or even an entire country.

🌆 Where + What to scrape (search term + city)

Perfect for scraping trials and fast results.

This method is much easier than geolocation settings. Just indicate what you are searching for (restaurants) and where (New York). Then hit Start ▷ and the scraper will do all the work and extract every place that matches your search.

And regarding the search term, 🔍 Search term can be anything. Types of places: restaurants, cafes, vegan museums, gas stations, pubs, bakeries, ATMs – whatever category of place you’re searching for. Names of places: Starbuck's, Zara, Pizzahut. Here are just a few examples of what you can put in the Search term:

Remember that you can also use search terms in languages other than English 😉

📮 Where + What to scrape (search term + extended address)

This method is good for scraping areas by zip code or smaller cities and towns.

For this method, you define the location with the Country, State, US County if applicable, City & Postal code of the area that interests you. In our case, we’re scraping all restaurants in Syracuse by knowing the city specs. The more specs, the better results. And don’t forget to fill out the 🔍 Search term as well.

For the most accurate search, fill out as many fields as you can in this section. The scraper will create map grids and adjust the zoom level in the background; no need to set that up. Just hit the Start ▷ button.

🕵️‍♂️ Understanding how Google Maps Scraper works

If you really want to know the backstory about how this scraper works its magic, prepare for this longread.

Creating map grids 🗺

There’s no magic here, of course. What Google Maps Scraper 🔗 will do is split the large map into many smaller maps. It will then search or scrape each small map separately and combine the end results.

Why would the scraper do this, you ask? Besides, each small map seems empty from this distance (no pins visible 👀), would there be any results at all? To find an answer, we need some more detective work on the Google Maps website.

Automatic zooming on Google Maps 🔎

If you zoom out from the map far enough, you won’t see many places on the map, only city and state names. The further you zoom out, the fewer places (represented by Google Maps pins 📍) you can see, and the lower the zoom level is.

Zoom 1 on Google Maps = the whole planet, 21 = tiny street. You can always check the zoom level in the URL. For instance, to display the whole East Coast, Google Maps uses 5 for zoom level (as seen in the URL). But the more you zoom in, the higher the level is, the more detailed the map is, and the more places it displays.

For our New York City example, compare the number of pins at zoom level 16 (closer) vs. zoom level 15 (further). Can you see how many places are missing in the latter?

This zoom difference would influence your API results as well because - you’ve guessed it – the 120 places limitation applies no matter the zoom level. So how does the scraper deal with this issue?

The scraper will split the NYC map into a number of smaller, more detailed mini-maps. For each map, the zoom level will be adjusted automatically to the most optimal and rich-in-results level (usually 16), so the mini-maps won’t be empty. This way the scraper will be able to find and scrape every single place on a small map. It’s important to note that scraper does allow you to override the standard zoom level and get even more results (but for higher usage of compute units).

Finally, the scraper sums up and patches together the results of all mini-maps. That way we can get accurate map data from the whole city of New York, which makes this method the best for large-scale scraping.

However, this method’s advantage comes at a cost: while being thorough, map grids can be quite slow and consume a lot of credits. There might be very few restaurants on the mini-maps, way less than 120, so why does it take so long to scrape them? The reason is every small map is a separate page the scraper has to open. So that's one extra request for the scraper to do, hence extra work, hence extra consumed credits.

🛠 Extra feature for scraping large areas efficiently

There used to be one core issue when trying to scrape large areas: empty or lesser populated territories. Be it a natural park, a body of water, or even a desert with obviously no restaurants, hospitals, or other specific public places, the scraper would try to find them there anyway – which wastes its resources. The same happened for the outskirts of cities or towns where there are higher chances for large parks, forests, or industrial areas with no pins.

Which is why we've added a special feature for our scraper – 🏙 Deeper city scrape. Enabling it will command the scraper to skip those territories and instead focus on the denser parts of the map. It will scrape only pins and skip deserted or natural areas once it reaches them and simultaneously it focuses on cities and densely populated areas and not the outskirts.

So give it a try next time you need to scrape a truly large area with more than 120 places, and let us know how it worked for you.