Search marketers are facing a growing disconnect between what their data shows and what’s actually happening in the SERPs. Impressions are up. Clicks are down. Pages that once drove consistent results now seem invisible, even if rankings haven't changed.
The root of the issue is AI search interfaces, such as Google AI Mode, Perplexity, and ChatGPT. These interfaces synthesize answers from multiple sources and present users with complete responses. Instead of ten blue links, users see summaries, citations, and follow-up prompts generated by large language models.
For many teams, the only way to investigate these AI search results has been manual checks: entering prompt after prompt into Google AI Mode, ChatGPT, or Perplexity, scanning summaries to see how they’re structured, and noting which sources are cited.
That approach doesn’t scale. It’s slow, inconsistent, and impossible to operationalize, especially when:
- AI features roll out unevenly by region
- Results change by model and interface
- You need structured data, not just screenshots
And the usual SEO toolkit doesn’t help. Rank trackers ignore AI answers, GSC blends AI-driven clicks into overall traffic, and GA4 provides no way to isolate LLM-originated visibility.
The cost problem with AI visibility tools
The limitations of traditional SEO tools become even more apparent when you look at the alternatives. Established tools like Profound — which is built specifically for tracking how brands appear in AI-generated answers across engines like ChatGPT, Perplexity, and Google AI Mode — are positioned as specialized AI visibility platforms rather than conventional rank trackers, and often come with enterprise-oriented pricing.
Other emerging AI visibility solutions similarly charge $99/month and up for even basic monitoring of AI search citations, with more comprehensive plans quickly increasing in price as you scale coverage across engines and queries.
Meanwhile, legacy platforms such as Semrush and Ahrefs — often used for traditional rank tracking — have only recently added AI features, and these are typically bundled into higher-tier subscriptions rather than standalone, LLM-focused analytics.
An affordable way to track AI search results at scale
Web scraping is the only reliable and affordable way to get AI search data at scale. But traditional web scraping methods come with challenges: custom scripts break easily, pagination and retries are tricky, and blocking can stop a project cold.
Apify’s Google Search Results Scraper handles all of that out of the box. And it captures not only classic SERPs, but also Google AI Mode, ChatGPT answers, and Perplexity AI answers.
With that data, you can analyze:
- The AI-generated answers themselves — to understand how models phrase responses, what they emphasize, and what formats they prefer
- The cited sources — to see which brands, pages, or passages are being pulled into AI answers, and which are being ignored
- The model’s query fan-out — additional search queries ChatGPT generates behind the scenes to answer a single question. This retrieval step is normally hidden from users, but it reveals how the model expands, reframes, and explores a topic before producing its final response
And you can get all this data in a single dashboard from a single run.
You don’t have to configure complex settings or write code. You just type your queries, toggle AI Mode, Perplexity, and ChatGPT, and export clean, structured results.
Here’s how to use it to get the data you need.
How to scrape AI search results
Step 1. Go to Google Search Results Scraper (on Apify Store)
Click the Try for free button to sign up for a free Apify account and start using Google Search Results Scraper straight away.
It’s easy to sign up with your GitHub or email account from any provider. You’ll enter Apify Console, a workspace to run or build web scraping tools.
You can test the scraper with $5 of monthly compute usage and no credit card. But note: to use the AI search features, you need to be on a paid plan (starting from $29/month).
Step 2. Fill in the input
Now fill in the input fields. You can configure the input using the form or JSON. Provide keywords in the search terms field. We’ll use best Google Maps scrapers as our search term.
Step 3. Customize your search
Google Search Results Scraper allows you to specify language preferences, result geolocation, and other parameters.
Step 4. Enable AI search: Google AI Mode, Perplexity, or ChatGPT
If you're on any of Apify's paid plans, you can scrape Google AI Mode, Perplexity AI, and ChatGPT. While dedicated GEO tools cost hundreds per month, Apify's Google Search Results Scraper starts at $2 per 1,000 AI Mode results.
queryFanOut, showing additional search queries the model generated to answer your question. This is usually hidden from users.Once you run the scraper, it will generate structured AI-generated responses either on its own or along with your organic search results (depending on how you configured it), with a description, related links, and key information.
Step 5. Run the scraper by clicking Start
Hit Save & Start to save your configuration and run the Actor.
Once the status changes from Running to Succeeded, you can export your dataset.
Step 6. Export your dataset
Now that the run is complete, you can see all the results in 'Overview', or select the specific data you want. For example, you can choose AI Mode results:
Clicking on the URL in the output shows you exactly what users see for the search term, 'best Google Maps scrapers' in Google AI Mode:
You can download your scraped data in many formats by clicking on the storage tab. You can also select specific fields to view or download. For example, let's take a look a the Perplexity search results from this run in XML:
Now, you can export scraped data to CSV, JSON, or Excel and integrate the results with Looker Studio, Power BI, Airtable, or internal systems. This allows you to analyze which brands are being cited, how your domain appears (or doesn’t), and compare this to traditional SERPs.
Frequently asked questions
What is the SEO impact of AI-powered search?
AI search changes how visibility works. Models select passages based on semantic relevance, not just rankings. A lower-ranking page can be cited while a top-ranking page is ignored. Traditional analytics tools don’t isolate this behavior. Scraping AI answers is currently the most reliable way to see what users actually see.
What’s the difference between Google AI Mode, ChatGPT, and Perplexity?
They use different models, retrieval methods, and citation behavior. Google AI Mode is tightly coupled to Google’s index. ChatGPT and Perplexity blend search retrieval with conversational context. Scraping all three lets you understand how visibility shifts across ecosystems.
Is it legal to scrape Google AI Mode data?
Yes. You're allowed to scrape publicly available data for personal or business insights, as long as you comply with local regulations, which you can learn more about in this article on the legality of web scraping.
