Experiment to Track AI Overviews in Google Search Console

June 18, 2024 at 1:27:26 PM

TL;DR AI overviews are appearing in Google search results, but tracking them in Google Search Console is not possible. Brodie's experiment revealed that URL inclusions in AI overviews are not trackable in GSC. The experiment faced limitations, such as requiring users to be logged in and lack of external data. Results showed a low click-through rate (CTR) and inconsistencies in AI overview appearances. The study concluded that AI overview data is recorded in GSC but tracking remains challenging.

Experiment to Track AI Overviews in Google Search Console

AI overviews have started to appear in Google’s search results, but many aspects of this feature remain unclear to the SEO community. Brodie conducted an experiment to understand how AI overviews are tracked in Google Search Console (GSC).

The Problem

Currently, URLs included in AI overviews cannot be tracked in GSC. There's also uncertainty about the global rollout of AI overviews, which are currently more prevalent in the US.

Experiment Outline

Participants were asked to click a Google search query link and then click on the citation for Brodie's website within the AI overview. Social media posts on Twitter and LinkedIn were used to gather participants, and a VPN was required for non-US participants to trigger AI overviews.

twitter-linkedin-posts-seo-experiment-ai-overviews-google-search-console.webp

Limitations

AI overviews only trigger for logged-in users.
External tools like Semrush and Ahrefs cannot track these inclusions.
Data from logged-in experiences may not be representative of broader user experiences.

Results Interpretation

Data outside the US was excluded, leaving 362 impressions from the US.
The experiment yielded a CTR of 13.5% and an average position of 1.7.
The low CTR suggests potential issues with GSC’s tracking capabilities for AI overviews.
The dataset includes users who have opted into Search Labs, which may affect the results.

only-including-US-data-in-AI-overviews-google-search-console.webp

Results varied among participants, with some not seeing AI overviews at all. This inconsistency makes external tracking challenging.

How GSC Tracking (Likely) Works

There are three core situations for how citations for websites can appear within AI overviews:

Visible Citations: Similar to featured snippets, but with multiple links.
Partially Visible Citations: Impressions are triggered once 'show more' is clicked.
Hidden Citations: Require a click to be visible, leading to higher CTRs.

Findings Summary

Public AI overview data is recorded in GSC, but it's unclear if Search Labs data is included.
Only about half of US users can see AI overviews due to login requirements.
AI overview answers and citations are as inconsistent as featured snippets.
High CTRs in tiers 2 and 3 do not necessarily translate to high traffic.
It's unlikely that AI overview filtering will be included in GSC.

True insights into AI overviews will only be possible once Google extends their visibility beyond logged-in users.