Groundbreaking Study Reveals AI Overviews Significantly Depress Organic Clicks, Challenges Google’s Quality Claims

New research from an updated, randomized field experiment indicates that Google’s AI Overviews (AIOs) lead to a substantial decrease in organic website clicks, with the quality of these displaced clicks appearing to be no different from those that persist. This finding directly challenges Google’s narrative that AIOs primarily filter out low-value "bounce clicks," suggesting a broader, more impactful shift in search behavior and potential traffic erosion for content publishers.

A revised working paper by researchers Saharsh Agarwal and Ananya Sen presents compelling evidence of a 39.8% reduction in organic clicks when AI Overviews are displayed. Crucially, the study finds no statistically significant difference in engagement metrics – such as back-button usage, short visits, or time on site – between clicks received when AIOs are present and those that return to websites when AIOs are absent. This suggests that the clicks absorbed by AI Overviews were not inherently of lower quality, contrary to previous assertions by Google.

The implications of this rigorous, data-driven analysis are profound for the ecosystem of online content creation, search engine optimization (SEO), and the future relationship between generative AI in search and the open web. It reignites the debate over how Google’s evolving search interface impacts publishers’ ability to attract users to their sites and monetize their content.

The Unfolding Narrative: A Chronology of Discovery and Debate

The discussion surrounding the impact of AI Overviews on web traffic has been ongoing since their initial rollout, albeit with varying degrees of clarity and empirical data. Google’s introduction of these AI-generated summaries at the top of search results pages (SERPs) was presented as an enhancement designed to provide quick answers and improve user experience. However, concerns from publishers and SEO professionals quickly emerged regarding potential reductions in organic traffic to websites.

Initial Findings and Early Concerns:
The initial iteration of Agarwal and Sen’s working paper, first covered by Search Engine Journal in April, reported a 38% reduction in organic clicks when AI Overviews were shown. This early figure sent ripples through the digital marketing community, quantifying for the first time with a randomized field experiment the scale of potential traffic loss. The study’s methodology, involving controlled exposure to AIOs, provided a more robust basis for analysis than observational studies.

Google’s Counter-Narrative Emerges:
In response to growing anxieties about traffic decline, Google’s Vice President of Search, Liz Reid, offered an explanation centered on click quality. Reid suggested that AI Overviews might reduce "bounce clicks" – instances where users click on a search result only to quickly return to the SERP because the content wasn’t what they were looking for. The implicit argument was that AIOs, by satisfying users directly or by helping them refine their search without visiting a low-quality site, were effectively improving the overall quality of outbound clicks, even if the total volume decreased. This assertion, however, was made without public release of supporting data from Google.

The Revised Study: A Deeper Dive into Click Quality:
The most recent revision of Agarwal and Sen’s paper directly addresses Google’s "bounce click" hypothesis. By incorporating new analyses of click quality, treatment switches, and query types, the researchers sought to understand what kind of clicks were being affected by AIOs. Their updated findings not only solidified the previously reported reduction in clicks (now 39.8%) but critically examined the nature of those clicks. The fact that the revised paper cites Search Engine Journal articles in its discussion of Google’s claims underscores its direct engagement with the industry dialogue.

This chronological progression highlights a critical juncture: the move from anecdotal observations and initial quantified impact to a direct empirical challenge of Google’s official stance on user experience and click quality. The study’s methodology, involving careful measurement and comparison of user behavior under different conditions, provides a robust framework for evaluating these claims.

Supporting Data: Unpacking the Evidence of Impact

The strength of Agarwal and Sen’s research lies in its methodical approach and the detailed data it provides across several key areas. By conducting a randomized field experiment, they were able to isolate the effect of AI Overviews with a high degree of confidence, avoiding many of the confounding variables that can plague observational studies.

Quantifying the Organic Click Reduction

The headline figure remains stark: a 39.8% decrease in organic clicks to external websites when AI Overviews are displayed. This represents a significant portion of traffic that would otherwise flow from Google Search to content creators and businesses. The researchers further clarify how this translates to overall search behavior:

• Outbound Clicks Per Search: When AIOs are present, the number of outbound clicks per search query measurably decreases. This means that for every search performed, users are less likely to click away from Google to an external website.
• Zero-Click Searches: Conversely, the rate of "zero-click searches" – queries where users find their answer directly on the Google SERP without clicking any organic results – rises when AI Overviews are shown. This is a direct indicator that the AIOs are successfully (from Google’s perspective) satisfying user intent directly within the search interface.

Given that AI Overviews were triggered on approximately 41% of all queries during the experiment, the aggregate effect across the entire search landscape is a "measurable reduction in outbound clicks across all searches." This isn’t an isolated phenomenon affecting a niche subset of queries but a systemic shift impacting a substantial portion of search interactions.

The Crucial Test: What the Click Quality Data Reveals

At the heart of Google’s justification for AIOs was the premise that they improve the quality of outbound clicks by filtering out less engaged users. Agarwal and Sen directly tested this hypothesis using three key metrics to assess click quality for queries where an AI Overview would have appeared:

1. Back Button Usage: How often do users click on a search result and then quickly hit the back button to return to the search results page? This is often seen as an indicator of a poor-quality click, where the destination page did not meet the user’s expectations.
2. Short Visits (Ending within 10 seconds): How often does a visit to a destination site end within 10 seconds without any discernible interaction? This metric also suggests a lack of engagement or relevance from the user’s perspective.
3. Total Time Spent on Destination Site: A longer time spent on a site generally correlates with higher user engagement and satisfaction with the content.

The findings from this analysis are unequivocal: none of the three metrics showed a statistically significant difference between the group exposed to AI Overviews and the group where AIOs were suppressed.

• Approximately 4 out of 10 same-tab clicks led back to the results page in both conditions.
• Roughly 18% of visits ended within 10 seconds in both scenarios.
• The time spent on the destination site was statistically indistinguishable between the two groups.

These results directly contradict Google’s assertion. If AI Overviews were indeed absorbing primarily low-value visits, then the additional clicks observed in the "no-Overview" group should have demonstrated worse quality metrics. However, they did not. The authors succinctly summarize this finding, stating that the result is "at odds with the view that AIOs primarily eliminate low-engagement website visits." This powerful conclusion indicates that the clicks being siphoned off by AI Overviews were just as valuable as those that continued to reach external websites.

Behavioral Reversal: The Impact of Treatment Switches

To further bolster their findings and ensure that the observed effects were directly attributable to AI Overviews, the researchers implemented a "treatment switch" after the initial two-week experimental period. This involved rotating the assignments: participants who had previously been exposed to AIOs were now in the control group (no AIOs), and vice versa.

The results of this switch were striking and provided strong causal evidence:

• Participants who began receiving AI Overviews saw their external clicks per search decrease.
• Conversely, those who no longer received AI Overviews experienced an increase in their external clicks per search.
• Interestingly, the zero-click rates mirrored these changes, rising for the group newly exposed to AIOs and falling for the group from which AIOs were removed.

This reversal of behavior, directly correlated with the presence or absence of AI Overviews, acts as a powerful validation of the study’s core premise: AI Overviews are a direct and significant factor in altering user click behavior and reducing traffic to external sites. It removes any lingering doubt that the observed effects could be due to inherent differences in user groups or other external factors.

Impact by Query Type: Where the Losses Are Concentrated

The study further disaggregates its data by query type, revealing that the impact of AI Overviews is not uniform across all searches. This granular analysis provides critical insights for content creators and SEOs looking to understand which areas of their traffic might be most vulnerable.

The most significant losses in outbound clicks are concentrated in informational searches.

• AI Overviews were triggered or intended to trigger on a substantial 53% of informational queries. This high trigger rate directly correlates with the observed reduction in clicks, as users are more likely to find a direct answer within the AIO for these types of searches.
• When a top-of-page Overview was removed for informational queries, the top three ranked organic results gained the most clicks, with the first position nearly doubling its click share. This suggests that for informational queries, AIOs are directly competing with and often displacing clicks that would otherwise go to high-ranking organic results.

In contrast, navigational and transactional queries showed no measurable change in outbound clicks per search.

• AI Overviews were triggered on only 15% of navigational queries (e.g., "Facebook login") and a mere 6% of transactional ones (e.g., "buy running shoes"). This much lower trigger rate explains the minimal impact observed, as users performing these searches typically have a clear destination or intent that an AIO is less likely to fully satisfy.

This breakdown by query type highlights that the battleground for organic traffic in the era of AI Overviews is predominantly in the realm of informational content. Publishers heavily reliant on "how-to" guides, explanatory articles, and detailed information will likely feel the most significant pressure.

Official Responses: Google’s Stance and the Study’s Rebuttal

Google’s official stance regarding the impact of AI Overviews on web traffic has consistently emphasized user experience and the quality of clicks. As previously noted, Liz Reid, Google’s VP of Search, has publicly stated that AI Overviews help reduce "bounce clicks," implying that any reduction in total outbound clicks is offset by an increase in the quality and engagement of the remaining clicks. The underlying message is that Google is providing more relevant and satisfying results, even if it means users spend less time navigating away from the SERP.

However, Google has not released any empirical data to substantiate these claims regarding click quality. This lack of transparency has fueled skepticism and left publishers without concrete information to assess the true impact on their businesses.

The Agarwal and Sen study directly addresses this void. By meticulously testing the very premise of Google’s argument, the researchers provide the data that Google has withheld. Their finding that there is "no measurable difference" in click quality between AIO-exposed and AIO-free conditions serves as a direct rebuttal to Google’s position. The authors’ conclusion that their results are "at odds with the view that AIOs primarily eliminate low-engagement website visits" is not merely an observation but a direct challenge to the official narrative.

This study shifts the burden of proof. While Google can continue to assert that AIOs improve user experience by filtering low-quality clicks, the scientific evidence now available suggests otherwise. For Google to maintain its position convincingly, it would need to present its own robust, peer-reviewed data that demonstrates a statistically significant improvement in click quality or a reduction in genuinely low-value bounces due to AI Overviews. Until then, the findings of Agarwal and Sen stand as a significant counter-argument.

Implications: Navigating the Evolving Search Landscape

The findings of this revised study carry profound implications for a wide array of stakeholders in the digital ecosystem, from individual content creators to large corporations, and fundamentally alter the discourse around the future of search.

For Publishers and Content Creators: A New Reality of Traffic Dynamics

The most immediate and tangible impact falls upon website publishers, particularly those who rely heavily on organic search traffic for audience engagement and revenue generation. A 39.8% reduction in organic clicks, concentrated heavily in informational queries, translates directly into:

• Reduced Ad Revenue: Fewer clicks mean fewer page views, which in turn leads to lower ad impressions and decreased advertising revenue, a critical lifeline for many publishers.
• Lower Affiliate Sales: Content creators who leverage affiliate marketing for product recommendations or informational guides will see fewer users clicking through to merchant sites, impacting their commissions.
• Diminished Brand Exposure: Websites, especially those providing authoritative informational content, may experience a decline in brand visibility and authority as fewer users reach their pages directly from search.
• Rethinking Content Strategy: Publishers may need to pivot their content strategies, perhaps focusing more on transactional or navigational queries where AIO impact is less pronounced, or developing content that is less susceptible to direct summarization. The value proposition of long-form, detailed informational content might need to be re-evaluated.
• The "Open Web" Under Threat: This trend contributes to the broader concern about the "enshittification" of the internet, where platforms like Google increasingly capture value that once flowed to external websites, potentially leading to a less diverse and robust open web.

For SEO Professionals: Adapting to a New Frontier

SEO practitioners face the challenge of adapting their strategies to a search environment increasingly dominated by generative AI.

• Prioritizing Non-Informational Queries: SEOs might advise clients to focus more heavily on optimizing for navigational and transactional keywords, where the impact of AIOs is currently minimal.
• Optimizing for Direct Answers: While AIOs reduce clicks, understanding how they generate answers might inform content creation. Websites still need to be the source of truth, and clear, concise answers could still be referenced, even if not directly clicked.
• Beyond Google Search: Diversifying traffic sources will become even more critical. Social media, email marketing, direct traffic, and other channels will gain renewed importance as Google’s organic pipeline becomes less reliable.
• Monitoring SERP Volatility: The rapid evolution of AI in search means SEOs must remain vigilant, constantly monitoring SERP features and understanding how new implementations affect click-through rates.

For Google: Scrutiny and the Balance of Power

The study puts Google in a more challenging position. While Google’s stated goal is to improve user experience, the data suggests that this improvement might come at a significant cost to the web ecosystem that Google itself relies upon for its content.

• Transparency Demands: The absence of Google’s own data to support its click-quality claims will face renewed scrutiny. There will be increased pressure for Google to provide transparent, peer-reviewed data to justify its AIO implementation.
• Ethical Considerations: The findings raise ethical questions about Google’s responsibility to the open web and the sustainability of content creation. If AIOs systematically reduce traffic without a demonstrable increase in click quality, it suggests that Google is prioritizing its own interface over the health of the broader internet.
• Regulatory Scrutiny: As AI becomes more pervasive, regulators globally are scrutinizing the power of tech giants. Studies like this could fuel arguments for greater oversight or even antitrust actions if Google is perceived to be unfairly leveraging its dominant search position.

Looking Ahead: The Future of AI Overviews and Search

The authors of the study issue a stark warning: aggregate traffic losses could grow if AI Overviews appear on more queries over time. This projection is not speculative but based on the observed trigger rates and their correlation with click reductions. As Google refines its AI models and potentially expands AIO deployment to a wider range of query types, the cumulative impact on organic traffic could become even more severe.

It’s important to remember that this paper is a working draft on SSRN and has not yet completed peer review. While its methodology appears robust and its findings compelling, the academic process of peer review is crucial for independent validation. However, the data presented is sufficiently strong to spark immediate and critical discussion within the digital industry.

The emergence of AI Overviews represents a fundamental shift in how users interact with search engines. This study provides crucial empirical data to understand its consequences, moving beyond speculation to quantifiable impact. For anyone involved in creating, publishing, or monetizing online content, these findings serve as a wake-up call, demanding strategic adaptation and continued vigilance in a rapidly evolving digital landscape.

Featured Image: takasu/Shutterstock