As I delve into the ongoing data battles, I’m struck by how they’re reshaping the AI landscape and the answers we rely on. It’s fascinating to observe the pivotal deals, restrictions, and lawsuits that are creating a fragmented visibility landscape in AI.
This journey through 2023 to 2026 reveals how platform shifts are altering the way data access impacts AI answers. Each step is integral to understanding the changing dynamics of this tech-driven era.
Inspired by this post on HiGoodie Blog.
I’ve just learned that Perplexity AI’s Comet browser agent can no longer make purchases on Amazon. This decision comes after a federal judge ruled in Amazon’s favor, expressing concerns about AI shopping bots.
Why this matters to us. The ruling challenges AI’s ability to simplify tasks, such as online shopping, by acting on our behalf. If similar restrictions are enacted, AI agents might face significant hurdles when trying to access logged-in areas of popular platforms.
The situation as it unfolded. U.S. District Judge Maxine Chesney in San Francisco issued a preliminary injunction, favoring Amazon’s position.
Getting up to speed. Back in November, Amazon filed a lawsuit against Perplexity, accusing it of computer fraud and unauthorized platform access. Allegedly, Comet completed purchases on user accounts without properly identifying itself as a bot.
Next steps. There’s a one-week suspension on the order, giving Perplexity the chance to appeal.
What Amazon says. According to Lara Hendrickson, an Amazon spokesperson, this injunction is crucial for stopping Perplexity’s unauthorized Amazon access and is a vital move towards maintaining trust for customers.
Inspired by this post on Search Engine Land.
When I first learned about SerpApi’s move to dismiss Google’s lawsuit, my immediate thought was about the bold challenge SerpApi is undertaking. They’re arguing that Google is twisting copyright laws to restrict access to public search results all to protect their ad revenue, not copyrights.
The motion to dismiss was officially filed on February 20th, as mentioned in a recent blog post by SerpApi’s CEO, Julien Khaleghy. This legal battle stems from Google’s accusation in December that SerpApi bypassed security measures to scrape and resell content from Google Search.
The details: According to Khaleghy, Google is improperly applying the Digital Millennium Copyright Act (DMCA). Here’s what I found compelling:
The DMCA is meant to protect copyrighted works, not online platforms or advertising ventures. In addition, Google doesn’t actually own the content that appears in its search results, and accessing publicly available pages doesn’t qualify as “circumvention” under this law, SerpApi argues.
Google claims that SerpApi managed to evade bot-detection and crawling controls using rotating bot identities and large networks to scrape licensed content from features such as images and real-time data. However, SerpApi insists that they do not decrypt systems or breach authentication protocols, and merely gather the same data any user could see via a browser, without needing to log in.
Khaleghy also points out Google’s admission that its anti-bot systems primarily secure its advertising interests, which weakens the DMCA claim against SerpApi.
SerpApi references significant legal precedents, including the Ninth Circuit’s hiQ v. LinkedIn, which cautions against monopolizing public data, and the Sixth Circuit’s Impression Products v. Lexmark, reinforcing that public-facing content shouldn’t be blocked by merely technical measures.
Catch up quick: This lawsuit is the latest in a series of escalating legal clashes over data scraping and AI usage:
Back in October 2022, Reddit filed suits against SerpApi, among others, alleging they indirectly scraped content from Google Search. Reddit claims these companies obscured their identities and operated at an “industrial scale.” In turn, SerpApi has vowed to robustly defend itself, emphasizing that public data should remain accessible.
By December, Google further escalated the legal situation by suing SerpApi for ignoring its security measures and attempting to resell protected content. SerpApi stands firm, citing lawful operation and First Amendment rights to access public search data.
By the numbers: If Google’s interpretation of the DMCA holds, SerpApi suggests potential damages could skyrocket to $7.06 trillion — more than the entire U.S. GDP. However, this staggering figure is a theoretical estimate based on potential penalties, not an actual demand.
What’s next: It all boils down to the court’s decision on whether Google’s claims should move forward. Depending on the outcome, this case could significantly impact how SEO platforms, AI tools, and competitive intelligence software access search results data in the future.
A triumph for Google might hinder third-party access to search data, while a victory for SerpApi could reinforce that publicly accessible search outcomes are indeed fair game.
For deeper insights, I recommend reading Google v. SerpApi: We’re filing a Motion to Dismiss. Here’s why we’re in the right.
Don’t miss Inside SearchGuard: How Google detects bots and what the SerpAPI lawsuit reveals for in-depth analysis.
Inspired by this post on Search Engine Land.
I recently explored Google’s SearchGuard, an advanced system that safeguards Google Search from bots. This groundbreaking technology has been thrust into the limelight due to a lawsuit against SerpAPI, revealing how Google differentiates between human users and automated scripts.
After meticulously dissecting the JavaScript code, I gained rare insights into how Google distinguishes humans from automated scrapers in real-time.
What happened: On December 19, Google filed a lawsuit against SerpAPI, accusing them of bypassing SearchGuard to extract copyrighted data from Google Search results on a colossal scale. Instead of focusing on terms-of-service breaches, Google cited DMCA Section 1201, emphasizing anti-circumvention clauses.
This case underscores what Google deems worth protecting, which is crucial for anyone in the SEO and marketing sectors who might be using tools that interact with Google Search.
Why we care: Understanding SearchGuard is vital because any large-scale automation with Google Search invokes this system. If you’re using scraping tools, this is the barrier they encounter.
Here’s where it gets interesting: SerpAPI isn’t just another scraper. OpenAI utilized Google search results, obtained through SerpAPI, to enhance ChatGPT’s capabilities. Although OpenAI’s request for direct access to Google’s index was flatly denied in 2024, they still needed real-time data.
This situation highlights a strategic move by Google, focusing on a key element in the competition’s data supply chain.
In investigating SearchGuard, I fully decrypted version 41 of the BotGuard script, which started with an unexpected greeting:
Anti-spam. Want to say hello? Contact botguard-contact@google.com
Don’t let the friendly tone fool you; behind it lies one of the most complex bot detection systems ever created.
BotGuard vs. SearchGuard: BotGuard, internally termed Web Application Attestation (WAA), shields most Google services. Google’s legal complaint disclosed that the specific system guarding Search is known as SearchGuard, which when implemented in early 2025, disrupted nearly all SERP scrapers.
Unlike traditional CAPTCHAs, BotGuard operates invisibly, seamlessly analyzing user behavior using sophisticated algorithms to separate bots from people.
It leverages a highly protected bytecode virtual machine to ensure it remains impervious to reverse engineering.
How Google knows you’re human: The system evaluates multiple behavioral metrics in real-time, including mouse movements, keyboard rhythm, scroll behavior, and timing jitter, painting a comprehensive picture of a user’s natural interactions.
Google observes the fluidity of mouse motions, capturing deviations that indicate a human touch, unlike the straight paths typical of bots.
A perfectly linear mouse action raises alarms, as it is atypical of human movement, usually characterized by imperfections.
Keyboard rhythm: Everyone types differently. Google captures inter-keystroke intervals, error patterns, and post-punctuation pauses to form a user’s unique typing ‘fingerprint.’
The aspects of natural scrolling and timing jitter are also scrutinized, as context-specific nuances help discern human from machine.
Google’s system even enlists over 100 HTML elements for browser environment fingerprinting to further ensure authenticity.
Performance monitoring: Google captures intricate details such as navigator properties, screen metrics, and engagement with browser APIs for an exhaustive analysis.
Despite efforts to outsmart it, SearchGuard employs cryptographic measures similar to those developed by the NSA to protect its integrity, making circumvention fleeting at best.
The statistical ingenuity behind SearchGuard: Algorithms like Welford’s and reservoir sampling give SearchGuard the upper hand, continuously refreshing a composite profile of expected user behavior.
SerpAPI’s stance: Julien Khaleghy, CEO of SerpAPI, notes Google never reached out before filing the lawsuit, suggesting it’s an attempt to stifle competition from innovative services using their platform to power advanced applications.
Google’s assertiveness poses a monumental challenge to the SEO industry, redefining how anti-scraping measures might be perceived legally. Should SearchGuard be recognized as a legitimate protective measure under DMCA, it could set significant precedent.
Inspired by this post on Search Engine Land.
Today, I came across an intriguing development where Google has initiated legal proceedings against SerpApi. This lawsuit revolves around allegations that SerpApi has been bypassing Google’s security systems to scrape and resell copyrighted content from search results.
The Allegations: According to Google, SerpApi has:
Google’s Stance: Describing SerpApi’s actions as “brazen” and “unlawful,” Google expressed concerns over how stealthy scrapers like SerpApi override crawling directives, stripping sites of their choices. Alarmingly, Google noted a significant increase in SerpApi’s activities over the last year.
Quick Update: Interestingly, Google’s lawsuit mirrors similar legal action by Reddit, which also targeted SerpApi, Perplexity, Oxylabs, and AWMProxy. Reddit accused them of scraping content via Google Search results and concealing their identities to evade restrictions.
SerpApi’s Previous Statements: In defense, SerpApi has maintained that “public search data should be accessible,” viewing its actions as protected by the First Amendment. They also warned that lawsuits like the one from Reddit could endanger the “free and open web.”
Why It Matters to Me: Should Google triumph in this case, acquiring reliable SERP data might become increasingly challenging and costly. This could particularly impact teams reliant on services like SerpApi, as they navigate the complexities of understanding search results, performance metrics, and achieving success in an evolving digital landscape.
Inspired by this post on Search Engine Land.