I embarked on an SEO audit exploring how platforms like ChatGPT, Claude, and Perplexity leverage technical optimization, content, and conversions to scale their operations.
Generative search engines, such as ChatGPT, have cleverly woven SEO into their growth strategies. Despite claims to the contrary, these platforms have not abandoned this vital marketing channel.
I was curious to learn how well ChatGPT, Perplexity, and Claude are doing in the SEO realm, and what makes ChatGPT’s dedication to this strategy so effective.
ChatGPT’s annual investment in SEO, estimated at $600,000, is yielding significant returns for generative AI platforms. With Semrush data showing ChatGPT’s monthly organic traffic at 76.5 million visits, and with a conservative conversion rate of 0.5% at a $20/month entry price, I foresee a potential annual revenue of around $92 million (a remarkable 15,200% ROI) for ChatGPT.
Both Claude and Perplexity also showcase positive returns, albeit more modestly, ranging from 82% to 240% ROI, highlighting the persuasive potential of SEO investment.
OpenAI has shown great foresight by investing heavily in SEO and content, offering up to $393,000 annually for an SEO-savvy content strategist. This significant investment underscores how seriously OpenAI takes the role of SEO in its growth strategy.
Additionally, they’ve pursued roles centered on growth, SEO, CRO, and web strategy, offering salaries between $410,000 and $600,000 for two essential roles, excluding benefits and other costs. Their commitment to SEO showcases the profound belief in its capacity to act as a cornerstone for expansion.
SEO, a tool as versatile as it is durable, taps into human behavior — a fundamental necessity for survival instincts like searching for food or shelter. By extension, search engines elevate this natural behavior.
The OpenAI team is acutely aware of this evolution and has decisively incorporated SEO into the architecture of ChatGPT.
Inspired by the insights from a competitive keyword analysis via Semrush, I delved into the authority, keyword distribution, and rankings across ChatGPT, Perplexity, and Claude. ChatGPT leads with a formidable authority score of 99, far ahead of Perplexity (81) and Claude (75), setting a benchmark for deriving authority through robust public relations and strategic media visibility.
The journey through the keywords and paid versus organic strategies highlights an under-recognized opportunity: integrating search strategies could optimize conversions and reduce PPC acquisition costs, significantly boosting brand presence.
Gleaning Key Insights:
ChatGPT indexes approximately 287,800 keywords.
Perplexity follows with around 184,800 keywords.
Claude trails with about 36,000 keywords.
ChatGPT capitalizes on user-generated content, while Perplexity and Claude focus on niche, high-intent professional content. However, ChatGPT stands distinguished due to its alignment of strong branding and robust SEO.
Using our agency’s 3Cs SEO and AI optimization framework — code, content strategy, and conversions — I emphasize the importance of optimizing key technical components like the robots.txt file and URL structures that significantly influence search rankings.
In examining content, there’s a considerable gap in SEO optimization on pages from Perplexity and Claude, evident in their oversight of meta titles, descriptions, URLs, and tag optimizations, leading to some not even being indexed by Google.
Leveraging descriptive image names and integrating user-generated content could further bolster search engine performance, as demonstrated by ChatGPT’s steady keyword ranking growth.
Understanding conversions’ role, I see that these platforms seamlessly convert trial users into paying customers by offering trial access before prompting a commitment.
The Road Forward: Optimization remains a never-ending journey. By aligning with OpenAI’s successful model, businesses can bet on SEO as a dynamic component of growth strategies. As the landscape evolves, so should our tactics to ensure visibility and conversion remain at the forefront.
Every week, I join thousands of other media buyers in the same ritual. We open the Meta Ads Manager, eyes scanning the metrics, striving to identify the winning and losing campaigns. A positive ROAS gives us a sense of contentment, while a negative one sends us scrabbling to disable the underperforming asset. This is where many advertisers find themselves trapped in the scoreboard mentality.
By treating metrics as a mere scoreboard, I only see the final outcome, missing the bigger picture that could guide future improvements. It’s like judging a game’s score without considering that my strikers aren’t receiving any passes from the midfield.
If I want to scale performance, it’s crucial to transition from mere reporting to diagnosing. By viewing metrics both as individual KPIs and as parts of an interdependent system, I can uncover the real narrative within my account and make informed optimization decisions.
The Dashboard Illusion
Meta’s interface, with its linear grid format, can sometimes give a false sense of clarity. While one column points at high CPM as an issue, another blames low CTR. In reality, these metrics are often connected, revealing much deeper insights.
A high CPM might not necessarily mean an expensive audience. Instead, it could indicate that my creative isn’t up to par, prompting Meta to charge more due to a subpar user experience.
On the flip side, while a high CTR seems like a win initially, if my CVR is declining, then it’s not really a victory. I find myself paying for high-intent customers that my landing page fails to convert.
The dashboard might tell me what happened, but understanding the system explains why.
A visual of an example of Meta Ads Manager CTR and CPM reporting columns.
To better comprehend the system, I visualize metrics as parts of a sports team. Each player has a unique role. If the team loses, I don’t bench them all. Instead, I review the plays to identify areas for improvement in the next game.
The Scouts: CPM and Reach
CPM acts as feedback from the auction on my total value, combining my bid, estimated action rates, and user value. Together, they play the role of market resonance.
If I notice a spike in CPM compared to historical averages, these metrics hint at an overly crowded market or my creative’s ineffectiveness in maintaining volume.
The Midfielders: CTR and Hook Rate
Their role emphasizes moving the engagement from Meta’s ad placement to my website. A high hook rate but low CTR shows my ad snags attention but falters in driving clicks. It effectively stops the scroll, but people aren’t compelled to click.
The Strikers: CVR and AOV
Representing the final journey step, they depend on my website. A high CTR and low CPC, yet a low ROAS, indicate issues. Although my ad performed well, my landing page or offer didn’t convert the visitors.
The real analysis occurs between the columns displayed in Ads Manager.
Hook vs. Hold Rates
By examining the ratio between hook and hold rates, I can prevent creative fatigue that impacts ROAS.
If my ad has a high hook rate but low hold rate, it captures attention initially but rapidly loses it. This suggests I should enhance the latter part of the ad with a compelling CTA.
If I observe a low hook rate but a high hold rate, most people disengage early, although those who engage tend to convert. This scenario presents a chance to test new hooks that align with the rest of the video, aiming to boost initial engagement and conversions.
Link Clicks vs. Landing Page Views
The discrepancy between these metrics often goes unnoticed. Out of 1,000 clicks, if only 450 landing page views are recorded, there may be a technical issue. It’s essential to check my page speed and ensure my tracking functions properly.
Such a drop isn’t typically due to a creative problem but likely a slow server issue since people expect quick site loading times, and any delay results in bounces, wasting my budget.
CPA vs. Frequency
If increasing CPA is baffling, I should examine the frequency. A rise in both suggests ad fatigue among my audience.
An exhausted audience and system require fresh input, not just increased bids or budgets. I should refresh my creative assets or expand targeting if it’s too narrow.
A visual of an example of Meta Ads Manager reporting columns.
When I encounter an underperforming campaign or creative, I ask myself:
Is volume constant? Have impressions or spend decreased? This might indicate the system devaluing or rejecting my ad, especially the creative component.
Where is the friction occurring? I trace it across hook rate, CTR, and CVR.
Upon identifying the bottleneck, I focus on altering only that variable. Changing multiple elements simultaneously obscures the actual issue. For example, if CVR is low, I focus on the landing page experience, not the ad itself.
Am I directing traffic to a detailed product page while promoting various products in a single creative? It’s crucial to eliminate this friction by creating a product collection landing page, offering an intuitive experience for all interests once they click.
Becoming a Media Architect
With Meta’s AI guiding targeting, my role evolves into a system architect.
While a scoreboard highlights something isn’t winning, a system map unravels the full narrative, such as slow site speeds affecting ROAS or creative appealing to the wrong audience.
Next time I check my account, I’ll resist the urge to immediately glance at the ROAS column. Instead, by focusing on ratios and tracing the user’s journey, I’ll unlock the story from ad to website. Shifting focus from winners to detecting friction points is the key to engineering substantial growth.
I’ve come to realize that AI has dramatically simplified the publishing process, but it also means standing out amidst the noise is increasingly challenging. The good news is, by focusing on clarity, intent alignment, and a few strategic SEO adjustments, we can make significant progress.
As AI breaks down the barriers to production, the web is getting flooded with content that is polished, optimized, but often lacks distinctiveness. When everything seems competent, you and I must strive harder to differentiate our voices.
Though AI has transformed how content is churned out, the core of what users seek—intent—remains unchanged. They sift through headlines and descriptions, rewarding clarity and effectiveness. This is why foundational elements matter even more now.
I find that keeping content fresh isn’t about being novel for novelty’s sake. It’s about diving back into what makes content truly unique: distinct messaging, structured delivery, and a deep grasp of our audience’s needs.
The Real Problem with AI Content
The crux of the issue with AI-generated content isn’t its factualness—it’s its sameness. AI draws from vast pools of existing content, often reproducing unremarkable tropes and conclusions. Individually, they seem fine; collectively, they’re indistinguishable.
This homogeneity is why so much content today feels the same. Even when relevant, it seldom provides a unique reading experience.
Both users and search engines are responding in kind. In a sea of similar content, differentiation becomes key. At this juncture, originality, specificity, and intent alignment have taken on heightened importance.
Ironically enough, AI has increased the value of originality. As automated content inundates the web, signals like clarity, usefulness, and intent alignment become beacons of high-quality content.
Many teams falter here, competing with AI by focusing on quantity over quality. Freshness isn’t about novelty; it’s about crafting content that feels distinctly human and undeniably helpful.
Fresh, Unique Content is Still Built on Classic SEO Principles
Ever since content creation tools evolved, what’s been constant is how people interact with search engines. Users still show up with an issue to solve, skimming through results to pick what seems most relevant.
Despite the rise of AI, this behavior endures.
Page titles, headings, and meta descriptions serve as that crucial first contact with the user. They function almost like ad copy, contrary to assumptions that these elements are becoming obsolete.
Classic SEO principles—clear search intent alignment, descriptive language, organized structure—continue to underpin fresh content.
Although these aren’t groundbreaking ideas, their importance has surged. A tweak in clarity doesn’t just help search engines index a page; it helps users find answers to their questions.
Small SEO Changes Can Lead to a Strong Impact
A recent experiment on my website examined whether more descriptive titles could boost clicks without altering the underlying content. We tested the hypothesis by aligning page titles more closely with search intent and user needs.
The result? A greater alignment led to a substantial increase in click-through rates, proving that small changes can powerfully impact visibility and engagement.
Strategies for Keeping Content Fresh in an AI-Saturated World
Remaining fresh in the AI era isn’t about jumping on every new tool but requires intentionality in creating, positioning, and maintaining content.
1. Treat Intent as Strategy
The essence of SEO has always been search intent, not keyword stuffing. Before crafting content, ask what problem the searcher is trying to address and what a good answer would look like in their context.
2. Use Page Titles and Headlines as Tools
In a crowded SERP, an effective title is crucial to catch a user’s attention and make them click.
3. Refresh Before You Create
Oft-overlooked is the power of improving existing content. You don’t need to produce new content incessantly when updates can achieve better results.
4. Lean into Specificity and Constraints
While AI excels at general advice, human-guided content shines through specificity and context, offering expert insights and breaking down misconceptions.
5. Use AI as an Accelerator
AI should accelerate tasks that don’t require judgment. Editorial responsibilities still lie with us, ensuring content aligns with our goals.
6. Measure Freshness by Behavior
It’s not the volume of content but engagement metrics like time on page and scroll depth that define freshness.
7. Accept that ‘Traditional’ Doesn’t Mean Outdated
Mainstays like clarity, structure, and relevance have only gained importance in our AI-driven landscape.
Why Fresh Content Actually Wins
While AI has revolutionized content speed and accessibility, truly effective content remains appealing and relevant, aligning with users’ search intent and preferences.
Incomplete terminology often results in an incomplete strategy. To bridge this gap, I’m here to offer a clearer framework for optimizing when AI systems both recommend and act.
Search engine optimization (SEO) – be found. Answer engine optimization (AEO) – be the answer. AI engine optimization (AIEO) – be the recommendation. Lastly, assistive agent optimization (AAO) – be chosen when there’s no human in the loop. These are four distinct stages, each absorbing the one before it.
The constant term across the latter two stages is “assistive.” It highlights the purpose: what the system provides the user. The shift happens when “engine” becomes “agent,” marking our industry’s move from systems that recommend to those that act.
For me, this naming debate distracts us from the real work. The SEO industry has splintered across multiple terms that essentially describe the same discipline. Each term has its advocates, and while debating these labels, we aren’t progressing with the actual work.
So, let’s cut to the chase: I’ll lay out why AAO is an effective solution so we can all get back to focusing on our jobs.
Every competing acronym offers partial coverage, none captures it all
Every AI system making recommendations or autonomous decisions—be it Google, Bing, ChatGPT, Perplexity, or Copilot—relies on three components: large language models, knowledge graphs, and traditional search. I refer to these as the algorithmic trinity.
The balance of these elements differs by platform, but the trinity itself remains universal. Even those at Google I’ve conversed with agree on this architectural structure.
SEO has always described the engine’s purpose, which I’ve appreciated. Let’s examine how the competing acronyms align against these three components.
GEO describes the mechanism over intent. It involves the LLM layer, includes search as necessary, but overlooks the knowledge graph entirely. This technology-specific term lacks longevity when the technology advances.
Entity SEO covers the knowledge graph layer but only acknowledges search as a delivery mechanism and LLMs secondarily. It fails the glossary test, often confusing non-specialists.
LLM optimization candidly reveals its scope but neglects the knowledge graph and search components entirely.
AI SEO tacks the term “AI” onto the traditional term, making it accessible to outsiders but lacking durability. As we move to 2026, users are more likely researching rather than searching.
All these terms are incomplete, and it naturally follows that incomplete terminology leads to incomplete strategy. Practitioners tend to optimize only for the part their acronym emphasizes, neglecting others.
Assistive agent optimization (AAO) evolves cleanly from answer engine optimization and encompasses everything required for crafting a comprehensive strategy:
“Assistive” clearly defines the purpose for the entire algorithmic trinity.
“Agent” identifies the actor deploying all three components to reach a decision.
“Optimization” captures what we do.
It’s a stable three-legged stool, ensuring consistency, much like sitting on a stool with evenly matched legs—one that doesn’t wobble.
The glossary test shows AAO isn’t flawless, but it’s our best option
Generative engine optimization, entity SEO, and LLM optimization all require niche understanding, failing the glossary test.
Although “assistive” in AAO isn’t instantly recognizable, “agent” is now a part of popular vocabulary. We see every tech company promoting agents, and “optimization” is self-explanatory. Two out of three terms land smoothly, and the third is easily understood.
If you can propose a more fitting term that perfectly covers the algorithmic trinity and passes the glossary test, I’m open to it. After all, what matters is the discipline, not the terminology.
Importantly, AAO describes a role: optimizing so the assistive agent favors your brand. Roles endure beyond technologies. The right term will endure for years, independent of prevailing model architectures or retrieval methods.
What changes when you adopt the AAO framework
Your brand identity becomes foundational rather than optional. When an agent reviews hotel options, supplier choices, or consultant recommendations, it doesn’t thumb through pages seeking the best title tag. Instead, it assesses the brand: its essence, service, audience, reliability, and confidence in those facts.
This trust originates from the entity home—the page you own that roots everything the algorithmic trinity knows about your brand—and extends through all corroborating sources. If your brand isn’t clearly understood, the agent will select one that is.
The funnel resides within the agent now. The well-trodden acquisition funnel (awareness, consideration, decision) used to bounce users around, with search engines acting as traffic sources. Now, under AAO, this entire journey takes place within AI, without users encountering a list of options. The agent becomes aware of, evaluates, and decides on your brand before presenting the result. Your mission is thus to ensure your brand is the answer when the agent processes its funnel internally.
You might think, “We’re not there yet.” Yes, that’s true for most, but the funnel is already within the assistive engine. With platforms like ChatGPT, Perplexity, Google AI Mode driving users to the perfect click—the pinnacle in AI zeroing in on a single user solution—most tend to accept what’s presented. What’s presently lacking is the agent making the purchase decision.
The web index is no longer the sole source of truth it once was. For two decades, it dominated, but that monopoly is crumbling:
Proprietary datasets feed agents directly, evolving search into what I term ambient research, where in-app pushes surface brand suggestions without a query.
Agents and engines utilize APIs, booking systems, and internal databases that don’t intersect traditional web indices. The index will persist as an essential anchor, but it’s no longer the sole gatekeeper. It’s time we strategize with that understanding.
The push layer is also resurfacing. For years, we depended on search engines to understand our content—rendering JavaScript, deciphering complex pages—and they responded. This passive approach will continue, but proactive methods are gaining ground.
IndexNow, nurtured by Fabrice Canel at Bing, along with MCP and whatever Google deploys next, all facilitate one key function: enabling us to push structured data to action-oriented systems instead of waiting for them to retrieve it. It’s reminiscent of the 1990s, with proactive URL submissions and active ecosystem feeding.
Google’s absence from IndexNow isn’t due to the concept’s flaws—it’s quite ingenious—but perhaps because it wasn’t Google’s brainchild, sparking aspirations for a proprietary adaptation.
We must also consider that JavaScript rendering was Google’s generous favor, not an industry standard. Many AI agent bots don’t process JavaScript, so content reliant on client-side rendering may never be seen by an increasing number of agents.
(This all aligns with the 10-gate DSCRI-ARGDW pipeline, which I’ll detail in the next series segment.)
Your SEO skills remain relevant; the focus shifts from engines to agents.
You don’t need to perfect each intermediary step before embracing AAO, as AAO encompasses AIEO, AIEO encompasses AEO, and AEO encompasses SEO—the skills stack remains, only the focus shifts: aim to be chosen by the agent, recommended during research, and mentioned during inquiries.
Those adopting this perspective will consistently build pipeline confidence while others remain entangled in debates over acronyms, further widening the gap over time.
The discipline now has a name, the agents are already operational, the push layer is in play, and the era of complacency has ended.
The initial two articles explored the “what” and the “why.” Next week, I’ll delve into the “how.” I plan to unveil the 10-gate pipeline I’ve been referring to: DSCRI-ARGDW, a crucial conduit between your content and a conversion by an AI engine.
Discovered: The bot becomes aware of your existence.
Selected: The bot deems your data worthy of retrieval.
Crawled: The bot captures your content.
Rendered: The bot transcribes what it retrieves into a readable form.
Indexed: Content is committed to the algorithm’s system memory.
Annotated: The content undergoes classification across various dimensions.
Recruited: The algorithm leverages your content.
Grounded: The content’s credibility is confirmed against multiple sources.
Displayed: The content is showcased to the user.
Won: The moment of triumph – the engine secures the perfect click.
When I first heard the term “contact page,” my mind immediately envisioned a simple space filled with contact info and a form. However, it turns out that this is a major oversight from a local SEO standpoint. Let me guide you on crafting a contact page that not only elevates your Google prominence but also converts more leads.
Google pays special attention to your contact page
Joel Headley, the former head of Google Business Profile Support, once shared with me that Google actively crawls and interprets your contact page to extract details about your business. This revelation illuminated the common inadequacy of contact pages that simply display a business’s name, address, and phone number (NAP), coupled with a basic contact form.
Google is essentially requesting, “Provide me with your business data,” while you might be responding, “No data for you.” Instead, I encourage you to treat your contact page with the same importance as a multi-location landing page. Here’s what your contact page needs to transform visitors into paying clients:
Business identity.
Contact information.
Trust factors and social proof.
Location-specific content.
Amenities.
Call to action.
1. Business identity
Your contact page should be a reflection of your brand, just like every other page on your site. Here’s what to include:
Your business logo, matching all marketing materials and signage.
Your slogan, with potential keywords for SEO enhancement.
A concise introduction detailing your business’s function, location, and unique value proposition (UVP).
Your contact page isn’t just about providing contact avenues; it should convince visitors of their decision’s wisdom before they reach out.
Clear expectations
Clearly communicate what a customer can expect post-contact to solidify their choice to connect with you:
Expected response times.
Upcoming steps and confirmations from your team.
Additional useful information about your team, location, or differentiators.
Experience and credentials
Boost trust and conversion rates by displaying involvement in:
Industry associations, locally and nationally.
Chamber of commerce groups.
Professional organizations.
Meetup and neighborhood associations.
Better Business Bureau ratings.
Tip: Link association names to your business listing on their sites.
Awards and accomplishments
Include any awards and press mentions, with links to the relevant articles or sites. If there are many, consider a dedicated media section.
Reviews and testimonials
Embed external reviews and include testimonials to enhance trust. Enhance authenticity by showing reviewer photos, names, cities, and profiles.
Your review section is also an excellent place to request additional Google reviews, especially from repeat customers, using a link and call to action.
Review your Google Business Profile’s attributes and list those on your contact page, along with other unique attributes. This specificity aids traditional and AI searches in understanding if you meet distinct needs.
6. A clear CTA button
With a well-structured contact page, a compelling call to action (CTA) is essential. Use vibrant, eye-catching CTAs throughout the page to encourage engagements.
Treat your contact page like a local SEO asset
Your contact page should be seen as a local SEO asset. By investing effort similar to creating a multi-location landing page, you elevate your engagement and conversion rates, surpassing most competitors. Keep this list handy to ensure all necessary sections are covered.
When I’m crafting paid search ads that beat the competition, I always remember to review them in context, not isolation. This helps me understand how my ads stand against others. By doing this, I gain practical insights to enhance messaging, leverage AI effectively, and create PPC copy that truly converts.
How frequently do I analyze my PPC ad copy? I don’t just focus on performance metrics within the ad platform. I make it a point to assess how my ads appear alongside competitor ads, ensuring my message stands out.
Am I using the same messaging as my competitors? What makes my offer unique? I strive to create ads that feature clear calls to action and convincing selling points, avoiding bland and generic content.
Here are several strategies I follow to make my paid search ads stand out and attract customers to my brand.
1. Think about how assets will appear together, not just individually
When I’m working on Responsive Search Ads, it can be tempting to simply fill out all 15 headline options and the four descriptions. But I know that if each headline essentially repeats the same message with minor variations, the ad copy can appear monotonous and repetitive.
To avoid this, I ensure the headlines offer a variety of angles and points of interest. For example, instead of having headlines like “Project Management Software – Project Management Solution – Project Management,” I use options such as “Project Management Software – Trusted by 3 Million Users.”
If I want to experiment with several headlines, I pin them to the same position so the platform can rotate between them without showing similar options simultaneously.
While checking the ad strength rating is common, I focus on the bigger picture instead of just chasing an Excellent score.
I’m more concerned about whether each headline and description accurately reflects my benefit points. Although pinning can negatively impact ad strength, it’s worth it for cleaner messaging.
3. Use AI as a partner, but don’t blindly outsource all your copy to AI
I utilize AI tools from Google and Microsoft to generate text for my ad assets, but I don’t use them without review. These tools provide a starting point, but I always add the human touch to ensure alignment with my brand voice and compliance with industry guidelines.
When I claim to be the “Best Local Contractor,” I provide evidence, such as “Voted Best Local Contractor by [News Outlet].” I use numbers where possible to enhance credibility and reinforce my claims.
5. Highlight ease of effort
I emphasize how my product or service saves time and effort. Whether it’s “Open an account in 10 minutes” or “Schedule a same-day appointment,” I ensure these claims reflect reality to build trust.
To catch potential customers’ attention, I highlight free offerings like “Free trial” or “Free quote.” Such offers encourage prospects to take the next step.
7. Turn off automated assets
Given the possibility for concerns over compliance and accuracy, I disable the setting for automatically generated assets. This ensures the messages and links presented are ones I’ve approved.
8. Highlight pricing where it makes sense for your brand
In scenarios where I can highlight competitive pricing, I do so to help my ad stand out, especially during comparison shopping. When pricing is higher, mentioning it can effectively filter out less suitable prospects.
9. Mention locations in regional campaigns
Mentioning specific locations in my ad copy, like “Now Open in Buckwheat County,” helps create a local alignment, making the ad more relevant to users in that area.
With these strategies in mind, I consistently review and refine my ad copy. I ask myself where I can improve asset combinations, highlight unique value propositions, or better tailor my wording to customer concerns.
In the end, my ad doesn’t just compete in isolation; it competes in the search results alongside others. Understanding this helps me ensure my ad stands out and delivers results.
I often find myself over-crediting Google’s understanding of my web pages. It’s easy to imagine Google as an AI wizard that fully comprehends nuances, expertise, and quality. Yet, during the DOJ antitrust trial, I learned something intriguing.
Google’s VP of Search, Pandu Nayak, testified about a first-stage retrieval system that relies heavily on word matching, rather than any magical AI trick. The foundation is based on older information retrieval techniques, like inverted indexes and postings lists. Okapi BM25, a well-known lexical retrieval algorithm, was cited as a crucial link in Google’s system evolution.
After this initial stage, which is all about word matching, Google employs advanced AI models like BERT on a smaller set of content. These content tools are key to optimizing documents for this stage, yet many use them incorrectly, despite their real value.
In this exploration, I’ll dive into the mechanics of first-stage retrieval, its significance, what content tools actually reveal, and how to effectively use these tools to get noticed by Google without obsessing over perfect scores.
How first-stage retrieval works and why content tools map to it
Understanding BM25 is essential. This retrieval function, crucial to Google’s first-stage system, prioritizes topicality by scanning vast amounts of data quickly, narrowing candidates for further processing.
And for me, as a content creator, certain details stood out.
Term frequency with saturation: At some point, repeating keywords has diminishing returns.
Inverse document frequency: Less common terms score higher, so specificity is rewarded.
Document length normalization: Longer documents can be penalized, as density matters.
The zero-score cliff: Not mentioning a term means zero visibility for related queries.
So, effectively using these tools means identifying gaps in my content and ensuring relevant terms appear. Tools like Surfer SEO and Clearscope guide me in avoiding the zero-score pitfall, offering significant value.
AI enhancements like RankEmbed can assist, but counting on them to fill vocabulary gaps is a gamble. I focus on ensuring my core content is strong at the first retrieval stage.
What the research on content tools actually shows
Research shows a weak-positive correlation between content tool scores and rankings, with studies yielding a 0.10 to 0.32 range. While meaningful, these findings are often derived from studies conducted by vendors using their own tools.
The real test remains: do these tools help a new page climb in rankings? The consistent finding is their efficacy in positioning content for retrieval, not securing high rankings against competitors.
Why not skip these tools altogether?
It’s a mistake to write off these tools, especially since expert writers, myself included, often use overly technical language that audiences may not search for or understand, a classic example of the “curse of knowledge.”
A real-world example is Clearscope helping Algolia align their language with their audience’s searches, ultimately lifting their content’s page ranking significantly.
By showing me what vocabulary is used by successful pages, content tools reduce hours of analysis to minutes, whether I’m a frequent publisher or a solo blogger.
What about AI-powered retrieval?
Dense vector embeddings power AI retrieval but supplement rather than replace word matching due to computational limits. Hybrid systems combining traditional and AI search techniques consistently perform best.
The takeaway for me is clear: AI matters, but traditional retrieval carries significant weight and serves as the foundation of effective content scoring tools.
How to actually use content scoring tools
Common advice tells me to get high scores with tools like Surfer SEO or Clearscope. However, I focus on using them wisely to target the zero-score terms and refine competitor analysis.
Running these tools during research, not during writing, ensures I remain focused on quality and audience relevance rather than just scoring high numbers.
A note on entities
Google’s Knowledge Graph processes the relationships between entities more deeply than most tools measure. Recognizing the gap between flat keyword lists and Google’s more complex understanding helps me focus on providing detailed context.
Retrieval before ranking
Content tools effectively decode retrieval stage vocabulary, a less sensational, but fundamentally honest function. They help me pass the first stage of Google’s pipeline, setting the stage for engaging with more advanced ranking factors later on.
As an SEO professional, Google Search Console is like a trusty sidekick for me. It’s no secret that this free tool from Google provides an in-depth look at how my website performs. It’s like having a pair of X-ray glasses to see through the web’s layers.
With its robust data, I can delve into reports to uncover hidden treasures like clicks, impressions, and Core Web Vitals. It’s like exploring a digital gold mine inside my site.
Search Console’s custom regex filters are my guide through my vast website, ensuring I navigate it seamlessly, page by page.
While I hope to sidestep any SEO-related disasters, especially with Google’s AI advancements, it’s always best to be prepared. That’s why diving into this Search Console guide is essential.
This guide has been crafted for those times when the SEO world becomes unpredictable, much like a thrilling adventure in a post-apocalyptic world.
For instance, as an SEO director, I rely on Search Console daily. It’s my go-to for monitoring content performance, validating technical enhancements, and tracking grows in branded and non-branded queries. It’s integral to my SEO strategy, helping me prioritize tasks with precision.
What does Search Console do? And how does it help SEO?
Search Console stands as Google’s free website analytics and diagnostic platform. It tracks how a site performs in search results, potentially expanding soon into Gemini and AI Mode, offering us what feels closest to first-party search truth.
To set it up, it’s as simple as having a Google account and visiting the website. If profiles aren’t visible, simply verify ownership via a domain or prefix URL.
Domain property is the default recommendation
By default, I prefer setting up a domain property. It offers a holistic overview of my site’s search performance, autonomously including HTTP, HTTPS, www, and non-www versions.
With a verified domain property, I enjoy an uncomplicated setup, often via a DNS TXT record through my hosting provider.
URL prefix property allows you to dissect sections of a site
For more detailed insights, the URL prefix property lets me focus on specific sections like subfolders or subdomains. This is especially handy for producing targeted reports and troubleshooting.
Working with colleagues, such as customer support teams, becomes seamless when I can provide detailed data on specific site sections their work influences.
Key moments in Search Console history
The journey of Search Console has been quite eventful. Launched as Google Webmaster Tools in 2005, it evolved significantly over the years, adding key functionalities like mobile usability reports, security issue improvements, and Core Web Vitals report.
The enhancements continue as we advance into an era increasingly intertwined with AI, making Search Console a dynamic tool for SEO professionals like myself.
Was Google preparing us for AI through Search Console all along?
Reflecting on its evolution, I see a clear narrative. Search Console is transitioning from a mere technical tool into an AI visibility intelligence platform. Google’s approach suggests a future-bound strategy where not just queries but topic clusters define our analysis.
Breakdown of Search Console for SEOs
Within Search Console, I explore various features like URL inspection, search results, Core Web Vitals, and sitemaps, each offering unique insights into the health and performance of my sites.
With advanced tools like regex filters and manual action alerts, Search Console stands as a fortress of data, informing my SEO tactics with precision.
Overview
The Overview section quickly outlines key data sets, setting the stage for deeper dives into performance metrics across my websites.
I interact with LLMs daily, both at work and in my personal projects. For many of us in tech, leveraging these language models has become second nature.
It’s well-known that folks in the tech sector, like me, engage with LLMs at twice the rate of the general population. In my case, LLM usage often exceeds a full day each week.
Even as regular users, we sometimes find ourselves frustrated when an LLM doesn’t quite deliver the responses we expect. Here’s how I effectively communicate with LLMs during vibe coding sessions. These insights are just as valuable when navigating extended interactions with an LLM UI like ChatGPT.
Choosing My Vibe-Coding Environment
Vibe coding is the art of co-creating software with AI. I lay out my vision, the AI generates code, and together we refine it to match my intent. However, the process isn’t always smooth sailing.
The first step in my workflow involves choosing a coding environment. This space serves as a hub for interacting with the LLM, drafting, and executing code. I’m partial to Cursor, having started on their free Hobby plan, but I’ve since upgraded to the Pro+ account due to my extensive usage.
For those interested, here are some environment options:
Cursor: Widely used by vibe coders for its customizable interface.
Windsurf: An alternative that executes terminal commands independently.
Google Antigravity: A unique option favoring agent-driven development.
In my examples, I’ll be using Cursor, but the principles are applicable across platforms. Even if you’re simply delving deep into LLM conversations, the same guidelines apply.
Why Prompting Alone Isn’t Enough
You might ask why we’d even need a tutorial for vibe coding. It’s true—the basic idea is simple: specify an outcome, and the LLM delivers. However, once the complexity increases, especially when dealing with multifile systems or tools, context management becomes crucial.
The context window is a pivotal concept. It’s the memory scope LLMs use to handle input/output data, a window defined by token limits. For example, GPT-5.2 allows a 400,000-token window, while Gemini 3 Pro goes up to 1 million. Understanding this helps in avoiding token overflow, which can diminish retrieval accuracy.
Expert commentator Matt Pocock explains the nuances of context windows well—view his YouTube video for more insight. For now, keep in mind that effective planning minimizes verbosity and assumes clear window management.
One team, one dream. Divide projects into manageable phases, clearing LLM memory regularly between tasks.
Do your own research. While you don’t need exhaustive detail, grasp general methods and potential build paths.
Trust but verify during troubleshooting. Get clarifications from the LLM and cross-check details externally.
Tutorial: Creating an AI Overview Question Extraction System
To produce high-ranking content in AI Overviews, address the questions they respond to. This tutorial guides you in developing a tool to extract such questions, not just to provide a use case but also to demonstrate effective system development via vibe coding. It’s not a guaranteed path to AI prominence but offers strategic insights.
Step 1: Planning
Before diving into Cursor or any other tool, identify your goals and necessary resources. Although it’s early days, using generative AI for initial brainstorming can be beneficial. I often start by articulating my end goal in a sentence or two, alongside requisite steps, in AI tools like Gemini or ChatGPT. Missteps here are okay—this stage is about outlining thoughts, not finalizing builds.
For instance, I could outline:
I’m an SEO, aiming to leverage Google's AI Overviews to inspire our authors' content. We need to extract implicit questions addressed by AI Overviews. Proposed steps include:
1 – Choose a keyword target.
2 – Run a search and collect the AI Overview.
3 – Deploy an LLM to derive underlying questions from the AI Overview.
4 – Preserve questions in an accessible format.
With a clear direction, select your preferred LLM. While I’m partial to Gemini for chats, modern models with robust reasoning will suffice. Initiate a session, state your intent to build an AI Overview extractor, and share your planning prompt.
Step 2: Laying the Foundation
Cursor offers diverse models which I find advantageous. For this task, start in Plan mode, allowing for structured discussions and informed decision-making.
Kick off discussions with our defined project prompt.
Making modifications is crucial, so carefully review the LLM’s plan to ensure alignment with your vision. Address any disparities through collaborative discussions with the model.
Consider seeking insights into possible project failure points and implement preventive measures accordingly. For efficiency, I tend to request models to generate outline files for improved context window management, validating internal consistency before proceeding.
Step 3: The Build
With the foundation laid, shift to Agent mode using your selected model—in my case, Gemini 3 Pro—to execute the building phase. Keep an eye out for required approvals during script execution to ensure a smooth process.
Once script development is complete, proceed with library installations via the provided requirements.txt file. For organized dependency management, setting up a virtual environment is recommended.
Running your first script execution often surfaces unforeseen challenges. Tackle these by leveraging comprehensive diagnostic feedback, ensuring issues are resolved before moving forward.
Troubleshooting and Improvements
My initial run revealed a lack of expected AI Overview detection—a misstep rectified through close inspection of terminal outputs, model adjustments, and informed re-execution.
Embrace troubleshooting as a key growth component in the vibe coding journey, enhancing reliability and performance as you fine-tune system components.
Employ Weave for maintaining organized records of query inputs and LLM outputs. This robust tool aids in both immediate log assessment and long-term query-trace reference.
Use the analyze_query trace to monitor pivotal data points, fostering awareness of the direct connection between query intentions and AI Overview content insights.
Structure Over Vibes: A Strategic Approach
Across my years of vibe coding, I’ve learned structure creates reliability—increasing complexity demands methodical workflows, ensuring sustainable success. Remember to keep the vibes in your collaborations strong, united by a shared purpose and approach.
Stepping into the world of automation has always intrigued me. It brings a level of efficiency that every SEO team craves. Today, AI agents like n8n are revolutionizing how we automate SEO workflows, from data scraping to structured delivery—plus, they have their set of challenges.
What makes n8n particularly captivating is its flexibility and control. Let me walk you through how this platform functions and how it can be harnessed in modern SEO operations.
Understanding How n8n AI Agents are Deployed
Think of modern AI agent platforms as a more intelligent version of Zapier. Platforms like n8n don’t just shuffle data between steps—they interpret, modify, and decide on the next move.
Starting with n8n involves choosing your deployment method: cloud-hosted or self-hosted. While letting n8n host your environment could sound appealing, it has its downsides:
The environment can feel limited.
Customization, like modifying server interactions, becomes difficult.
No community nodes can be installed or utilized.
Costs are usually higher.
But there’s a silver lining:
Less management is required—n8n takes care of updates and patches.
It’s user-friendly with little technical expertise required.
Maintenance stress is reduced significantly.
n8n offers various license packages. The self-hosted option is free, though it poses challenges for larger teams due to limitations in version control and change tracking.
How n8n Workflows Run in Practice
API credentials from providers like Google and OpenAI are necessary to leverage AI models and LLMs. Once installed, n8n’s interface is reminiscent of Zapier—a simple canvas for process design.
You can add nodes and pull data from external sources. Workflows can be triggered via webhooks, schedule, or another system interaction.
The executed workflows transmit outputs to places like Gmail, Microsoft Teams, or HTTP request nodes, triggering further n8n workflows or interacting with external APIs.
Take, for instance, a workflow that scrapes RSS feeds, generating a summarized update. It’s not a full-scale article, but it trims down recap times substantially.
Building AI Agent Workflows in n8n
Within a webhook trigger node, you can generate a webhook URL that Microsoft Teams calls, activating the n8n workflow. It streamlines requests for search news updates directly in a Teams channel.
Once the workflow runs, AI agent nodes communicate with LLMs like those from OpenAI and Google. This opens up numerous possibilities.
Variables from the scraping node, including content from multiple RSS feeds, get transferred to the prompt for summarization. Both user and system prompts guide the AI in processing and formatting this data.
While a single AI node handles summarization, a second node converts this summary into HTML, proving effective for specific tasks where dual AI nodes function best.
The summarized news is delivered through Teams and Gmail, offering a look at efficient workflow execution.
n8n SEO Automations and Other Applications
While I’ve shared a rather straightforward project, n8n’s capabilities extend much further in SEO and digital applications, such as:
Creating full-length, in-depth content.
Crafting meta and Open Graph data snippets.
Analyzing content from a UX perspective.
Developing simple SEO scanners.
And much more!
Inspired by a colleague’s comment, “If I can think it, I can build it,” I ventured into complex systems using n8n to meet the changing needs of SEO.
Drawbacks of n8n
Despite its potential, n8n isn’t without limitations:
Platform immaturity can lead to transaction hiccups during updates.
Resistance might stem from fears about job redundancy or ethics.
The focus should be on supplementing roles, not replacing them.
Its utility is limited in extensive technical audits or large-scale data analysis.
Beginning with repetitive or tedious tasks and automating them might be the key to reducing friction within your team.
SEO’s Shift Toward Automation and Orchestration
AI agents don’t replace human expertise, but they enhance it. They free us from mundane tasks, allowing us to focus on strategic areas, showing the positive shift in SEO toward automation rather than the discipline’s demise.
The evolution of tools may continue, yet the trend toward automation and orchestration is undeniable. Building proficiency in these systems is on the horizon as a vital skill for SEOs.
