I’m seeing product feeds become far more important in ChatGPT Shopping, especially as AI systems look for clean, structured product information they can trust and cite.
Product detail pages still matter, but I no longer think brands can rely on PDPs alone when ChatGPT searches for product information. The signals that power AI shopping results appear to come from a broader mix of feeds, product data, availability, pricing, and clear brand-owned content.
After looking at what more than 1 million ChatGPT shopping offers revealed, I’d treat product feeds as a core visibility asset, not just a backend ecommerce requirement. If my feed data is incomplete, inconsistent, or hard to match to the product page, I’m making it harder for AI shopping systems to understand and recommend my products.
For brands, the takeaway is clear: I need to strengthen both my product feeds and my PDPs. The better my product data is structured, aligned, and easy to verify, the better chance I have of being cited higher in AI Shopping experiences like ChatGPT.
I used to think of search as retrieval. I would open tabs, compare sources, read reviews, cross-check details, and then make the decision myself.
Now I see search becoming something different: delegation.
More users are realizing they do not need to compare 15 pages or jump between Google, Maps, reviews, forums, and videos before they act. They can ask AI to do much of that work for them.
In many ways, this is the closest most people have come to having a personal assistant. For a long time, delegation was a luxury. It usually meant having someone else research options, summarize information, and make recommendations. In practice, that kind of help was mostly available to people with money or support teams around them.
Now that capability is much more widely available. I believe that changes search behavior at a fundamental level. Users increasingly want synthesis instead of retrieval, recommendations instead of endless exploration, and reduced effort instead of exhaustive research. They want help evaluating options and making decisions.
This is a real behavioral shift. Where people once might have phoned a friend, they now ask an LLM.
Why I believe users are delegating more decisions
At the heart of this move from search to delegation is basic human psychology. Our brains are wired for cognitive ease. We naturally gravitate toward behaviors that reduce effort, simplify decisions, and save time.
AI tools fit that pattern perfectly. They remove friction from the decision-making process by helping users open fewer tabs, make fewer comparisons, carry less cognitive load, and reach outcomes faster.
I also see users becoming more comfortable with answers that are good enough and delivered quickly, rather than perfect answers that require a lot of effort to uncover.
For years, search behavior was built around gathering as much information as possible before making a decision. AI has changed that value exchange. Users do not always need every possible answer. They need confidence that the answer in front of them is sufficient.
Reflect Digital’s SearchPulse research found that up to 61% of AI users say they use these tools because of their speed and ease. Disclosure: I am Reflect Digital’s founder and CEO.
As technology has become part of everyday life, our expectations around convenience have evolved with it. We are already conditioned to optimize more of our lives than ever before, and AI is becoming another mechanism for doing exactly that.
Why delegation in search will not look the same for everyone
One of the biggest mistakes I think businesses can make right now is assuming this shift to delegation is happening evenly across all audiences and all search journeys. It is not.
People also delegate differently depending on the task they are trying to complete. Vacation planning is a useful example. Building an itinerary is an ideal delegation task because it traditionally requires maps, travel sites, timing decisions, logistics, and constant comparison.
Now, a user can ask AI something like: "Plan me a five-day itinerary around Tuscany with wine tasting, scenic towns, and minimal driving." That is decision outsourcing in action.
But choosing the vacation itself may still involve more exploration. A person may still want to browse destinations, look at imagery, watch videos, or validate ideas independently before narrowing the options.
The key point is that delegation is contextual. I believe businesses need to understand where delegation naturally fits within their audience’s decision-making process.
How I identify delegation opportunities in an audience
The important thing to understand is that delegation is rarely universal across an entire customer journey. AI adoption is not binary. People delegate specific types of decisions at specific moments.
I look for delegation opportunities in moments where users experience high cognitive load, too many variables, time pressure, repetitive comparison, decision fatigue, or information overload.
These are the moments where delegation becomes appealing. To understand what that means for a specific audience, I ask where they get overwhelmed, where they compare too many options, where they are trying to save time, and where they repeatedly ask for reassurance or recommendations.
I also look for the parts of the journey that feel effort-heavy rather than emotionally enjoyable. The more effort a task requires, the more likely delegation becomes.
Then I compare those answers with the areas where users may still want exploration, such as inspiration, entertainment, identity expression, aspirational browsing, and emotionally led decisions.
For example, a user may delegate the work of building a travel itinerary but still enjoy exploring vacation destinations on their own.
That distinction matters. The businesses that win in this new search environment will understand not only what their audience is searching for, but also what they are trying to offload.
Once I start looking for delegation-driven decisions, they become surprisingly easy to spot. They often appear when users ask AI to narrow down options, recommend the best fit, validate a choice, summarize information, compare alternatives, or reduce effort.
That means searches start to sound more like: "What’s best for me?" "What would you recommend?" "Compare these options." "Give me the top three." Or, "Summarize this for me."
Traditional search behavior, by contrast, is more exploration-heavy. It involves deeper comparisons, source checking, manual research, and detailed information gathering.
Most users will move between these two modes depending on what they are searching for and why. But I do not think businesses should rely only on internal assumptions or gut instinct to understand where those delegation moments exist.
Gut instinct only goes so far. To understand this shift properly, I believe businesses need to speak directly with their audience and combine behavioral observation with research such as surveys, customer interviews, roundtables, usability testing, journey analysis, search behavior analysis, and AI prompt analysis.
The goal is to understand where users experience friction, feel overwhelmed, seek reassurance, want recommendations, and feel comfortable outsourcing decision-making.
The real competitive advantage comes from understanding what your audience no longer wants to do themselves.
This is where the shift becomes commercially important. I believe businesses now need both search-support content and decision-support content because both behaviors still exist.
Search-support content is designed for exploration. It is usually comprehensive, detailed, comparison-driven, educational, and deeply indexable. It helps users who still want to research extensively and validate decisions themselves.
Decision-support content serves a different purpose. It needs to be synthesized, recommendation-oriented, clearly structured, trust-heavy, and outcome-led.
This kind of content helps both users and AI systems quickly understand what a business offers, who it is for, when it is appropriate, and why it should be trusted.
For example, a traditional search-support page might compare every CRM platform feature in detail. A decision-support page might clearly explain the best CRM for a 50-person B2B sales team with limited implementation resources.
One page supports exploration. The other reduces decision-making effort.
Websites increasingly need to support two parallel journeys: humans who are exploring and humans who are delegating. Put another way, they need to support journeys for both people and AI agents.
How I audit content for delegation behavior
If delegation is becoming part of an audience’s decision-making process, the next question is simple: does the content support it?
I usually start by auditing existing content through two lenses: exploration support and decision support.
First, I ask whether the content helps someone explore. This is traditional search-support behavior. It includes detailed explanations, comparisons, educational depth, broad keyword coverage, manual research support, and multiple options without strong direction.
That type of content helps users gather information and evaluate independently.
Then I ask whether the content helps someone decide. Decision-support content reduces effort by offering clear recommendations, summarized takeaways, structured comparisons, strong trust signals, direct answers, contextual guidance, and outcome-focused language.
One of the easiest ways I spot gaps is by asking: "If an AI system landed on this page, would it clearly understand what we recommend, who this is for, and why it matters?"
Many businesses currently have a lot of exploration content but very little decision-support content. That creates a gap. Delegation is no longer only about being discoverable. It is about being usable within a decision-making process.
Some businesses are already making the mistake of abandoning traditional search behavior too early. I think that is a serious error because traditional search is not disappearing.
At the same time, delegation behavior cannot be ignored. Different audiences, moments, and decision types now require different search experiences.
The businesses that succeed will not be the ones chasing every AI trend. They will be the ones that deeply understand when users want exploration, when users want delegation, and how to support both effectively.
That matters because users increasingly seek help evaluating options and making decisions.
The brands that succeed in the future of search will be those that truly understand their audience and let that knowledge guide their strategy.
If it feels like the whole internet woke up and decided every sentence needed to start with “AI,” I get it. I feel that fatigue too.
As marketers, we are getting hit every day with LinkedIn hot takes, rushed prompt hacks, and promises that ChatGPT will either 10x our productivity or replace us completely.
And right in the middle of all of that is the digital marketer trying to figure out whether AI is just another buzzword cycle or the start of a major rewrite of how we handle content, SEO, PPC, reporting, and almost everything else.
So I want to break it down in plain English.
Think of this as my AI starting guide for marketers who are tired of needing someone younger to translate every new acronym, the same way many of us once had to help our parents get online or open an AOL chat window.
Defining AI and LLMs, and why they matter
I am not asking “what is AI” just to chase keyword density. I want to start with a shared definition, because a lot of these terms get used interchangeably, and not always correctly.
At its core, artificial intelligence refers to machines performing tasks that usually require human intelligence, such as understanding language, recognizing patterns, making decisions, and generating content.
The kind of AI getting the most attention right now is generative AI: models that can create text, images, code, video, and other outputs based on patterns learned from huge datasets.
Tools like ChatGPT, Gemini, and Claude do not “think” the way people do. They predict the next most likely word, phrase, or response based on what they have been trained on.
That matters because AI is not a magic shortcut to instant wealth, overnight automation, or effortless headcount reduction. I see it more as large-scale data aggregation and pattern recognition.
Large language models, or LLMs, are not creating net-new truth from nothing. They process massive amounts of existing information and produce answers based on patterns, probabilities, and what looks like internet consensus.
For content creators and marketers, that is a major shift. I am no longer thinking only about optimizing for a traditional search engine click. I also have to think about whether machines can understand, summarize, cite, and reuse my content.
The biggest implication is the rise of zero-click search. AI systems can answer users directly through experiences like Google AI Overviews or ChatGPT responses, often without sending that user to the original website.
That changes SEO from a pure traffic game into an authority, visibility, and data-ingestion game.
That is why I think marketers need to understand what AI does well, what it struggles with, and where it actually belongs in a broader marketing strategy.
AI jargon I think marketers need to know
Before going deeper, I want to separate a few terms that often get mashed together: AI, machine learning, NLP, generative AI, LLMs, and AI agents. They are related, but they are not the same thing.
Understanding the difference helps me make better decisions about which tools to use, where to trust them, and where human judgment still matters most.
Artificial intelligence (AI)
Artificial intelligence is the broad umbrella term for machines performing tasks that usually require human intelligence. That includes problem-solving, learning, speech recognition, language understanding, and decision-making.
In marketing and search, AI has become a catch-all phrase. But in practice, most of the tools I use fall into more specific categories.
Example of AI: Siri and Google Assistant use AI to interpret voice commands and respond in context.
Machine learning (ML)
Machine learning is a subset of AI. Instead of giving a system explicit instructions for every possible situation, we feed it data so it can identify patterns and make predictions.
In marketing, machine learning powers ad targeting, customer segmentation, recommendations, predictive analytics, and plenty of optimization systems we already rely on.
Example of machine learning: Netflix uses machine learning to recommend shows based on viewing history.
Natural language processing (NLP)
Natural language processing helps machines understand, interpret, and generate human language.
NLP is why ChatGPT can carry on a conversation and why Google can understand that “cheap running shoes” and “affordable sneakers” are closely related searches.
Example of natural language processing: Google Translate uses NLP to understand and convert language in real time.
Generative AI
When people casually say “AI,” they often mean generative AI, which is a branch of artificial intelligence that creates content instead of only analyzing existing data.
Generative AI models are trained on massive datasets to learn patterns in language, images, audio, code, or video. Then they use those patterns to produce something new.
But I always remind myself that these systems are still predicting likely outputs. They are not thinking, reasoning, or understanding the world like a person.
That is also why generative AI can go off track. When a model confidently makes something up, we call it a hallucination.
Some of the most infamous hallucination examples include AI answers suggesting people eat small rocks or use glue to keep cheese on pizza. Funny in hindsight, but a serious reminder that fact-checking is not optional.
Large language models are a specialized type of generative AI trained on huge amounts of text, including books, websites, code, and other online sources, to generate human-like responses.
I think of LLMs as the engine behind many chatbot experiences. They are the part that interprets what I type and produces a response.
When I use an LLM effectively, I do not treat it like a replacement for my brain. I give it context, examples, constraints, and direction. It can help refine a draft, suggest wording, or organize messy thoughts, but I still own the strategy and final judgment.
In short, LLMs react to input. They do not act independently unless they are connected to tools and workflows that let them take action.
GPT models from OpenAI, used in ChatGPT.
Claude models from Anthropic.
LLaMA models from Meta.
AI agents
AI agents go beyond responding to prompts. They can work through multi-step tasks, use tools, navigate websites, fill out forms, call APIs, analyze files, and complete workflows with less hand-holding.
They are still powered by LLMs under the hood, but the key difference is that they have goals, tools, and a degree of autonomy.
That is why AI agents feel more consequential for marketers. They are not just talking; they are beginning to do the work.
ChatGPT can search the web, analyze files, and review code.
Google Gemini in Workspace can summarize email threads and suggest replies.
Microsoft Copilot can assist across Microsoft 365 workflows.
How I see AI affecting marketing today
Once the terminology is clearer, the marketing impact becomes easier to see. AI is changing how people search, how content is produced, how visibility is measured, and how stakeholders talk about growth.
People have been saying SEO is dying for years. I do not think SEO is dead, but I do think “SEO is changing” undersells the size of the shift.
We are in the middle of a major industry pivot, and AI is at the center of it.
Organic traffic is being cannibalized
AI Overviews are Google’s automated summaries that appear at the top of some search results, often pulling from multiple sources.
I think of them like Featured Snippets turned up several notches. They do not simply quote one source and send the click back. They blend sources, rewrite information in Google’s voice, and may push attribution lower on the page.
For broad informational queries, that means the first thing a user sees may be Google’s answer instead of my blue link. The likely result is a lower click-through rate and fewer visits to publisher and brand websites.
Before AI Overviews, informational queries were often useful for introducing a brand early in the research journey. Now, more of that attention and trust can stay with Google.
Claim: AI Overviews only appear for fluffy queries, so my traffic is safe.
Reality: Google is testing and expanding AI Overviews across more serious query types, including YMYL, product, and B2B searches.
What I would do next: Stop chasing every possible click, measure visibility and influence alongside conversions, and build enough topical authority that my brand becomes a source AI systems can confidently cite.
Content creation is exploding, and so is the noise
Generative AI has removed one of content marketing’s biggest bottlenecks: production time. Work that used to take a team a month can now be drafted by one marketer in a week.
That is not automatically bad. The problem is that when everyone can publish “good enough” content quickly, the internet gets louder and less useful.
Claim: More content means more traffic.
Reality: That was already questionable before AI. Now, search systems are increasingly tuned to reduce the visibility of generic, low-value, quickly produced content.
Google’s Helpful Content updates, Bing’s spam improvements, and social platform feed changes all point in the same direction: thin content is easier to produce, but it is also easier to ignore.
What I would do next: Focus on authority-driven content such as case studies, original data, expert analysis, and proprietary insights. I would publish less, promote more, and use AI for research, outlining, repurposing, and refreshing instead of simply flooding the web.
Search results are becoming deeply personalized
Traditional SEO has dealt with personalization for years through local results, logged-in history, and device context. LLM-powered tools like ChatGPT, Perplexity, and Gemini take that much further.
The same question can produce different answers depending on the user, their prompt, their past interactions, available data, and the model being used.
For example, if someone asks, “What is the outlook for Tesla?” a financial analyst may get an answer focused on stock performance and filings, while a new driver may see information about models, battery life, and charging infrastructure.
A Semrush Enterprise AI source analysis view tracks how often domains are cited in ChatGPT results, revealing shifting visibility trends across LinkedIn, Wikipedia, Semrush and industry publishers.
Claim: I will just optimize for the top answer in ChatGPT the way I optimize for position one in Google.
Reality: The idea of one universal top answer is breaking down. Personalization makes it harder to define, track, and reverse-engineer a single ranking position.
What I would do next: Track visibility across search engines and LLMs, build a recognizable brand entity, invest in multiple content formats, use structured data, and create clear, citable answers that machines can understand.
Attribution is breaking
When Google, Bing, Perplexity, or another AI-driven platform answers a question directly, users may never visit the website that influenced the answer. Even when they do visit, their journey may start in an AI tool, move through another search, and only later reach the site.
That breaks the clean channel → click → conversion model marketers have relied on for years.
Claim: I will measure traffic from LLMs directly in analytics.
Reality: That assumes users are clicking through from AI answers. In many cases, they are not.
A Semrush-style AI sentiment dashboard visualizes how brands appear across AI search, with mention trends, sentiment mix, and a competitive leaderboard.
What I would do next: Move beyond last-click attribution, pay more attention to assisted conversions, and track broader demand signals such as direct traffic, branded search volume, brand mentions, sentiment, and “How did you hear about us?” responses.
I would also budget for influence that is hard to perfectly track, including podcasts, PR, thought leadership, community visibility, and media coverage.
Clients and bosses expect magic
Because AI hype is everywhere, stakeholders often expect it to make everything faster, cheaper, and better without understanding the risks, learning curve, or human oversight involved.
Claim: We can replace our SEO or content team with AI tools and get the same results.
Reality: AI can accelerate tasks, but it does not replace strategy, judgment, subject-matter expertise, or a real understanding of customer needs.
What I would do next: Set expectations early. AI can make some work faster and cheaper, but it is not a push-button strategy. I would show stakeholders the hidden work behind good AI output, including prompt refinement, editing, fact-checking, compliance, and final review.
The best use of AI is not to remove human thinking. It is to free up more human time for the strategic work that actually moves the business forward.
Search is evolving
I am not interested in getting stuck in a debate over Generative Engine Optimization (GEO), Answer Engine Optimization (AEO), or any other acronym. The important point is simpler: search today is not what it was yesterday.
Organic visibility is no longer only about ranking in Google. Search now includes AI answers, YouTube, Reddit, newsletters, communities, social platforms, and every place people go to discover, compare, and validate information.
If I am only thinking about the traditional search bar, I am already behind. The better path is to build authority, create content worth citing, understand how AI systems interpret information, and measure visibility across the full discovery journey.
AI is not the end of SEO. It is a major shift in how search works, how content is consumed, and how brands earn trust. The marketers who adapt will be the ones who separate useful strategy from the noise.
I have watched the debate around llms.txt become one of the most polarized conversations in web optimization.
Some people treat llms.txt as essential infrastructure for AI discovery. Others, especially longtime SEO practitioners, see it as speculative theater. Platform tools are starting to flag missing llms.txt files as site issues, yet server logs still show that AI crawlers rarely request them.
Google even appeared to adopt it. Sort of. In December, Google added llms.txt files across many developer and documentation sites.
At first, the signal looked obvious to me: if the company behind the sitemap standard was implementing llms.txt, maybe the file really mattered.
Then Google removed it from its Search developer docs within 24 hours.
Google’s John Mueller said the change came from a sitewide CMS update that many content teams didn’t realize was happening. When asked why the files still exist on other Google properties, Mueller said they aren’t “findable by default because they’re not at the top-level” and “it’s safe to assume they’re there for other purposes,” not discovery.
The llms.txt research
I wanted data, not another debate.
So I tracked llms.txt adoption across 10 sites in finance, B2B SaaS, ecommerce, insurance, and pet care. I looked at the 90 days before implementation and the 90 days after.
I measured AI crawl frequency, traffic from ChatGPT, Claude, Perplexity, and Gemini, and the other changes each site made during the same window.
Here is what I found:
Two of the 10 sites saw AI traffic increases of 12.5% and 25%, but llms.txt was not the cause.
Eight sites saw no measurable change.
One site declined by 19.7%.
The 2 ‘success’ stories weren’t about the file
The Neobank: 25% growth
One digital banking platform implemented llms.txt early in Q3 2025. Ninety days later, its AI traffic was up 25%.
That sounds compelling until I looked at what else happened during the same period.
The company ran a PR campaign around its banking license and earned coverage in major national publications.
It restructured product pages with extractable comparison tables for interest rates, fees, and minimums.
It published 12 new FAQ pages optimized for extraction.
It rebuilt its resource center with new banking information and concepts.
It fixed technical SEO issues, including header structure problems.
When a company earns Bloomberg coverage in the same month it launches optimized content and fixes crawl errors, I cannot isolate llms.txt as the growth driver.
The B2B SaaS platform: 12.5% growth
A workflow automation company saw AI traffic jump 12.5% two weeks after implementing llms.txt.
The timing looked perfect. It would be easy to call the case closed. But the surrounding context told a different story.
Three weeks earlier, the company had published 27 downloadable AI templates covering project management frameworks, financial models, and workflow planners. These were functional tools, not ordinary content marketing assets, and they drove the engagement behind the spike.
Google organic traffic to those templates rose 18% during the same period and kept climbing throughout the 90 days I measured.
Search engines and AI models surfaced the templates because they solved real problems and created an entirely new site section. They did not surface them simply because the URLs appeared in an llms.txt file.
The 8 sites where nothing happened after uploading llms.txt
Eight sites saw no measurable change after adding llms.txt. One of them declined by 19.7%.
The decline came from an insurance site that implemented llms.txt in early September. Based on the data, the drop likely had nothing to do with the file.
The same pattern appeared across all traffic channels. Llms.txt did not prevent the decline, and it did not create any visible advantage.
The other seven sites, which included ecommerce brands in pet supplies, home goods, and fashion, plus B2B SaaS, finance, and pet care sites, used llms.txt to document their best existing content. That content included product pages, case studies, API docs, and buying guides.
Ninety days later, nothing changed. Traffic stayed flat. Crawl frequency was identical. The content was already indexed and discoverable, and the file did not change that.
The pattern was clear: sites that launched new, functional content saw gains. Sites that only documented existing content saw no gains.
Why the disconnect?
No major LLM provider has officially committed to parsing llms.txt. Not OpenAI. Not Anthropic. Not Google. Not Meta.
“None of the AI services have said they’re using llms.txt, and you can tell when you look at your server logs that they don’t even check for it.”
That is the reality I saw in the data. The file exists. The advocacy exists. But platform adoption does not show meaningful use yet.
The token efficiency argument and its limits
The strongest case for llms.txt is efficiency. Markdown can save time and tokens when AI agents parse documentation. It gives agents clean structure instead of forcing them through complex HTML, navigation, ads, and JavaScript.
That matters, but mostly for developer tools and API documentation. If your audience uses AI coding assistants like Cursor or GitHub Copilot to interact with your product, token efficiency can improve integration.
For ecommerce brands selling pet supplies, insurance companies explaining coverage, or B2B SaaS companies targeting nontechnical buyers, token efficiency does not automatically translate into traffic.
llms.txt is a sitemap, not a strategy
The closest comparison I can make is a sitemap.
Sitemaps are useful infrastructure. They help search engines discover and index content more efficiently. But I would not credit traffic growth to simply adding a sitemap. The sitemap documents what exists; the content drives discovery.
Llms.txt works in a similar way. It may help AI models parse a site more efficiently if they choose to use it, but it does not make the content more useful, authoritative, or likely to answer user queries.
In my analysis, the sites that grew did so because they:
Created functional assets such as downloadable templates, comparison tables, and structured data.
Earned external visibility through press and backlinks.
Fixed technical barriers such as crawl and indexing issues.
Published content optimized for extraction, including FAQs and structured comparisons.
Llms.txt documented those efforts. It did not drive them.
What actually works
The two successful sites showed me what actually matters.
Create functional, extractable assets. The SaaS platform built 27 downloadable templates that users could deploy immediately. AI models surfaced them because they solved real problems, not because they appeared in a markdown file.
Structure content for extraction. The neobank rebuilt product pages with comparison tables for interest rates, fees, and account minimums. That is data AI models can pull directly into answers without heavy interpretation.
Fix technical barriers first. The neobank fixed crawl errors that had blocked content for months. If AI models cannot access your content, no amount of documentation will help.
Earn external validation. Coverage from Bloomberg and other major publications drove referral traffic, branded searches, and likely influenced how AI models assessed authority.
Optimize for user intent. Both sites answered specific queries, such as “best project management templates” and “how do [brand] interest rates compare?” Models surface content that maps to what users ask, not content that is merely well documented.
None of this requires llms.txt. All of it can drive results.
Should you implement an llms.txt file?
If you run a developer tool and AI coding assistants are a primary distribution channel, I would implement llms.txt. In that context, token efficiency matters because your audience is already using agents to work with documentation.
For everyone else, I would treat llms.txt like a sitemap: useful infrastructure, not a growth lever.
It is good practice to have. It likely will not hurt. But the hour spent implementing llms.txt is often better spent restructuring product pages with extractable data, publishing functional assets, fixing technical SEO issues, creating FAQ content, or earning press coverage.
Those tactics have shown real ROI in AI discovery. Llms.txt has not, at least not yet.
The lesson I take from this is not that llms.txt is bad. It is that we are reaching for control in a system where the rules are still being written. Llms.txt offers comfort because it is concrete, actionable, and familiar. It looks like the web standards we already understand.
But looking like infrastructure is not the same as functioning like infrastructure.
My focus would stay on what is already working:
Create useful content.
Structure it for extraction.
Make it technically accessible.
Earn external validation.
Platforms and formats will change. The fundamentals will not.
Recently, I dove deep into a 2023 Google patent that outlines how AI systems could evolve to grasp a deeper understanding of businesses, brands, products, and other entities by drawing from websites and public data.
This patent details a method for AI to extract information, recognize relationships, and eventually create what Google refers to as a ‘deep, holistic characterization’ of an entity.
As AI systems hold more sway in search results, it seems our SEO strategies might need to pivot. We may need to ensure that Google comprehends not just what we say, but who we truly are.
Historically, Google has been helping users discover information published on webpages for more than two decades now. But with their search products becoming more conversational and driven by recommendations, just understanding individual documents doesn’t seem to cut it anymore.
For AI to efficiently suggest a business, compare products, or detail a brand, it first needs to understand the entity standing behind the content.
This is where Google’s intriguing ‘Data extraction using LLMs’ patent comes into the picture. On the surface, it might seem like your everyday content extraction tool, yet Google speaks of a larger ambition here.
The patent posits that AI should help build and enrich a comprehensive, nuanced profile of a specific entity. Google’s definition of an entity stretches across people, businesses, places, objects, and concepts.
Rather than merely skimming facts or indexing content, the system aims to interpret data, connect relationships, produce summaries, and ultimately grasp the entity those details represent.
To illustrate this, the patent includes diagrams showcasing how AI processes various information sources and forms an understanding of an entity’s identity, attributes, and relationships.
This AI-driven model of entity understanding transforms traditional SEO strategies by focusing not just on page content but on the holistic representation of a business or product across multiple platforms and data points.
The patent’s strategy involves capturing and interpreting information across diverse media and formats, underscoring the need for brand consistency across all public communications.
If you’re anything like me, tapping into this new perspective in SEO involves analyzing your own digital footprint, ensuring your brand’s story, values, and attributes are consistently communicated across all channels, including your website, social media, and third-party platforms.
Both local businesses and large enterprises could benefit substantially from this approach by presenting a coherent digital identity. When Google’s AI can accurately piece together who you are, you’re more likely to be the name that AI recommends.
Ultimately, this shift in SEO from focusing on isolated webpage optimization to fostering comprehensive entity understanding presents a new challenge—creating an intertwined digital narrative of who you are and what you offer.
Recently, I discovered that Google introduced an AI opt-out feature, and it got me thinking.
For as long as I can remember, we’ve been pushing Google for more insight into AI traffic and control over our content’s portrayal in AI settings.
Now, this week, Google answered us with new controls allowing site owners to opt out of AI-powered experiences, like AI Overviews and AI Mode, coupled with fresh AI reporting tools in Google Search Console. Although still in early beta, it signals progress.
Despite this being a step forward, it’s sparked a split. Some are excited about the reporting aspect, while others debate whether opting out is wise.
What intrigued me wasn’t the announcement itself, but how swiftly the conversation pivoted from seeking visibility to potentially forfeiting it.
Let’s clarify what Google really launched with their announcement. The new controls don’t hinder AI Overviews or user engagement with AI Mode, nor do they stall AI’s momentum. Users will continue to engage with AI for searching and queries.
Essentially, publishers have a newfound ability to determine whether their content appears in AI-powered experiences. Was it Google’s plan or a response to external pressure, such as the UK Competition and Markets Authority?
This isn’t a debate about AI itself disappearing. What changes is brand eligibility within AI interactions. If a site like Expedia opts out, people will still plan trips—they’ll just find someone else in the AI-generated responses.
The choice is not about AI’s success, but rather about whether your brand remains present when users turn to AI solutions.
I get it—the appeal to opt out stems from fears around lost traffic and how AI uses our content.
Yet, assuming that opting out changes user behavior is where I disagree. Users aren’t concerned about a brand’s participation; they’re using AI to get quick answers.
Opting out may seem like a decision to curb AI adoption, but it more so enhances your competitors’ visibility. They snag the spotlight and gain trust while yours potentially fades.
The goal isn’t just visibility reduction—it’s about evolving with search behavior changes to remain seen.
Google’s announcement didn’t just focus on opting out but also on the new AI data they’re offering. Though imperfect, it’s a step towards greater transparency in AI search interactions.
Despite demands for more comprehensive reports, reality shows SEO has long dealt with imperfect data. Some of SEO’s big wins came from leveraging imperfect data.
Hence, we shouldn’t be stuck waiting for flawless data. While not perfect, it’s more than what we had before and will likely evolve further.
In my approach, reporting must expand beyond traditional SEO metrics, encompassing a wider discovery landscape, including AI and interaction insights.
We need to assess brand mentions, citation frequency, and how they’re perceived across differing AI platforms. Visibility stretches beyond mere traffic metrics.
Ultimately, we must rethink our questioning. Instead of asking, ‘Should I opt out of AI?’, ask, ‘Can I afford to be absent where users find brands?’ They’re already in these spaces—why shouldn’t we be?
Google’s update isn’t just a feature but a strategic pivot. By choosing to opt out, you aren’t erasing AI; you’re simply amplifying someone else’s presence.
Are you ready to adapt, or will you stay behind, longing for Google’s ‘free clicks’?
When I first heard about the Profound Index, it intrigued me as the ultimate leaderboard for AI search. Its reputation precedes it, setting a benchmark for excellence in AI-driven search solutions.
The image above perfectly encapsulates what the Profound Index represents—a fusion of innovation and performance in AI search technology. This impressive leaderboard not only showcases top contenders but also encourages competitive enhancement within the AI community.
For anyone deeply invested or casually interested in AI advancements, understanding the Profound Index provides insights into where AI search is headed. It’s a journey worth exploring for its potential to revolutionize how we interact with and leverage AI search capabilities.
When I think about how artificial intelligence is revolutionizing advertising, a common belief is that AI is killing advertising. But, in reality, AI is not the end of advertising; it’s merely transforming it into new dimensions. With AI seamlessly integrating into search, assistants, productivity tools, and beyond, it’s only natural for advertising to follow suit.
I’ve noticed that while the density of ads may shift in AI-led experiences, the opportunities for advertising are actually broadening. There are new surfaces emerging continuously, and they all offer exciting chances for advancers and advertisers alike.
To me, the divide between paid and organic isn’t as clear-cut anymore. The same AI systems powering search experiences are also driving ad campaigns and influencing brand visibility across Google’s expansive ecosystem.
This calls for a change in how we brands perceive visibility. Paid and organic aren’t just isolated competitors vying for clicks; instead, they’ve become alternative strategies influencing the same AI systems. As a result, the signals that shape organic visibility may also impact paid performance.
The traditional search engine results page (SERP) we once knew, consisting of 10 blue links, a handful of ad slots, and a side panel, no longer holds the same dominance. Back then, dedicated teams managed paid and organic strategies separately, each with its own set of tools and quarterly goals.
Things changed for me when Dynamic Search Ads (DSA) appeared, using my website’s content to cleverly create ad titles and determine bids, merging the lines between our organic strategies and paid efforts.
Stepping into the modern age, Performance Max (PMax) campaigns took the very logic of DSAs and applied it across every Google surface—importantly altering how ads are placed from Search and YouTube to Maps and more.
Of course, it isn’t without its nuances. If Google’s Gemini doesn’t have a thorough understanding of our brand, the system has to fill the gaps with assumptions, which may not align with our intended brand narrative. It’s crucial to train these AI systems deliberately, or we risk losing control.
Strategically, I’ve come to realize that paid campaigns help me discover which target audience-intent-profit combinations convert best. I can then build my organic content around these successful elements, creating a feedback loop where each strategy amplifies the other.
I’ve witnessed AI tools become indispensable in automating complex processes that traditionally demanded a lot of manual effort. However, I’ve also seen them used without any real benefit just because they are available.
That’s why I prefer focusing on AI applications that save time and address genuine challenges.
Recently, I was tasked with aligning the SEO architecture for over a dozen websites across three separate businesses, eight regional domains, and numerous languages, including three English dialects, Italian, Japanese, Spanish, Thai, French, and Korean.
Mapping thousands of URLs to create seamless hreflang XML sitemaps traditionally required specialized software or extensive spreadsheet work. Instead, I used Google Gemini to develop a custom Python script to handle the heavy lifting.
Here’s how an initial prompt evolved into a fully customized automation tool and what it taught me about utilizing AI for technical SEO.
Where AI Delivers the Most Value
I leverage AI primarily for practical, time-saving tasks, including:
Generating regex patterns when I need quick solutions without researching syntax from scratch.
Creating complex spreadsheet formulas for reporting workflows that depend on manual data exports.
Speeding up research and planning for projects requiring competitive analysis across business lines.
Building custom automation tools for recurring SEO and data-processing tasks.
The hreflang project I discuss here fits perfectly into the last category.
Mapping hreflang at Scale
The challenge was straightforward: accurately map thousands of URLs across multiple multilingual websites into cohesive hreflang XML sitemaps.
I chose not to tackle this manually. Instead, Google Gemini helped me build a custom Python solution.
Here’s a walkthrough of how the process unfolded.
Phase 1: Asking for an Approach, Not Just a Script
One common pitfall of using generative AI for coding is asking it to sprint before understanding the course. Typing, “Write a Python script to create an hreflang sitemap,” will yield generic code prone to breaking with real-world data.
Instead, I started by asking for an approach. I detailed the scenario: multiple regional domains, organic growth over several years leading to mismatched URL slugs, translated subfolders, and appended revision years.
Gemini suggested a multi-step, data-driven approach:
Crawl the websites to collect live URLs and their metadata.
Use Python in Google Colab to process the raw data.
Run an exact match cluster to group identical slugs.
Use an advanced semantic AI model (like SentenceTransformers) to fuzzy match translated pages based on their titles and normalized URLs.
Phase 2: Crawling and Data Collection
Following the recommended strategy, I used a crawler to spider all regional websites to generate a unified CSV file with live URLs, status codes, title tags, and H1s. Screaming Frog proved ideal for this task.
The quality of AI output relates directly to the quality of your crawl data, so make sure it’s robust.
An AI script can miss an obvious “exact match” if a target URL is a 404 or a 301 redirect. Before feeding data into the script, filter your CSV to include only indexable content.
Google Colab offers a free, cloud-based Jupyter notebook environment for coding, bypassing local installations or environment variable issues. I used Google Drive to access it. The free version sufficed for this project.
After uploading the CSV to Colab, Gemini provided an initial Python script that utilized a domain-mapping routine to assign language codes, clean the URLs, and generate an XML tree. The initial results required refinement.
Phase 4: The Iteration (Where the Real Work Happens)
If you expect AI to produce a flawless script on the first try, you’ll be disappointed. Like an intern, AI requires oversight. The true value lies in iteration.
After running the initial script, several unmatched URLs left orphaned pages rather than grouping them with international counterparts. Here’s how I iteratively guided AI through the complexities of human-managed websites.
The Directory Flattening Problem
The U.S. site had recently reorganized its blog into topical folders, unlike the Mexican and Italian sites. I presented these mismatches to Gemini, leading to a script adjustment that flattened directories, allowing slugs to align.
The Aggressive Semantic Trap
Concept traps we implemented were initially strict. A UK article about manufacturing wouldn’t match its Italian counterpart due to a slightly different title. By loosening these traps for general industries and enforcing them for critical terms, the AI became adept at delivering better matches.
The Translated Slug Epiphany
The pivotal insight arrived when examining Mexican blog orphans. A Spanish URL /detras-de-escenas-historias... matched the English /behind-the-scenes-stories..., which I pointed out to Gemini. As a result, Gemini updated the script to create a “Combined Semantic Signature,” dynamically translating slugs and efficiently bridging language gaps.
This project reinforced a simple truth: AI excels as a collaborator rather than a shortcut.
Be the strategist, let AI be the coder: Rather than demanding a finished product, discuss architecture and logic first, treating AI as a junior developer needing guidance.
Provide concrete examples: Don’t simply state, “It’s broken.” Give specific failed URL examples or mismatches to help AI refine its logic.
Embrace the iterative loop: Run the code, identify issues, and iterate. Each iteration enhances the tool’s intelligence.
Leverage Google Colab: You don’t need to be a Python guru to apply Python in SEO. Colab bridges the gap, providing access to complex data science libraries in your browser.
In the end, I had a fully customized Python script capable of processing a massive CSV to generate a cross-referenced hreflang XML sitemap in minutes.
Though AI isn’t replacing technical SEOs, those who collaborate with AI to build scalable tools will have a significant edge.
I recently discovered that Google has updated its guidelines on optimizing for AI Search, and they’ve made it clear that LLMS.txt files on your site won’t impact your search rankings. It’s a relief to know that Google Search doesn’t actually utilize these files.
The portion of Google’s update that caught my attention explains that there’s no need to create new machine-readable files, such as AI text or Markdown files, to appear in Google Search, even with generative AI. Google will still discover, crawl, and index a variety of files, but these won’t receive special treatment.
Google also mentioned that maintaining LLMS.txt files for other services is perfectly fine and won’t influence your visibility in Google Search. In short, these files neither harm nor enhance your standing in search rankings.
For those interested, here is a valuable section screenshot along with more resources on the topic:
Expressing why I care about this, there’s ongoing confusion around how Google handles such files. Remember, having them on your site won’t help but also won’t hurt your SEO efforts.
