I recently discovered that Google has made some updates to their JavaScript SEO basics documentation. This change has brought clarity to how Google’s crawler deals with noindex tags on pages utilizing JavaScript. The main takeaway? If you’re aiming to have your page indexed, definitely avoid including a noindex tag in the original page code.
What’s New: Google has adjusted this section to specify that when Google encounters a noindex tag, it may bypass rendering and executing JavaScript. Consequently, efforts to modify or remove the robots meta tag using JavaScript might not yield the desired results. So, if indexing is a goal, keep the noindex tag out of the original code.
Previously, the guidelines indicated a certain certainty: if a noindex tag was detected, Google skipped rendering and executing any JavaScript. This meant any attempts to counter this with JavaScript adjustments would simply not work. The advice stood firm—keep noindex tags out of the original code if there’s any chance you need the page indexed.
Reason for Change: Google clarified that while it can render pages employing JavaScript, this behavior is not consistently defined and is subject to change. If there’s any chance you want your page to show up in search, play it safe and leave out the noindex tag from the original code.
Why This Matters: It’s often safer to steer clear of JavaScript when setting crucial protocols, especially concerning the blocking of Googlebot or other crawlers. If you need a search engine not to rank a particular page, avoid using JavaScript to execute those directives.
Today, I am thrilled to share that Search Engine Land is celebrating its 19th anniversary!
Nineteen years is an incredible milestone. For almost two decades, we have been diving deep into the ever-evolving world of search engines, always striving to make sense of the changes and challenges Google and the search industry present.
This year, 2025, has been one of the most transformative since our launch in 2006. The rapid pace of change has been exhilarating.
Through it all, our mission remains steadfast: to provide clear news, insightful analysis, and practical guidance to help you navigate the world of search.
Before we look to the future, I want to express my heartfelt thanks for your support and reflect on the past year with you.
Thank you for reading
Sincerely, thank you for being with us.
Every day, we focus on you: what you need to know, what really matters, and what changes will impact your work today or your strategy months down the line.
Our goals include:
Focusing on meaningful stories, not filler.
Delivering news clearly and quickly.
Providing essential context and expertise.
Being a dependable resource in a fast-changing industry.
Helping you anticipate where search is heading, even when it’s unclear.
If you haven’t yet, I encourage you to subscribe to our daily newsletter for a curated summary of all things search, helping you stay updated without feeling overwhelmed.
Thank you to the Search Engine Land team
Our team’s passion is what has driven our success for almost two decades.
Though small, our team accomplishes significant and impactful work because we are mission-driven and dedicated to search.
I extend my greatest thanks to:
Barry Schwartz. With 22 years of experience, Barry’s passion for search ensures complex topics become understandable. He is indispensable.
Anu Adegbola. Focusing on paid media, Anu offers clarity amidst constant changes with her insightful writing.
Angel Niñofranco. Angel plays a crucial role in our SME articles through his coordination and editorial oversight.
Kathy Bushman. Kathy’s behind-the-scenes expertise ensures SMX events are seamless and valuable.
And to the entire team at Third Door Media within Semrush, whether or not your name appears here, your contributions are invaluable.
Top highlights from the past year
Despite the uncertainties of this year, Search Engine Land thrived, thanks to the trust of our community.
SMX Advanced returned in person for the first time in 6 years
This was arguably the highlight of the year. SMX Advanced’s return in person after six years was electrifying.
With attendance surpassing expectations, the sessions were dynamic, and conversations felt like reunions for the search marketing community. It was clear that we all missed these face-to-face exchanges about AI, Google’s updates, and more.
We learned again that when great minds gather, extraordinary things happen. We eagerly await our next gathering in Boston, June 3-5.
Defining industry coverage of AI Overviews and the new era of search
This year, more than ever, transformed the search landscape. We’ve provided the clarity and reporting needed in this evolving environment.
Our readers rely on us for insights during times of change, and we take pride in shaping the industry’s future understanding of search.
Subject Matter Expert (SME) program growth
This year saw a surge of new and returning readers turning to us for insight into SEO and PPC shifts, from AI to SERP experiments.
Our growth owes much to our fantastic contributors, and I extend my gratitude for their impactful work.
Looking ahead: What’s next for Search Engine Land
As we embrace our 19th year, our resolution is steadfast: to offer unparalleled coverage of search-related topics.
This year, you can anticipate:
Continued breaking news on SEO, PPC, AI, and more.
In-depth analysis, guides, and contextual explainers on industry evolution.
SMX events tailored around the nuances of AI-driven search.
Enhanced expert viewpoints, data, and market clarity.
Mark your calendars for:
SMX Advanced: June 3-5
SMX Next: Nov. 18-19
We have much in store for you, with the aim of equipping you with the insights necessary for your best work.
A brief look back to where it all began
Launched on Dec. 11, 2006, Search Engine Land began with a vision of search as a vast community. A place of exploration, connection, and evolution. Over these years, it’s grown beyond our expectations.
The mission remains the same:
Search Engine Land is your destination to remain informed, educated, and connected within the world of modern search engines.
Thank you for 19 incredible years
From everyone here at Search Engine Land and Semrush, thank you for your readership, engagement, and passion for the evolving world of search.
Here’s to a promising rest of 2025 and a remarkable 2026.
I recently learned about a significant ruling that will impact Google’s longstanding agreements with tech giants like Apple and Samsung. This decision means that moving forward, Google will only be able to secure its place as the default search engine on devices for one year at a time. Despite this change, I’m not expecting a major shift in Google’s dominance over the search market anytime soon.
Here’s what’s driving the news: On Friday, Judge Amit Mehta described this one-year cap as a crucial step in enforcing antitrust measures. This follows his 2024 decision, which concluded that Google was unlawfully monopolizing the realms of search and search advertising. According to Business Insider, the requirement aims to enforce fair competition in the industry.
Additionally, Judge Mehta’s earlier ruling outlined restrictions for Google:
Google must avoid any exclusive contracts regarding the distribution of Google Search, Chrome, Google Assistant, and the Gemini app.
They cannot condition licensing agreements of the Play Store on the preloading of these applications on devices.
Revenue sharing cannot be contingent on placing or maintaining these applications on devices beyond one year.
Partners are free to distribute alternative GSEs, browsers, or GenAI products simultaneously.
Why I care: This landscape shift could mean that user searches originate from a wider array of platforms. If AI-powered competitors like OpenAI, Perplexity, or Microsoft make even modest advances, we could see a more diverse and challenging search terrain emerge.
Reality check: In my view, this is more of a bump in the road rather than a disruption. Google’s financial resources, brand strength, and user habits continue to provide significant leverage in annual negotiations.
In the past day, I’ve noticed that ChatGPT and Perplexity have launched new AI-driven shopping tools designed to create more intuitive and personalized shopping experiences. These innovations focus on helping us effortlessly discover, compare, and purchase items using conversational queries tailored to our preferences and history.
ChatGPT
Shopping Research. OpenAI is revolutionizing the way I shop by transforming ChatGPT into my personal product researcher.
When I describe what I need, like a “quiet cordless vacuum” or a “gift for my art-obsessed niece,” ChatGPT kicks in to ask clarifying questions and pulls relevant data from the web. In no time, I receive a customized buyer’s guide.
Using my preferences and previous interactions, ChatGPT updates recommendations as I react to items with “More like this” or “Not interested.” It’s a truly adaptive experience.
This feature uses a specialized GPT-5 mini model that’s optimized for shopping and sources reliable information from trusted sites.
It’s available now for both free and paid ChatGPT users, on web and mobile, with extensive use available through the holiday season.
Next up, I’ll be able to purchase items directly within ChatGPT thanks to upcoming Instant Checkout integrations.
Perplexity
New Shopping Experience. Perplexity has rolled out a free, U.S.-based shopping feature centered around enhancing my shopping without replacing the experience.
I simply initiate searches with conversations like “best winter jacket for San Francisco ferry commute,” and Perplexity maintains context even when my needs shift.
It remembers my style and preferences, adjusting future product suggestions accordingly, all while avoiding endless scrolling by providing clear, intent-driven product cards.
Purchases are quick and seamless, thanks to a partnership with PayPal, while still allowing merchants to manage customer relationships.
Retailers might pay attention to this, as conversational shopping reportedly increases purchase intent, although some studies caution that AI-driven conversions aren’t always more successful than traditional methods.
This innovative experience is available now on desktop and web, with mobile apps arriving soon.
AI shopping assistants like ChatGPT and Perplexity are changing the ecommerce landscape. ChatGPT focuses on deep research while Perplexity offers smooth discovery and integrated checkout, both striving to be our go-to platforms by providing personal and custom shopping recommendations.
I’m thrilled to share that Google Posts now includes features that support scheduling and multi-location publishing within Google Business Profiles. These updates are designed to make it easier for us to manage our Google Posts, whether they are for our businesses or clients.
Scheduling. One exciting new feature when adding a Google Post within our Google Business Profiles is the option to “schedule this post.” We can now select the exact date and time when we want our posts to go live.
Lisa Landsman from Google shared on LinkedIn, “Plan your entire week or month in advance! You can now schedule your Google Posts to go live automatically at the perfect time.”
Multi-location publishing. If you, like me, manage several locations for a business, you’ll find the new multi-location feature incredibly convenient. It allows us to quickly copy Google Posts to some or all of our locations with just a click. Lisa Landsman explained, “Easily create a single post and apply it instantly to multiple business locations in one click.”
What it looks like. Here’s a GIF that shows this functionality in action:
Why we care. I care about these updates because I know how busy businesses can be. Often, we don’t have the time to pause everything just to create a timely Google Post about an upcoming event or important message. Now, we can schedule these posts in advance and copy them effortlessly across locations we manage.
As Lisa Landsman from Google pointed out, “We know the upcoming holiday season is a crucial, and hectic, time for your business. It’s also your biggest opportunity to get your events, offers, and updates in front of potential customers who are actively searching.”
I recently came across a fascinating study highlighting how seasonality adjustments can actually backfire for advertisers during Black Friday, driving up costs and reducing efficiency.
A thorough analysis over three years, involving up to 6,000 advertisers, indicates that using Google’s seasonality bid adjustments during Black Friday and Cyber Monday (BFCM) often undermines efficiency, despite the platforms recommending them.
The big picture. Smart Bidding models are crafted to foresee predictable retail surges. Optmyzr analyzed tens of billions of impressions between 2022 and 2024, finding that advertisers who avoided seasonality adjustments usually had better efficiency metrics.
Without adjustments, Smart Bidding:
Recognized the BFCM conversion lift independently
Increased bids rationally
Maintained stable or improved ROAS, particularly in 2024
With adjustments: CPCs surged faster than the actual conversion rates, eroding efficiency.
Reality check: Google doesn’t need your “heads up.” Seasonality adjustments prompt Google to expect a conversion rate rise and to bid accordingly. If your prediction is off—and it usually is—Smart Bidding overshoots.
For example:
You predict a +50% CVR lift
The actual lift is +40%
This results in an overbid of about 7.1%
During BFCM’s high sales volumes, even minor mistakes become costly quickly.
The data: 3 years of the same story
1. Smart Bidding already adjusts for the CVR spike
2022: +17.5%
2023: +11.9%
2024: +7.5%
No additional guidance needed.
2. CPC inflation doubles with adjustments
Across all observed years, CPCs increased approximately twice as much when a seasonal adjustment was used.
3. ROAS drops significantly
Advertisers relying on Smart Bidding saw stable or improved ROAS, whereas those who intervened suffered double-digit losses.
The one exception: “Volume at all costs.” If the aim is pure revenue growth, disregarding margins, seasonality adjustments can be beneficial.
Revenue lifts were notably higher with adjustments:
2022: +50.5% vs. +25.0%
2023: +52.8% vs. +30.3%
2024: +39.9% vs. +33.8%
Efficiency may decline, but volume certainly increases.
When seasonality adjustments make sense. They’re useful when Google doesn’t have prior signals, like one-off or niche events.
Good for:
One-time flash sales
Email-only offers
Surprise clearance sales
Niche seasonal spikes
Not recommended for:
Black Friday
Cyber Monday
Christmas
Valentine’s Day
Any event with a predictable historic pattern
Why we care. Google already recognizes the significance of Black Friday. Smart Bidding is trained with years of BFCM data and can detect conversion rate spikes independently. Overriding this can lead to excessive bidding, increased CPCs, and reduced ROAS, so many marketers might be wasting their budget during this crucial week.
By recognizing when Smart Bidding has an adequate signal, advertisers can avoid expensive errors, maintain efficiency, and reserve seasonality adjustments for when they add true value.
Bottom line. Smart Bidding effectively manages major retail holidays. Seasonality adjustments often bring more chaos than benefits during predictable retail peaks. Keep them for unique, brand-specific events that Google can’t predict.
Smart move: Trust the algorithm — use tools like anomaly alerts, pacing monitors, and bid caps for control without conflicting with Smart Bidding’s core models.
During Black Friday, I’ve noticed many retailers, including myself, wasting substantial advertising budgets on Google Shopping ads. The main issue arises when these ads are still running for products that have already sold out, clearly demonstrating a pressing need for real-time stock management.
As we all know, Black Friday marks the peak of the retail season. However, it’s disheartening to find that so many brands, myself included, end up losing money on Google Shopping ads for items no longer available in inventory.
The problem: The ads continue to run even after items are out of stock, incurring cost-per-click charges with no possibility of conversion. Through a comprehensive study by ShoppingIQ involving 500 global retailers, it was revealed that a staggering 97% kept paying for clicks on items no longer in stock, sometimes persisting for 24–48 hours.
Why I care. Out-of-stock ads are not just a financial drain; they also skew campaign performance and disrupt algorithmic learning. When conversion rates plummet for unavailable products, it damages rankings, reduces ROI, and hampers future bidding strategies.
Example: Take Argos, for instance; they reportedly advertised items that were out of stock during Black Friday, leading to frustrated customers and depleted ad budgets.
Stock update refresh rates:
~24 hours: 90% of retailers
6–23 hours: 5%
48 hours: 2%
Other: 3%
Retailers’ response: Some companies, such as Mamas & Papas, have started leveraging ShoppingIQ’s real-time stock technology. This helps them focus ads solely on products that are actually available. Samantha Dabek, Senior Digital Marketing Manager, shares that they have managed to cut unnecessary costs and ensure advertising is targeted toward in-stock products.
The bigger picture: Google Shopping commands around 75% of US retail search spending. However, the default settings let out-of-stock ads run unchecked. ShoppingIQ strongly advocates for retailers to seek more transparency and control from Google to prevent wasted spending.
Bottom line: For those of us running high-stakes campaigns during Black Friday and other peak times, real-time stock management is essential. Otherwise, each wasted click represents money lost.
I recently discovered a game-changing update from Google that’s bound to catch the attention of many advertisers. Google’s Performance Max now allows me to upload video files directly in the “Edit assets” panel, simplifying the campaign setup process significantly. What’s even better? I don’t need a YouTube channel or Shared Library for this.
Here’s the scoop. This handy feature pops up as an “Upload” tab in the Google Ads UI, making it super easy to add video assets during PMax campaign creation. Just a simple drag-and-drop, and I’m set to move on, especially helpful if I’m new to video advertising.
In the YouTube ad setup, I’ll find a clear, highlighted box prompting me to drop in my video file, smoothing out what used to be a more complicated process.
How does it work? These video files are stored in a Google-managed channel, not on my personal YouTube account. While they’re usable in ads, they don’t function like typical YouTube uploads, which might affect how I manage my content.
Why it matters to me. This update is a boon if I don’t have a YouTube presence or need a quick way to upload video assets. However, I should be mindful of the trade-offs: I’ll have no analytics, no remarketing capabilities, no metadata access, and crucially, I won’t own the assets long-term. It’s a convenient option for quick setups, but I must proceed with caution and ideally upload through a proper brand channel when possible.
Important limitations. Using this method imposes several restrictions:
No YouTube Analytics
No remarketing audiences
No metadata editing
No custom thumbnails
No ability to appeal rejections or restrictions
No brand-channel presence or asset ownership
How I found out. The first mention of this update came from Web Marketing Consultant Dario Zannoni, who shared it on LinkedIn. I appreciated his insights into how this could change my advertising approach.
The takeaway. This feature is a great shortcut if I’m in a hurry or don’t have a robust YouTube setup. Still, maintaining best practices by using my official brand channel ensures I preserve analytics, gather audience data, and retain creative control.
Recently, I’ve noticed Google has started automatically linking YouTube channels with Google Ads accounts. This innovation allows advertisers like me to quickly tap into valuable audience data, though it does require careful permission management.
When Google’s system detects a strong connection between a YouTube channel and a Google Ads account, it takes action by linking them. This gives us richer audience signals without us having to do a manual setup.
What’s happening now? Google will set up these links automatically if a strong relationship is identified, notifying us 30 days in advance. This email notification allows us to decide whether to opt out or connect sooner.
How does it work?
During the 30-day period, if no one opts out, the link will be completed automatically. If I manage both accounts, I can even connect them immediately. There’s flexibility here, too, as I can always adjust permissions or unlink later if needed.
Why this matters to us. This development simplifies how we, as advertisers, access YouTube audience data. It makes it straightforward to target viewers and construct data segments. However, it also introduces uncertainties about control over our assets and the permissions we’ve set.
Benefits for advertisers. Once linked, I can:
Use YouTube interactions to run more effective ads.
Leverage organic views and earned actions for performance insights.
Create data segments from how audiences engage with my channel.
Consider channel engagement as conversion activities, like subscriptions.
Limitations I’ve noticed
Channel owners gain no control over the actual Google Ads account.
Copy or edit capabilities for channel videos are not given to advertisers.
If personalized ads are disabled, audience data reports are also turned off.
Restrictions on Video Ads Certification (VAC) are still applicable; removal of these is specific to the linked Ads account.
Managing these links. If I, as an admin, choose to opt out, I can easily do so through the links provided in the notification emails from Google. If opted out, the link won’t be made. Meanwhile, manual linking can always be done via the traditional Google Ads settings menu.
Initial discovery. The new auto-linking feature was first highlighted by Hana Kobzová, founder of PPC News Feed. More on this can be read here.
Final thoughts. With Google’s new auto-linking, we as advertisers can enjoy less setup hassle and better YouTube performance insights. However, it’s crucial to monitor our notifications to ensure that data sharing aligns with our privacy preferences and company policies.
In my experience, the open web often feels like the Wild West, especially in recent times. Many creators, myself included, have watched as our hard work is scraped and fed into large language models without any hint of permission.
This situation has become a free-for-all, leaving website owners with almost no means to opt out or safeguard their creative endeavors. There have been attempts to address this, such as Jeremy Howard’s llms.txt initiative. Much like robots.txt helps us manage site crawlers, llms.txt aims to provide guidelines for AI companies’ crawling bots.
However, a promising new protocol is on the horizon, potentially granting site owners like myself more control over how AI firms utilize our content. It looks like this might become part of robots.txt, allowing us to set definitive rules around AI system access and usage.
IETF AI Preferences Working Group
In response to this issue, the Internet Engineering Task Force (IETF) began the AI Preferences Working Group earlier this year in January. Their mission is to craft standardized, machine-readable rules to empower site owners to articulate AI usage preferences for their content.
Since its inception in 1986, the IETF has established core Internet protocols like TCP/IP, HTTP, DNS, and TLS. Now, they’re laying down foundations for the open web’s AI era. Leading this group are co-chairs Mark Nottingham and Suresh Krishnan, joined by figures from Google, Microsoft, Meta, and more.
Of particular interest is Google’s involvement via Gary Illyes, who is part of this working group.
“The AI Preferences Working Group will standardize building blocks that allow for expressing preferences about how content is collected and processed for Artificial Intelligence (AI) model development, deployment, and use.”
What the AI Preferences Group is Proposing
This group aims to deliver new standards that empower site owners to determine how LLM-powered systems can utilize their open web content.
A standard track document detailing a vocabulary to express AI-related preferences, independent of content association methods.
Standard track document(s) that explain how to associate these preferences with content using IETF-defined protocols and formats, for example, Well-Known URIs and HTTP response headers.
A standard approach for reconciling multiple preference expressions.
At the time of writing, nothing is set in stone yet. Early documents, however, provide a sneak peek into potential standards.
This working group published two crucial documents in August.
These documents propose significant updates to the Robots Exclusion Protocol (RFC 9309), suggesting new rules and definitions enabling site owners to specify AI content usage permissions.
How It Might Work
AI systems on the web are categorized and assigned standard labels. Whether a directory will exist for site owners to identify system labels remains unclear.
Currently, the defined labels include:
search: for indexing/discoverability
train-ai: for general AI training
train-genai: for generative AI model training
bots: for all types of automated processing, such as crawling and scraping
For each label, you can set two values:
y to allow
n to disallow.
I found it interesting that these rules can be applied at the folder level and customized for different bots. In robots.txt, they’re implemented using a new Content-Usage field, akin to existing Allow and Disallow fields.
Here’s an example robots.txt that the working group shared in their document:
Explanation Content-Usage: train-ai=n indicates that no content on this domain may be used for training any LLM model, whereas Content-Usage: /ai-ok/ train-ai=y permits model training using content within the /ai-ok/ folder.
Why Does This Matter?
There’s significant buzz about llms.txt within the SEO community and its use alongside robots.txt. Yet, no AI company has confirmed adherence to these guidelines, and Google disregards llms.txt.
Website owners, including myself, crave more explicit control over how AI companies leverage our content—be it for training models or RAG-based responses.
I feel that the IETF’s new standards signify positive progress. With Illyes as a contributing author, I remain optimistic that once finalized, companies like Google will embrace these standards, respecting new robots.txt rules during content scraping.
