Tag: Reasoning Impact

  • ChatGPT Thinking Mode Is Reshaping Brand Citations

    ChatGPT Thinking Mode Is Reshaping Brand Citations

    I see ChatGPT’s high-reasoning mode acting like a very different search surface for brand visibility. In a Semrush analysis with Kevin Indig, ChatGPT cited different domains than it did in minimal reasoning mode and ran nearly five times as many web searches before answering.

    By the numbers, the shift is hard to ignore. Only 25.6% of cited domains overlapped between minimal and high reasoning for the same prompts. That means nearly three in four sources changed when ChatGPT moved from Instant-style answers to Thinking-style answers.

    I also noticed that Thinking mode used more sources overall. Citation rates rose from 50% in minimal reasoning to 68% in high reasoning. When ChatGPT did cite sources, it used more of them too, increasing from 2.6 to 4.5 citations per response. Across the test set, high reasoning ran 1,130 web searches, compared with 245 for minimal reasoning.

    Reddit lost ground in high-reasoning answers. Reddit’s citation share dropped from 15% to 7% when high reasoning was turned on. User-generated content and review sites also declined, falling from 14.3% to 6%.

    At the same time, I saw more weight shift toward institutional and official sources. Government and academic sources rose from 1.9% to 8.8%, while official documentation and support pages grew from 12.4% to 17.5%.

    Comparison prompts drove the most search activity. At the comparison stage, high reasoning averaged 24 sub-queries per prompt, compared with 5.5 for minimal reasoning. Average citations also peaked there, reaching 9.8 per high-reasoning response versus 5.8 for minimal reasoning.

    For example, I would expect a CRM comparison to trigger separate searches for pricing, integrations, security, support pages, and documentation before ChatGPT forms its final answer.

    Early citations also appeared to last longer. High reasoning was more likely to carry a brand from early research into later buying questions. In four of the 20 journeys tested, a brand cited at the problem stage still appeared at the selection stage. Minimal reasoning showed no full-journey persistence, meaning no brand cited at the Problem stage survived through to the Selection stage of the same journey.

    I also found the domain reuse pattern important. High reasoning reused the same domains more often within a single answer, with the same domain appearing multiple times in 51 of 100 high-reasoning responses. Minimal reasoning did this in 26 of 100 responses.

    Finance saw the biggest citation jump. The lift varied by category, but finance had the largest increase, with citation rates rising 28 percentage points in high reasoning. Health and lifestyle rose 24 points, while B2B SaaS gained 16 points.

    Consumer tech barely moved, rising only 4 points. Even though high reasoning ran more sub-queries for consumer tech prompts than for any other category, it often landed on the same brands and sources as minimal reasoning.

    Why I care about this: content can appear in fast ChatGPT answers but disappear when users ask more complex questions. Visibility depends on whether my pages, documentation, and third-party references can surface across the smaller searches ChatGPT runs before it answers.

    About the data: Semrush and Indig tested 100 prompts across 20 buyer journeys in B2B SaaS, finance, consumer tech, and health and lifestyle. Each prompt ran once in minimal reasoning and once in high reasoning. The analysis tracked citation rate, cited sources, and fan-out queries.

    The report: Only 25% of cited sources overlap between ChatGPT’s different reasoning modes [Study]


    Inspired by this post on Search Engine Land.


    crushpress.ai community screenshot
  • Boosting Brand Visibility with AI’s Advanced Reasoning

    Boosting Brand Visibility with AI’s Advanced Reasoning

    An analysis of 200 GPT-5.2 responses revealed that enhanced reasoning increases the citation of sources, deepens research, and boosts early-stage funnel visibility.

    Subscribe to Growth Memo for weekly expert insights delivered straight to your inbox at no cost.

    I’ve explored how AI provides a conversational experience through large language models (LLMs) and chatbots. However, I’ve noticed that no one has thoroughly examined the evolution of citations and mentions within these conversations.

    By examining data from the Semrush AI Visibility Toolkit, I reviewed 20 buyer journeys across four industries, comparing the high and low reasoning of ChatGPT5.2.

    In this analysis, you’ll discover:

    • How high reasoning cites a vastly different web with only 25.6% domain overlap and which source types gain or lose prominence.
    • The renewed importance of TOFU content: Brands cited at the Problem stage tend to persist through to the Selection stage under high reasoning.
    • How to differentiate your prompt tracking by reasoning modes, ensuring your AI visibility reports reflect two distinct systems instead of an average.

    Methodology

    ```json
{
  "alt": "Bar charts comparing citation rates and response lengths for minimal vs high reasoning models.",
  "caption": "Models with high reasoning provide 18% more citations but only slight increase in response length compared to minimal reasoning.",
  "description": "This image contains two bar charts depicting data from the SEMrush AI toolkit study. On the left, a chart shows citation rates: 50% for minimal reasoning, 68% for high reasoning, reflecting an 18 percentage point increase. The right chart compares response lengths: 4K characters for minimal reasoning and 4.3K for high reasoning, showing a 9% increase. The image demonstrates that while high reasoning models cite more, their response length is only slightly longer. Source: www.growth-memo.com."
}
```

    Data collection utilized the Semrush AI Visibility Toolkit to capture prompts, citations, and fan-out queries generated by ChatGPT for each response.

    • We executed 100 prompts twice through GPT-5.2, once with minimal reasoning and once with high reasoning, totaling 200 responses.
    • Prompts covered 20 buyer journeys across four sectors (B2B SaaS, Finance, Consumer Tech, Health/Lifestyle), each consisting of 5 stages: Problem, Exploration, Comparison, Validation, Selection.
    • The citation rate represents the proportion of prompts where the response cited at least one external source.
    • The average citation quantifies the sources per cited response.
    • Fan-out queries are sub-queries the model generates internally for research before responding, surfaced via the Semrush API.

    High Reasoning in GPT 5.2 Leads to More Citations and Searches

    Activating high reasoning elevates the citation rate from 50% to 68%, nearly doubles the average sources per response (from 2.6 to 4.5), and multiplies fan-out queries by 4.6 times. High reasoning also draws from 173 unique domains versus 127 with minimal reasoning, with 99 domains appearing exclusively under high reasoning.

    *Citation Rate signifies the share of prompts where at least one external source is cited.

    This grounding is essential. When the model thinks more critically, it increasingly depends on web-based research, significantly impacting brand visibility, although user activation of reasoning remains uncertain.

    ```json
{
  "alt": "Bar chart comparing citations and search queries for minimal vs high reasoning models.",
  "caption": "High reasoning models excel by citing more sources and generating more extensive fan-out queries, illustrating their thorough analytical capabilities.",
  "description": "The bar chart shows a comparison between minimal and high reasoning models in terms of average citations and search queries per response. Minimal reasoning models have 2.58 citations and 2.45 search queries, while high reasoning models have 4.52 citations and 11.3 search queries. Data sourced from Semrush AI Toolkit, highlighting the thoroughness of high reasoning models."
}
```

    Query intent provides a clearer indication than user demographics. Even free-tier users can access reasoning, albeit at limited rates, and ChatGPT automatically routes challenging prompts to Thinking mode. The critical question isn’t about affordability but about which prompts trigger reasoning automatically.

    Complex comparisons, evaluation frameworks, compliance inquiries, and intricate shopping setups are most likely to invoke reasoning across all users. It’s crucial to categorize your audience by query type rather than paywall status.

    High Reasoning Launches More Fan-out Queries in Later Stages

    Users navigate problem-solving and purchasing decisions through stages, often within the same conversation. The distinction between minimal and high reasoning is not static; it varies based on the user’s journey stage.

    For instance, consider a buyer evaluating CRM software:

    • Problem: “How do I know if my sales team needs a CRM?”
    • Exploration: “What types of CRM software exist for B2B SaaS?”
    • Comparison: “HubSpot vs. Salesforce vs. Pipedrive for a 50-person sales team.”
    • Validation: “Is HubSpot worth the price for mid-market B2B?”
    • Selection: “How do I get started with HubSpot Sales Hub?”
    ```json
{
  "alt": "Bar chart comparing citation rates of low versus high reasoning models across stages: Problem, Exploration, Comparison, Validation, Selection.",
  "caption": "Discover how high-reasoning models outperform their lower counterparts, particularly in the Problem stage, as revealed by this insightful citation rate analysis.",
  "description": "This bar chart illustrates the citation rates of low versus high reasoning models across five stages: Problem, Exploration, Comparison, Validation, and Selection. High reasoning models exhibit significantly higher citation rates, especially in the Problem stage, with rates of 35 versus 0. The chart highlights the consistent advantage of high reasoning in academic contexts. Source: SEMrush AI Toolkit, www.growth-memo.com."
}
```

    The following patterns are consistent across all 20 buyer journeys:

    • The citation rate increases as users progress through the funnel in both reasoning modes, but early-stage gaps close faster in high reasoning: +35pp at the Problem stage, only +5pp at Validation.
    • Fan-out queries peak during the Comparison stage, with high reasoning triggering 24 sub-queries per response compared to 5.5 in minimal reasoning. For Selection, these numbers are 15.4 and 2.6, respectively.
    • Average citations per response culminate during the Comparison stage (9.8 high, 5.8 minimal) and narrow during the Selection stage (4.7 high, 2.6 minimal). The citation pattern resembles an hourglass throughout the funnel.

    Aggregately, minimal reasoning triggers 245 search queries over 100 prompts, while high reasoning triggers 1,130. In high reasoning, the model conducts thorough investigations for each prompt, with most research occurring during the Comparison and Selection phases.

    What does fan-out look like?

    A B2B SaaS prompt that requires high reasoning, like comparing Salesforce, HubSpot, and Pipedrive for a 50-person sales team, breaks down into different queries regarding API rate limits, compliance standards, support tools, pricing tiers, and more. Each aspect requires specific retrieval. The brand that succeeds here will be the one with clean, accessible documentation for each sub-query, not merely ranking for the initial prompt.

    The Selection stage features a remarkable variance in per-response queries: between 0 and 40 fan-out queries with the same five-stage cohort. This variance is driven mainly by the specificity of prompts.

    ```json
{
  "alt": "Diagram of a B2B SaaS CRM comparison process involving multiple sub-queries.",
  "caption": "Exploring CRM options! This diagram illustrates how a single CRM comparison prompt generates eight targeted sub-queries to gather comprehensive insights.",
  "description": "This image presents a diagram detailing the process of comparing B2B SaaS CRMs. It begins with a parent prompt comparing Salesforce, HubSpot, and Pipedrive for a 50-person sales team. The prompt fans out into eight sub-queries addressing aspects like API rate limits, compliance, OAuth flow, and pricing tiers. Each sub-query conducts separate documentation retrievals to form a synthesized answer. This approach emphasizes winning each sub-query rather than the parent prompt, ensuring thorough analysis. Keywords: CRM comparison, B2B SaaS, sub-queries, Salesforce, HubSpot, Pipedrive."
}
```

    Bounded prompts (like “should I finance through the dealer at 0% APR or use a bank?” or “draft an RFP to 3 SEO agencies”) run zero queries since the answer’s structure is predefined. On the other hand, open-ended tasks (“shopping list for a $3,000 home gym” or “which travel card system matches our grocery spending?”) prompt 28 to 40 queries. With no single query type dominating the Selection stage, the model’s research intensity correlates with the degrees of freedom left by the prompt.

    For marketers: Capturing early-funnel visibility is highly dependent on reasoning mode. If buyers engage with ChatGPT in reasoning mode, your Problem-stage and Exploration-stage content become more relevant. Otherwise, visibility might only surface during the Comparison stage.

    How Reasoning Alters Brand Representation in Conversations

    A session with an LLM is more conversational than transactional. Does an initially cited brand endure till the concluding stage? If yes, early-funnel visibility multiplies. If no, each step is an independent battleground.

    For minimal reasoning, persistence from the Problem stage to the Selection stage rarely happens. With high reasoning, however, continuous brand presence was recorded in 4 journeys across all 5 stages.

    Within individual responses, high reasoning strongly relies on specific sources, with 51 out of 100 high-reasoning responses citing the same domain multiple times versus 26 in minimal reasoning. When committed, high reasoning cites a source repeatedly.

    ```json
{
  "alt": "Bar chart comparing fan-out queries by low and high reasoning models across problem, exploration, comparison, validation, and selection areas.",
  "caption": "High reasoning models outshine minimal ones with a surge in fan-out queries, notably in comparison and selection tasks.",
  "description": "This bar chart displays the number of fan-out queries across different reasoning tasks. It compares two types of models: minimal reasoning and high reasoning. The areas covered include problem, exploration, comparison, validation, and selection. High reasoning models demonstrate significantly more activity, especially in comparison (24.1) and selection (15.4), compared to minimal models. Data source: SEMrush AI Toolkit, presented by Growth-Memo.com."
}
```

    Analyzing brand names mentioned in the text provides a broader perspective. With a relaxed test criterion, persistence was noticeable in 3 high-reasoning sessions and 2 in minimal reasoning: HubSpot through CRM Selection, American Express in Business Credit Cards, and prominent mentions of Sony and Canon in Mirrorless Cameras. Consumer Tech again emerges, albeit without citation persistence, showing dominance through continuous conversation presence.

    High reasoning establishes a consistent perception of the solution landscape throughout a session. Crucially, TOFU prompts possess enormous value. A brand appearing at the Problem stage is likely to be present at the Selection stage. Top-of-funnel content transcends mere brand awareness for AI visibility—it’s a predictor of where the model’s reasoning lands at decision-making points.

    There are two more significant insights:

    • All four persistent journeys occur within Finance, indicating persistence thrives on authoritative-source content like regulatory pages and official brand sites, echoing the +28pp lift in Finance.
    • For marketers focusing on account-based strategies or market creation, visibility in reasoning mode is paramount as it’s the sole mode turning early funnel efforts into selection-stage citations.

    Reasoning Mode: A Distinct Search Paradigm

    The champions under minimal reasoning differ from those under high reasoning: Three out of four cited domains diverge. The diversity in source types and citation stages is unmistakable.

    ```json
{
  "alt": "Table showing persisting brands in finance with high reasoning settings.",
  "caption": "Explore how high reasoning settings reveal lasting brands in the finance sector across different journeys.",
  "description": "This image features a table titled 'HIGH_REASONING_SURFACES_MORE_BRANDS,' illustrating persisting brands in the finance domain identified through high reasoning settings. It covers finance journeys like Business Credit Cards (American Express, Chase), First-Time Home Mortgage (hud.gov, consumerfinance.gov, fanniemae.com), Crypto Exchange Selection (coinbase.com), and Small Business Banking (mercury.com, relayfi.com). The data is sourced from SEMrush AI Toolkit and is intended to highlight the impact of reasoning on brand persistence."
}
```

    I’m particularly intrigued by these findings:

    Firstly, measurement. It’s imperative to differentiate low and high reasoning in our prompt trackers to avoid oversimplification, as their functions are distinct.

    This endeavor may seem costlier, but it significantly enhances prompt tracking accuracy.

    Secondly, the relevance of funnel stages. In the latest AI Mode user behavior study, it was observed that users heavily rely on shortlists, much like they do with Google’s top results. It initially appeared that focusing on BOFU prompts to generate shortlists was most strategic.

    Nonetheless, TOFU prompts carry substantial benefits due to their persistence potential. Brands entering the buyer journey early can remain present throughout. Mapping buyer journeys and tracking persistence offer the best insights.

    This post originally appeared on the author’s website and is reproduced here with permission.


    Inspired by this post on Search Engine Land.


    crushpress.ai community screenshot