CrushPress.AI

Unlock AI Prompts in Google Search Console: A Step-by-Step Guide

```json { "alt": "Illustration of a magnifying glass analyzing a cracked black box with colorful dialogue boxes emerging.", "caption": "Unlocking secrets: This vibrant illustration captures a magnifying glass revealing insights from a fractured black box, as colorful dialogue boxes burst forth.", "description": "A dynamic illustration depicting a black box that is cracking open, releasing blue and green dialogue boxes. A magnifying glass is placed near the box, symbolizing analysis or investigation. The image conveys themes of discovery, analysis, and unraveling hidden information, useful for contexts involving data analysis, decryption, and investigative processes." } ```

Written by

shivamcrushpressai

in

,

I’ve been asked numerous times about how to track prompts effectively, especially by those using tools like Profound, Athena, and Peec. The big question on everyone’s mind is, “Which prompts are worth tracking?” In this ever-evolving landscape, it’s challenging to determine what buyers are querying about my company when they use LLMs.

Currently, there isn’t a reliable data source that puts my mind at ease. Unlike traditional search with publicly available Keyword Planner data, it’s unlikely that OpenAI or Google will fully release this kind of data for analysis. Though there have been recent proposals by the UK CMA about Google and data transparency, I’m not holding my breath for significant change.

Long story short, LLM tracking feels like navigating a black box. So, are there any alternative data sources we can use to track which prompts? Perhaps.

Back in November, Jason Packer published an interesting report highlighting how ChatGPT searches accidentally leaked into Google Search Console reports, featuring PII. When this was confirmed by Ars Technica, OpenAI stated the problem affected only a small number of queries.

```json { "alt": "Google Search Console screenshot with humorous text 'Bro, your ChatGPT is leaking' at the top.", "caption": "It seems ChatGPT is getting some curious searches! A humorous take on search queries in this Google Search Console screenshot.", "description": "This image is a screenshot from Google Search Console showing various search queries targeting ChatGPT-related phrases. The phrase 'Bro, your ChatGPT is leaking...' humorously headlines the image. The screenshot lists queries such as 'check this writing below' and 'how do I word this nicely', each with zero clicks. The Google Search Console logo is visible, adding context to the type of data displayed. This image combines analytics with a touch of humor, perfect for illustrating search trends or SEO discussions." } ```

This confirmed, for me, that ChatGPT queries do appear in some Search Console profiles. While privacy implications are significant and beyond this article’s scope, it shows that LLM queries are not impossible to capture.

Additionally, Barry Schwartz has reported that AI Mode data is available in Search Console. This supports the idea that Search Console can track how users interact with LLMs.

Based on my analysis, it seems that AI data appears to come from this area. By applying specific filters, I’ve noted steady increases in impressions over recent months, coinciding with Google’s roll-out of AI Mode features.

```json { "alt": "Line graph showing total clicks and impressions over time with a spike in February.", "caption": "A line graph reveals a significant spike in total clicks and impressions in early February, illustrating a sudden surge in online activity.", "description": "This image displays a line graph from a digital analytics tool, showing total clicks and impressions across several months. The graph indicates a notable increase in activity, peaking in early February with impressions reaching over 2,000. The graph measures daily data, and the spike suggests successful content engagement or a well-timed campaign. This visualization helps in understanding web traffic trends and user interaction with online content." } ```

So, how can I access user prompt data in Search Console? The key is focusing on longer queries. Using regex, we can filter queries with 10 or more words, unveiling prompt-like behavior:

1. Navigate to Search Console Performance > Search Queries

2. Select Add Filter > Query

```json { "alt": "Screenshot of a query filter interface with a regex in the keyword field.", "caption": "A glance at the query filter interface showcasing a regex pattern in action for refined data searches.", "description": "This image captures a section of a query filter interface where a regular expression (regex) pattern is entered in the 'Keyword' field. The interface displays options for filtering data queries based on the specified regex, aimed at capturing queries containing a certain pattern. The 'Apply' button is visible, offering a way to execute the filter settings. The design is clean, with a minimalistic style focusing on functionality and clear user interaction prompts." } ```

3. Choose Custom Regex

4. Input: ^(?:S+s+){9,}S+$

This method revealed understandable, prompt-styled queries when applied to various properties. Though the actual data cannot be shared, examples such as “Map out a full day in Glacier National Park…” highlight the trend.

```json { "alt": "Analysis of AI engine queries with CSV file illustration", "caption": "Delving into AI-driven queries: An analysis reveals unique patterns in AI-mediated search data, illustrating the stark contrast between human and AI search behavior.", "description": "The image showcases an analysis of AI engine queries, highlighting differences between AI and human search behaviors. It includes an illustration of a CSV file labeled 'Queries.csv' and text discussing the nature of AI-generated search data, which typically features longer queries compared to human searches. The image sheds light on patterns in AI-mediated search data and the distinctive traits of AI interactions, making it an insightful piece for understanding AI systems." } ```

Mind you, there’s no direct evidence these queries originate from ChatGPT or similar AI platforms. It’s possible they reflect new user behavior patterns within Google.

Regardless, analyzing these conversational query patterns provides invaluable insight into how customers search using longer strings.

Will Critchlow wisely said, “we’re doing business, not science.” In our shift toward less attributed, zero-click data collection, the choice to leverage this available data is up to us.

```json { "alt": "Image excerpt showing a breakdown of the five dominant prompt structures for user queries to LLMs.", "caption": "Exploring how users frame their questions to AI: A deep dive into the five dominant prompt structures that reveal user engagement with language models.", "description": "The image shows an analysis of common user prompt patterns to language models (LLMs). It specifically details the first two of five dominant prompt structures. The first is asking for curated rankings with queries like "What are the best/top/most...", commonly used for recommendations. The second structure involves 'How to...' requests, mimicking tutorial queries. This breakdown helps in understanding user interactions with AI systems." } ```

Currently, my preferred tool for prompt analysis is Claude. Its results are reliably robust, and its visualizations are effective. Integrating Claude into existing frameworks streamlines the process.

After export, uploading prompt lists to Claude lets it perform behavioral analysis, identifying data themes and trends for better prompt tracking.

Posing specific questions to Claude about customer behavior opens a treasure trove of insights. Analyzing this data reveals learning opportunities I would not have anticipated.

```json { "alt": "Spreadsheet listing email marketing platform and brand comparison prompts with categories and audience segments.", "caption": "Explore this detailed spreadsheet for email marketing platform recommendations and brand comparisons, tailored to specific audience segments.", "description": "This image shows a spreadsheet containing prompts for tracking various email marketing platform recommendations and brand comparisons. It is divided into categories such as 'Platform Recommendations' and 'Brand Comparisons'. Columns include 'Category', 'Prompt to Track', 'Audience Segment', 'Intent Type', and 'Why Track This'. The spreadsheet is aimed at helping businesses choose the right email marketing tools by segmenting choices based on needs like SMB, B2B SaaS, and Nonprofit requirements." } ```

For instance, I discovered searches probing a PR issue from over three years ago are still frequent and that searches often use one company as a benchmark against its competitors.

Finally, leveraging Claude to suggest new prompt-tracking methods, based on this data, offers an informed way to continually hone tracking efforts.

While there’s no definitive system for selecting which prompts to track, incorporating Search Console data provides a clearer direction. The insights derived can help unearth unique user prompts and discern scalable themes for ongoing data tracking.

This piece originally appeared on the Nectiv blog [as How To Mine Google Search Console For Conversation Data (Regex Included)] and is republished with permission.

Inspired by this post on Search Engine Land.

crushpress.ai community screenshot

FAQs

What is the main focus of this article?

This article explains how to uncover AI search prompts in Google Search Console and how to track prompts using practical methods. It also discusses data transparency considerations and limitations.

What technique does the article propose to surface prompt-like queries?

The article suggests filtering queries with 10 or more words using regex to surface longer, prompt-like queries. It provides steps to apply this filter in Search Console.

Which tool is mentioned as preferred for prompt analysis?

Claude is the preferred tool for prompt analysis mentioned in the article. The author notes that exporting prompt lists to Claude allows behavioral analysis and helps identify data themes.

What does the article say about data transparency and AI data in Search Console?

The article notes there is no reliable public data source for prompt tracking and cites UK CMA proposals about data transparency. It also mentions AI Mode data being available in Search Console and discusses how data can reveal trends.

What steps are outlined to access prompt data in Search Console?

The article provides a four-step approach: go to Performance > Search Queries, add a Query filter, choose Custom Regex, and enter a regex pattern to surface longer prompts. This helps reveal prompt-like behavior in user queries.

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *

More posts