Mastering Effective SEO Agent Skills: A Personal Journey

I’ve been on a journey to develop over 10 SEO agent skills in just 34 days. Six of these succeeded on the first attempt, while the remaining four taught me invaluable lessons, especially about the overlooked importance of folder structure that many LinkedIn posts on AI SEO skills seem to miss.

The reliability of these agents isn’t about crafting superior prompts; it lies in the architecture that supports them. Here’s my blueprint for building an agent from scratch, testing it diligently, refining it, and deploying it with full confidence.

Here’s why many AI SEO skills don’t make the cut.

A typical AI SEO prompt seen on platforms like LinkedIn usually looks something like this:

You are an SEO expert. Analyze the following website and provide a comprehensive audit with recommendations.

And that’s where it ends. One simple prompt, often coupled with some formatting directions, is shared with the world. The post then earns hundreds of likes, yet the output—while polished—is often up to 40% inaccurate.

I know because I’ve been there. Initially, I tasked an agent to identify SEO issues on a website, and while it came back with 20 findings, eight were non-existent. The agent hadn’t truly visited many of the reported URLs.

Here are three key issues that doom single-prompt skills:

• No tools: The agent can’t physically verify the website; it relies on training data to guess. Queries about canonical tags, for instance, result in assumptions rather than real-time analysis of HTML.
• No verification: There’s no check on the truthfulness of output. An agent might report missing meta descriptions across 15 pages, but without verification, we don’t know if these pages are even indexed correctly or intentionally set as noindexed.
• No memory: The agent’s feedback varies wildly with each use, showing inconsistency due to the lack of a template or structured history of previous runs.

In essence, if your skill is just a prompt within a lone file, you’ve got a 50/50 chance at best.

Every agent in my system has a dedicated workspace. Consider it akin to a new employee’s desk, equipped with all necessary resources. For example, our agent designed to crawl and map website architecture works within this kind of structured environment:

agent-workspace/
  AGENTS.md          instructions, rules, output format
  SOUL.md            personality, principles, quality bar
  scripts/
    crawl_site.js    tool the agent calls to crawl
    parse_sitemap.sh tool to read XML sitemaps
  references/
    criteria.md      what counts as an issue vs noise
    gotchas.md       known false positives to watch for
  memory/
    runs.log         past execution history
  templates/
    output.md        expected output structure

The workspace includes six key components services that just one prompt couldn’t dream of covering fully.

Within AGENTS.md, I’ve articulated a meticulous methodology comprising thousands of words. Instead of a simple instruction like “crawl the site,” I detailed each step: “Start with the sitemap; if it doesn’t exist, check various routes like /sitemap.xml, /sitemap_index.xml, and robots.txt for references.”

Scripts represent the tools the agent utilizes. Instead of writing curl commands from scratch for each crawl, the agent can run node crawl_site.js -url to analyze website data, which is far more efficient and reliable.

References consist of criteria that help the agent distinguish between significant issues and noisy false positives, using a wealth of knowledge I’ve amassed over two decades.

To ensure that every execution is informed by the past, I keep meticulous logs under memory, serving as institutional knowledge that empowers consistency across agent runs.

Through templates, I outline the exact format I expect from the output, thereby maintaining high quality across multiple iterations of the same task.

Building from scratch, the first naive attempt involved simple instructions that inevitably failed when confronted with modern CDNs. By iterating and incorporating tools like crawl_site.js, enhancing with rate limiting, and tackling JavaScript rendering, I’ve honed an architecture that delivers consistent outputs across runs.

The path involves a series of iterations where each failure metamorphoses into a permanent lesson, gradually shaping a sophisticated system. This methodically structured approach ensures that what we build is not just technically proficient but measurably better with every successive run.

Inspired by this post on Search Engine Land.