AI Slop Detection: Prove Quality With Content Provenance

You ran a page through an AI detector. It returned a high probability of machine-generated text. Now you have to decide whether to rewrite the page, remove it, disclose AI use, or ignore the score.

Do not make that decision from the score alone. AI detection, slop detection, content quality, and provenance answer different questions. Treating them as interchangeable can make you discard useful work, preserve polished nonsense, or spend hours rewriting text without improving what readers receive.

Stop asking one detector to answer four different questions

The first step is to separate four concepts that are often collapsed into one label:

• AI detection estimates whether a model may have generated or transformed text. It does not determine whether the text is accurate, useful, original, or fit to publish.
• Watermark detection looks for a signal deliberately introduced during generation. A positive result indicates that a participating system likely touched the output. It does not reveal how much was generated, what was edited, or whether a qualified person approved it.
• Slop detection is an attempt to identify low-value, repetitive, manipulative, or mass-produced material. Slop is an outcome, not an authorship category. Humans produced commodity content long before generative AI existed.
• Content provenance is the evidence trail behind a published asset: where its claims came from, who created and changed it, what automation did, how it was checked, and who accepted responsibility for publication.

These distinctions matter because the signals are imperfect. Text-watermark detectors generally need enough material to observe a pattern. Published benchmarks put the workable floor at roughly 100 tokens in favorable conditions, while SynthID evaluations truncate samples to 200 tokens. Short comments, titles, summaries, and rewritten excerpts may fall below that floor.

Editing creates another limitation. Paraphrasing, translation, model chaining, and combining marked output with other text can weaken or remove a watermark. A paraphrasing attack presented at ICML 2025 achieved nearly 100% success against seven watermarking methods at a reported cost of $0.88 per million tokens. Open-weight models add a more fundamental gap: watermarking is applied by the sampling pipeline, so someone running a model independently can omit that step.

This produces two dangerous errors. A false positive can send a strong page into unnecessary rewrites. A false negative can give weak or fabricated material an undeserved pass. Even a system reported at 94% accuracy can make consequential mistakes when it operates across enormous volumes, especially when you do not know the evaluation set, class balance, or error distribution.

Use detection as a routing signal. A high score can send a page to closer editorial review, but it should never be the reason the page fails. Make the final decision with four questions: Is the page accurate? Does it contribute something distinct? Can its important claims be traced? Is a named person accountable for it?

Distribution systems are reacting to low-value supply

Generative tools have made production cheap. They have not made attention abundant. When thousands of interchangeable assets can be produced in the time previously required for one, distribution systems become stricter selectors.

Platforms are responding at several points in that supply chain:

• Identification and reduced reach: LinkedIn has deployed systems intended to identify suspected slop, restrict its distribution beyond the creator’s immediate network, and collect user reports.
• Labels and user controls: Pinterest labels AI-modified material and lets people reduce it within selected categories. TikTok requires labels, uses invisible metadata, and has introduced controls intended to limit AI content.
• Spam and manipulation defenses: Reddit applies AI-based detection to manipulated and spammy activity rather than treating every AI-assisted contribution as the same problem.
• Economic penalties: YouTube can demonetize repetitive, low-effort, emotionally manipulative video. In January 2026, it terminated 11 channels and removed six more, erasing channels with about 4.7 billion lifetime views and 35 million subscribers.
• Supply removal: Spotify has removed more than 75 million spammy tracks while adding impersonation rules, spam filtering, and disclosure through music credits.

The implementations differ, but the operational lesson is consistent: publishing more units does not guarantee more distribution. A system may label an asset, suppress it, remove its monetization, filter it from recommendations, or delete it as spam. The marginal cost of production may approach zero while the cost of selection keeps rising.

None of this proves that search engines or frontier models apply a universal penalty to anything touched by AI. It shows that platforms increasingly act against repetition, manipulation, undisclosed synthetic media, and low-value supply. Do not turn that observation into an imaginary ranking factor. Turn it into a stricter publishing standard.

A page deserves publication when it performs a specific job that another page on your site does not already perform. It should resolve the promised question, support material claims, make uncertainty visible, and give the reader a usable next step. If you cannot name its distinct contribution in one sentence, producing another variation will increase inventory without increasing value.

Run a slop audit that measures usefulness, not writing style

Most detector-led cleanups begin at the wrong end. Teams scan thousands of URLs, sort by an AI probability, and rewrite whatever appears most synthetic. That process optimizes the detector’s reaction. It does not tell you whether the revised page deserves attention.

Use the following audit instead.

1. Write down the page’s job. Record the intended reader, the question or decision that brought them there, and the action they should be able to take afterward. If the job is unclear, the page cannot be evaluated coherently.
2. Identify the distinct contribution. Look for an original observation, a precise definition, a decision rule, a useful constraint, a first-party example, a sourced fact, or a synthesis that removes work for the reader. A topic is not a contribution. Neither is a fresh arrangement of familiar sentences.
3. Check every consequential claim. Mark statistics, dates, product behavior, legal obligations, quotations, named entities, and strong causal statements. Each one needs an appropriate basis. If the evidence cannot be recovered, soften the claim, replace it, or remove it.
4. Inspect the page as part of a collection. Compare it with assets targeting adjacent intents. Repeated introductions, interchangeable sections, overlapping target queries, and multiple pages with no independent purpose are stronger slop indicators than a model’s preferred punctuation.
5. Assign an accountable owner. A byline is not enough if no one checked the substance. Record who drafted, edited, verified, and approved the page. One person may fill several roles, but responsibility should still be explicit.
6. Choose a disposition. Keep, improve, consolidate, or withdraw the page based on reader value and evidence. Do not add a fifth category called rewrite until the detector turns green.

Your audit sheet only needs a small set of fields: URL, intended query or task, audience, distinct contribution, consequential claims, evidence status, overlap, owner, reviewer, last substantive update, and disposition. Add the detector result in a separate field if you use one. Keeping it separate prevents the score from masquerading as an editorial verdict.

Apply the dispositions consistently:

• Keep a page when it is accurate, distinct, appropriately supported, and still fulfills its intended job. An AI flag alone is not a reason to disturb it.
• Improve a page when it has a useful core but withholds the information needed to act. Replace generic explanation with evidence, constraints, examples, decision criteria, or a clearer sequence.
• Consolidate pages that repeat the same answer without serving meaningfully different intents. Preserve the strongest material, select one primary destination, and map the old URLs deliberately rather than creating another near-duplicate.
• Withdraw material that is wrong, untraceable, misleading, or functionally empty. Preserve a recoverable copy before a bulk removal and assess redirects, inbound links, and downstream references so cleanup does not create avoidable breakage.

The fastest diagnostic is subtraction. Remove the throat-clearing, generic benefits, predictable transition paragraphs, and unsourced superlatives. If nothing meaningful remains, the problem is not that the text sounds like AI. The problem is that the asset has no information payload.

When something useful does remain, edit around that value. Put the direct answer near the top. Attach evidence to the claim it supports. State who the advice is for, where it stops applying, and what could change the decision. This improves the page for readers, search systems, and answer engines without trying to reverse-engineer a detector.

Build provenance into publishing instead of adding it later

Provenance is strongest when it is captured during creation. Reconstructing it months later usually produces a folder of broken links, missing approvals, and vague memories about what the model did.

Keep a private production record

Create one record for each publishable asset. It can live in your content system, project tracker, or repository, but it should stay connected to a stable content ID or canonical URL.

• Purpose: the audience, target task, search intent, and expected reader outcome.
• People: the drafter, subject reviewer, editor, fact checker where applicable, and final approver.
• Evidence: the sources used for consequential claims, access dates where they matter, first-party data inputs, and any unresolved uncertainty.
• AI role: whether a model was used for ideation, outlining, drafting, transformation, extraction, classification, proofreading, or another defined task.
• Verification: what a human checked, which claims were changed, and what could not be independently confirmed.
• Version history: the published version, substantive updates, correction reasons, and approval status.

Record the model’s role at a useful level of detail. AI-assisted proofreading and unsupervised generation of product specifications present different risks. A single yes-or-no field hides that difference. At the same time, do not retain raw prompts or uploaded material indiscriminately. They may contain confidential information, personal data, unpublished strategy, or licensed text. Apply the same access and retention controls you would use for other production records.

A watermark can complement this record, but it cannot replace it. Anthropic announced machine-readable watermarks for Claude text and file output across its model access routes. Article 50 of the EU AI Act is a major reason model providers are moving toward machine-readable marking. That obligation concerns providers of generative systems; it does not make a marketer’s detector result a legal finding. If your organization provides or deploys a covered system in the EU, have qualified counsel assess the actual duty instead of relying on a content-scoring tool.

Publish the evidence a reader can use

Your private record establishes accountability. The public page should expose the parts that help a reader evaluate it:

• A real byline connected to a useful author profile, not an unexplained house persona.
• An accurate publication date and a modified date when the substance changes.
• A concise change note when an update corrects, replaces, or materially qualifies earlier information.
• Inline citations placed beside the claims they support.
• A methodology note for first-party tests, calculations, surveys, or datasets.
• An AI-use disclosure when the role of automation is material to interpretation, trust, rights, or platform policy.

Disclosure and provenance are not synonyms. A sentence saying that AI was used is disclosure. The chain showing what it did, which evidence informed the result, who reviewed it, and what changed is provenance. You may need both, but one cannot stand in for the other.

Structured data should mirror that visible evidence. On an Article or BlogPosting page, properties such as author, publisher, datePublished, and dateModified can make the stated identity and timing easier for machines to parse. They do not authenticate a weak byline, prove that a review happened, or turn an invented citation into evidence. Do not place claims in JSON-LD that the visible page does not support, and do not invent non-standard properties for internal provenance fields.

This is where provenance supports AI search without becoming schema theater. A frontier model or answer engine still needs a reason to select the page. Give it compact, attributable claim-and-evidence pairs; stable names for people, organizations, products, and concepts; a direct answer before elaboration; and a visible record of substantive updates. Consolidate interchangeable pages so the strongest evidence is not scattered across thin variants.

Provenance cannot guarantee rankings, citations, or inclusion in an AI-generated answer. It makes a more defensible asset available for selection. That is the useful goal: not proving that no machine ever touched the words, but showing why the result deserves to be trusted and distributed.

Key takeaways

• An AI score estimates origin patterns; it does not measure truth, usefulness, originality, or accountability.
• Watermarks can indicate that a participating model touched enough text, but editing, paraphrasing, translation, short samples, and unmarked open-weight pipelines limit what they can prove.
• Use detectors to prioritize human review, never as automatic publish-or-delete gates.
• Audit each page for a defined reader job, a distinct contribution, traceable claims, collection-level overlap, and a named owner.
• Capture sources, AI involvement, verification, approvals, and substantive changes while the asset is being produced.
• Keep visible content and JSON-LD consistent. Structured data exposes claims to machines; it does not create provenance by itself.

Start with five pages that matter to your business. Write down each page’s job, identify its unique contribution, trace its consequential claims, and assign an owner. You will learn more from that exercise than from rescoring your entire site, and you will have the beginnings of a provenance system that can survive the next detector, watermark, and distribution-policy change.

References

• Search Engine Land — Slop antibodies: The link between AI slop, watermarking, and commodity content