Python Keyword Clustering for an Actionable Content Plan

You do not have a keyword-volume problem. You have a page-decision problem. A long query export leaves you deciding which phrases belong on one page, which deserve separate pages, which match existing content, and which should be ignored.

A practical Python workflow can reduce that list to reviewable topic groups. The useful pattern is simple: clean the queries, represent them with TF-IDF, find natural groups with HDBSCAN, and apply editorial judgment before any cluster becomes a content brief. The algorithm handles repetition and scale; you retain control over intent, page scope, and priorities.

Decide what a keyword cluster is allowed to mean

Treat a cluster as a candidate content decision, not an automatic page recommendation. HDBSCAN can tell you that a collection of queries is densely related in the feature space. It cannot tell you whether those queries belong on a new page, an existing page, a product page, a comparison, or several separate assets.

This distinction prevents the most expensive clustering mistake: turning every machine-generated group into a URL. A useful cluster should support one dominant reader need for one recognizable audience. If the group contains people trying to learn, compare, buy, and troubleshoot, it is probably too broad even when the vocabulary overlaps.

Key takeaways

• Use clustering to reduce the review workload, not to replace search-intent analysis.
• Keep the original query beside its cleaned version so every assignment remains auditable.
• Choose TF-IDF plus HDBSCAN when you do not know the number of topics in advance.
• Expose cluster sensitivity and minimum cluster size as configuration, then tune them against editorially useful groups.
• Retain the noise label. Outliers can reveal valuable long-tail ideas, data contamination, or terms that need a different taxonomy.

Define the deliverable before writing the pipeline. For content planning, each output row should eventually answer four questions: Which cluster contains this query? What need does that cluster represent? What content action should you take? Which URL, if any, owns the topic?

That definition gives you a better quality test than cluster count. The best run is not necessarily the one with the most groups or the least noise. It is the run that makes page-level decisions clearer without concealing meaningful differences between queries.

Build a clean input without erasing useful meaning

Your clustering quality is bounded by the query list you feed it. If a Google Search Console property exports to BigQuery, you can work with query data that is not restricted to the interface’s 1,000-row export cap and is not sampled. The Search Console interface remains usable for a smaller exercise. In either case, the clustering input can be a text file containing one keyword per line.

Do not overwrite the raw phrases during cleaning. Create a working table with an original-query field and a separate normalized-query field. Cluster the normalized text, but carry the original wording into the final workbook. When a group looks wrong, this lets you determine whether the problem came from the data, the cleaning rule, or the clustering settings.

A defensible preprocessing sequence looks like this:

1. Load one query per row and remove blank records.
2. Preserve the exact original phrase in a read-only column.
3. Standardize superficial differences such as surrounding whitespace and inconsistent case in a separate working column.
4. Remove characters that are genuinely irrelevant to your dataset.
5. Apply stopword handling only after checking what those words mean in your niche.
6. Separate languages before clustering when the content operation serves them separately.
7. Deduplicate normalized phrases while retaining a path back to every original row.
8. Write excluded or unprocessable rows to a rejection log instead of silently dropping them.

Cleaning rules need editorial scrutiny. A blanket non-ASCII filter may be appropriate for a deliberately English-only run, but it can also erase valid names, accented terms, or entire languages. Stopwords can be equally treacherous. Removing a common preposition may have little effect in one dataset and destroy an important distinction in another. Test the cleaned output by reading actual before-and-after pairs.

Keep each run linguistically and operationally coherent. Combining unrelated markets, languages, or business lines forces the model to find density across data that your team would never plan together. Separate runs also make parameter tuning easier because the expected topic granularity is more consistent.

If you have useful fields beyond the query itself, retain them outside the clustering feature text and join them back afterward. A metric or business classification can help prioritize a cluster, but inserting it into the phrase changes what the text model is comparing.

Use TF-IDF and HDBSCAN when the topic count is unknown

Keyword planning rarely begins with a trustworthy answer to, “How many topics are in this file?” That makes a fixed-cluster method awkward. K-means requires you to choose the number of groups before clustering, which turns an unknown editorial outcome into a required input.

TF-IDF and HDBSCAN solve different parts of the problem. TF-IDF converts each cleaned query into a numerical feature vector. Terms that distinguish a phrase within the dataset receive more influence, while terms appearing throughout the list receive less. HDBSCAN then searches those vectors for dense neighborhoods. This pairing can discover groups without a predetermined cluster count and isolate queries that do not fit.

Organize the Python workflow into explicit stages rather than one opaque function:

1. Read and validate the flat keyword file.
2. Create raw and cleaned query fields.
3. Transform the cleaned phrases into TF-IDF vectors.
4. Pass those vectors to HDBSCAN with configurable clustering settings.
5. Attach the returned cluster identifier to every original query.
6. Generate a provisional label from the cluster’s most distinctive terms.
7. Export a cluster summary and a complete keyword-level table.

Keep configuration at the top of the notebook or script. Input path, language rules, stopword behavior, sensitivity, minimum cluster size, and output path should not be buried inside processing logic. You will rerun the model several times, and editable configuration makes those runs comparable.

HDBSCAN commonly represents unassigned queries with cluster ID -1. Do not translate that value to “bad keyword.” It means the query did not belong to a sufficiently dense group under the current settings. That can describe an unusual but valuable long-tail question just as easily as it can describe irrelevant input.

TF-IDF also has an important boundary: it is a lexical representation. It is good at identifying distinctive term patterns, but it does not automatically understand every paraphrase that uses entirely different vocabulary. Human review is still needed to reunite synonyms, separate ambiguous terms, and detect intent differences hidden behind similar words.

Your detailed export should preserve enough context to support that review:

Provisional labels are for orientation, not publication. A label made from prominent terms may name the subject while missing the searcher’s actual job. Rewrite it as a plain editorial topic only after examining representative queries.

Tune the model against recognizable content boundaries

There is no universally correct parameter set. Cluster sensitivity and minimum cluster size behave differently when the input contains 50 keywords instead of 50,000. Copying a setting without considering dataset scale and topic diversity can produce neat-looking output that is useless for planning.

Minimum cluster size controls how much local support a group needs. A larger requirement favors broader, well-supported themes and can leave niche phrases as noise. A smaller requirement allows compact long-tail groups to survive, but it can also fragment one viable topic into many tiny clusters.

Sensitivity controls how readily your implementation treats nearby phrases as one group. The exact direction and name can depend on how the notebook exposes the setting, so document what a higher or lower value does in your implementation. What matters editorially is the tradeoff: permissive grouping risks mixed intent, while strict grouping risks unnecessary fragmentation.

Use a controlled tuning loop:

1. Save the initial configuration as a named run rather than overwriting it.
2. Review the largest clusters, middle-sized clusters, smallest non-noise clusters, and a selection of -1 rows.
3. Mark groups that are coherent, too broad, unnecessarily split, or dominated by irrelevant data.
4. Change one setting at a time so you can attribute the effect.
5. Rerun the same cleaned dataset and compare assignments, not just the total number of clusters.
6. Stop when additional tuning shifts labels without improving page decisions.

A giant cluster built around a broad noun usually signals that the run is grouping too permissively or that the dataset needs to be segmented first. Several clusters differing only by minor wording usually signal excessive fragmentation. A large noise pool may mean the minimum group requirement is suppressing legitimate long-tail topics, but it can also reveal a messy source list. Read the rows before changing the model.

Do not optimize for zero noise. Forcing every query into a cluster removes one of HDBSCAN’s main advantages. The -1 set protects stronger groups from being diluted by phrases with no natural home. It also gives you a focused queue for manual classification.

Record the settings with every export. Without that record, you cannot explain why a keyword moved, reproduce an approved run, or compare whether a preprocessing change improved the result. A compact run log should identify the input file, cleaning configuration, clustering configuration, and output filename.

Convert machine groups into page-level content decisions

The content plan begins after clustering. Open each candidate group and read its queries as a set of needs, not a bag of terms. Identify the dominant question, the audience implied by the modifiers, and any phrases that change the expected answer or page type.

For every important cluster, make the following decisions:

1. Write a human topic label that describes the reader’s need rather than repeating the most frequent words.
2. Select representative queries that express the center and the boundaries of the group.
3. Check whether the queries imply one intent and one plausible content experience.
4. Inspect current search results for representative variants before committing them to one URL. If the result types or intended audiences diverge materially, split the group.
5. Compare the approved topic with existing site coverage.
6. Choose a content action: create a page, refresh an existing page, consolidate overlapping pages, add a supporting section, or defer the topic.
7. Assign one target URL when the site should have a clear owner for the cluster.
8. Record exclusions so a writer knows which adjacent needs the page should not try to satisfy.

A cluster should strengthen a brief, not become the brief. Give the writer a primary reader question, supporting subquestions, scope boundaries, relevant terminology, the intended content action, and internal-link relationships. A pasted column of keywords leaves the hardest planning work unresolved.

Use the cluster summary and keyword-level export for different jobs. The summary is the planning board: one row per reviewed topic, with its action and owner. The detailed view is the evidence: every query, its machine assignment, its cleaned form, and any editorial override. Keeping both views makes it possible to move quickly without losing traceability.

Review noise separately rather than at the end of an already long cluster sheet. Some -1 queries will be irrelevant and can be excluded. Others will be highly specific questions worth adding to an existing page, and a few may be early members of topics that need more data before they form stable groups. Record which outcome applies.

Do not let cluster size become the only priority signal. A large group may describe a broad topic your site already covers well, while a compact group may align closely with a valuable product, service, or audience need. Use the model to organize topical evidence, then prioritize with your site’s existing coverage and business goals.

Start with one coherent dataset and keep the first run deliberately provisional. Review the broadest clusters and the -1 queue, adjust one setting, and rerun. Once the groups consistently support clear page decisions, convert one approved cluster into a pilot brief. That brief will tell you more about the usefulness of the pipeline than a polished visualization ever will.

References

• Search Engine Land – How to build a keyword clustering tool with Python