AI-powered semantic keyword research for the AI era: my BrightonSEO talk, updated for 2026

Keyword research is the starting point of almost every organic marketing project I’ve worked on. At BrightonSEO in October 2025 I argued that the way most of us still do it (export, sort by volume, filter by difficulty, done) no longer describes how people search, or how search engines and AI systems interpret what they type.

This post is the long version of that talk. It walks through the whole workflow I presented, layer by layer, and updates it to where things stand in October 2026. A lot has moved in a year, mostly in the direction the talk predicted: more machine-generated queries, more context, and less value in a keyword’s search volume on its own.

Here’s what it covers:

• why “which keyword goes on which page” is the wrong question now
• the five layers of semantic keyword research I use (entities, query paths, augmented and synthetic queries, context, and intent) and how to research each one in practice
• how it all comes together into a semantic keyword universe and content briefs
• what I’d say differently if I gave the talk today

One note before we start. I teach every step of this, with notebooks and templates, on MLforSEO (start with the keyword research guides). This post is the strategy and the reasoning. Wherever there’s a hands-on tutorial, I link to it instead of repeating it.

Keyword research has a new job

Keyword research still does what it always did. It tells you how, why and when your target customer searches for businesses like yours, and what they need, based on their searches alone. What’s changed is what we do with that information.

For years, the job was to find keywords to target on a page. Today the job is to understand three things:

• the user’s content and information needs: what they want to know, and in what form
• what your brand can actually produce: your content production abilities, not just your ambitions
• how to appear, relevantly, on different platforms: Google, YouTube, Reddit, TikTok, marketplaces, and now AI assistants

User-centric organisations don’t create content on everything, everywhere, for a given topic. (Nobody has the budget for that, and nobody wants to read it.) Instead, they use semantic keyword research and user research to decide where to appear, with what type of content, what information to include, and in what brand voice.

That’s the core difference. Traditional keyword research prioritises metrics. Semantic keyword research puts the searcher’s thoughts and actions at the centre: why they searched, what prompted it, what they expect to see, which platform they expect to get it on, what format they prefer, how their searches connect, and what they already know.

Traditional vs semantic keyword research

This comparison is the heart of the talk, so here it is as a table:

Traditional keyword research looks at…	Semantic keyword research adds…
Keywords	Entities, entity attributes and attribute values (the EAV model)
Keyword length	Query sequences and query paths
Search volume	Query augmentation and synthetic queries
Keyword difficulty	Query context and session context
Traffic potential	User search behaviour and information gain
Competition	Search intent (macro and micro) and knowledge graphs

I’m not saying search volume and difficulty are useless. I still look at both. What I am saying is that they describe demand for a string, and the systems ranking and generating answers stopped working with strings a long time ago. If your research stops at the left column, you’re planning content for a search engine that no longer exists.

The rest of the talk (and this post) runs through the right column, one layer at a time.

Layer 1: Entities and the EAV model

Entities are distinct, well-defined things: people, places, objects, ideas. Keywords are phrases that hold some value from an SEO perspective. Sometimes they overlap, but not always. A query like [shop online nike jordan air force one] contains keywords of varying importance (“shop” signals transactional intent, “online” a website or store) and entities of different types: Nike (organisation), Air Force One (product) and Michael Jordan (person).

The EAV model takes this one step further:

• Entity: the thing (dog food)
• Attribute: a property of the thing (food type, dog breed, the dog’s need)
• Variable (value): the specific value of that attribute (kibble, canned, raw; Labrador, Husky; weight loss, sensitive stomach)

Why bother? Because it gives you three things a keyword export won’t:

1. How people refer to the same concept. “Barack Obama”, “former president of the United States” and “44th U.S. President” are three strings and one entity.
2. Which attributes people search for, and their most popular values. That’s your content structure and, often, your site structure.
3. Information gaps. If the web discusses kibble and canned dog food but barely touches raw, semi-moist or chews, that’s where you can add something new.

EAV in practice

The process I showed on stage has five steps:

1. Research the search landscape for your topic.
2. Compile a database of search terms: your rankings, competitors’ rankings, tool exports, site search, autocomplete.
3. Extract entities from those terms. With Google’s Natural Language API and a Google Sheets template this takes under 20 minutes and no code (here’s the tutorial).
4. Identify the terms that co-occur with each entity, using n-grams (bigrams and trigrams) or a keyword-extraction model like KeyBERT.
5. List every pattern of entity-related terms.

Step 5 is where it gets interesting. Patterns come in two flavours: single-variant (“best dog food for {dog breed}”) and multi-variant (“best {type of dog food} dog food for {dog breed}”).

Once you have the patterns, you can map out every combination programmatically. In the dog food example there was one entity, 356 breeds, 8 food types and 8 dog needs. That’s 22,784 possible three-attribute combinations before you even get to pairs. Most of them will never show up in a keyword tool export, because the volumes are too small to be reported. Some of them are exactly what your best customers type.

I’m not saying you should build 22,784 pages. Please don’t. What I am saying is that the pattern list tells you which templates, filters, sections and FAQs your content needs, and which niche combinations are worth a dedicated piece. If you want the full framework, MLforSEO has a practical guide to the EAV model.

Why entities matter even more in AI search

Entity recognition and entity linking are used at almost every stage of LLM-based search: understanding and expanding the query, personalisation, retrieving and filtering passages, disambiguation (which “Jordan”?), and deciding what counts as the same thing. If your content refers to entities clearly and consistently, it’s easier to retrieve. If it doesn’t, you’re relying on the system to guess. I go deeper on this in entity-based search in the LLM era.

Since I gave this talk: one practical note on tooling. Google’s Natural Language API has a newer v2 entity model, but it dropped the salience score and the Wikipedia URL metadata that v1 returns (Google’s release notes). For keyword research, where I want to know which entity a query is really about and link it to a knowledge graph ID, I still use v1. I also get asked a lot whether you can “just ask ChatGPT” to extract entities. You can, for brainstorming attributes. For extraction you need to be consistent across 50,000 keywords, a purpose-built API is cheaper, faster and doesn’t make things up. I compare the options in what entity extraction is and where it fits in SEO.

Layer 2: Query sequences and query paths

Search engines process enormous amounts of data on how people search: how they refine queries, how their queries progress, how many searches they run before settling on a result, and how their behaviour changes during a session.

When you run several searches in a session, Google looks at how close they are semantically, structurally or thematically. Related queries get placed into an established sequence; an unrelated query may start a new one. When lots of people run the same sequence, that relationship gets reinforced, and it shows up on the SERP as Related searches, People Also Ask, People Also Search For and People Search Next.

Two definitions, because people mix them up:

• A query sequence is the order of queries a person runs within one search session.
• A query path is everything they search until they reach a point of interest or exhaust the topic. It can be one session or many, sequential or not, and it depends on location, the query’s wording, the person’s prior knowledge, their content preferences and their history.

In other words, a path is highly personal. You can’t see anyone’s actual path. You can, however, reverse-engineer the common ones. (There’s more on the theory in what a query sequence and query path are.)

Researching query paths in practice

I showed two approaches.

Approach 1: measure query distance. Use fuzzy matching to find, for every keyword, its most similar keywords. Do it for your ranked keywords, internal site searches, competitors’ ranked keywords and your opportunity terms. One caveat I repeated on stage: terms that look similar aren’t necessarily sequential or on the same path. They might be. Analyse, don’t assume.

Approach 2: reverse-engineer the SERP. Collect SERPs for your important keywords and pull the sequential features (Related searches, PAA, People Also Search For, People Search Next). Run your core terms through Google Autocomplete to see what Google suggests next. You can do this by hand for a few queries, in a tool like Semrush for individual searches, or at scale with an API such as DataForSEO. MLforSEO has walk-throughs for SERP feature analysis and the Autocomplete and Places APIs.

Since I gave this talk: three things are worth knowing.

• Google now groups your queries for you. In late October 2025, a few days after BrightonSEO, Google added Query groups to Search Console Insights. It uses AI to cluster queries that mean the same thing (misspellings, rephrasings, other languages), for properties with enough query volume. It’s not a query path tool, but it’s a free, first-party look at which of your queries Google treats as one need. I use it as a sanity check for my own clusters.
• SERP collection got more expensive. Just before the talk, in mid-September 2025, Google stopped supporting the &num=100 parameter, so getting 100 results now takes ten requests instead of one (Zeo explains the reporting impact). For query path research this matters less than for rank tracking, because the sequential features sit on page one. Budget for it anyway.
• Embeddings are now the easy default for “distance”. Fuzzy matching measures how similar strings look. Sentence embeddings measure how similar they mean. For query distance I now run both: fuzzy matching to catch variants and typos, embeddings to catch rephrasings that share no words. The course’s sBERT + fuzzy vs BERTopic notebook shows the trade-offs.

Layer 3: Augmented and synthetic queries

Of everything in the talk, this is the part that has aged best.

Users, search engines and now agents all augment search queries, or create entirely new ones. An augmented query adds information derived from the entities in the original query. Search [blake lively and ryan reynolds] and Google recognises two person entities, then offers refinements like movies, kids and wedding: new queries built from the entities’ attributes, whose results it can blend into the original answer.

Augmentation queries are either user-generated or synthetic. A user-initiated query is a search triggered by a human action. A synthetic query is an information request issued by software, rarely coordinated by a human, though sometimes supervised or observed by one. Synthetic queries come in two kinds:

• Simple, traditional ones: query rewrites and reformulations, autocomplete, hover and click prefetching, batch query tracking.
• Complex, orchestrated, AI-driven ones: prompt rewriting and decomposition, query fan-out, deep research, grounding, and agentic workflows.

The second list is why I think keyword research changed more in 2025 than in the previous decade. Google said it out loud: “AI Mode uses our query fan-out technique”, breaking a question into subtopics and issuing many searches at once, and Deep Search “can issue hundreds of searches” for a single report (Google, May 2025). Google’s documentation for site owners confirms that both AI Overviews and AI Mode may use query fan-out across subtopics and data sources (AI features and your website).

So a growing share of the queries that decide whether your content gets retrieved are never typed by a human, never appear in a keyword tool, and never have a search volume. That’s the “synthetic” in the talk’s synthetic queries, and it’s why I teach people to research them deliberately. MLforSEO has primers on query augmentation and synthetic queries.

Researching augmented queries in practice

1. Run synthetic query fan-outs for your keyword universe to uncover new angles and different persona profiles. This is a task where generative AI earns its keep, because you want breadth and creativity, not precision. Ask for broader, narrower, comparative, implicit and persona-specific variants of each core query.
2. Make sure your content clearly identifies and references the significant entities for your topic, so the sub-queries have something to match.
3. Understand how entities combine logically in searches. On stage I used a person entity and showed how it connects to a spouse (person), an election (event) and a birthplace (place). Each relationship is a likely augmentation, and writing with these links explicit (entity linking) helps knowledge graphs, too. Google fills knowledge graph gaps by clustering facts from different sources, flipping known facts into inverse ones, and even issuing its own queries to find missing information.
4. Incorporate thematic searches. For a festival, the theme isn’t just “tickets”. It’s reviews (YouTube, Reddit), outfit inspiration (Instagram, Pinterest, TikTok), tickets plus hotel (booking sites), things to do (TripAdvisor) and shopping (Amazon). Different platforms, one theme.

Since I gave this talk: query fan-out has gone from a niche topic to a mainstream one, helped by AI Mode rolling out to more than 180 countries and territories in English in August 2025 (Search Engine Land). The patent most people point to, Google’s “Search with stateful chat”, describes generating query variants from the whole conversation rather than just the latest message. That links layer 3 straight back to the session context in layer 4. If you need to tell real user queries apart from machine-generated ones in your own data, there’s a synthetic vs user query classification notebook for that, and I wrote about how entities power query fan-out.

Layer 4: Query context and session context

Ask Google [what’s the capital of France?], then follow up with [popular tourist attractions]. The second query never mentions France, but the results are about Paris. Google noticed the missing context and filled it in from the session.

• Query context is what the person tells you in the words of the query.
• Session context is what they tell you indirectly: the sequence of queries, how often they search the topic, their location, and other signals.

Google also personalises on past interactions and content preferences. In the talk I connected this to three situations I see in audits all the time:

1. Prolonged cannibalisation leading to drops. Find pages on your site that rank for similar keywords, flag the ones that look like intent cannibalisation and have dropped, and pull them for revision or consolidation.
2. Re-ranking because the user has already seen you. If someone has already seen a result in a similar search and didn’t choose it, it may be ranked lower the next time. Find your pages that rank for the same keywords as competitors and have dropped significantly, then revisit or consolidate them.
3. Competitors’ pages that people keep viewing. Analyse competitors’ ranked queries for semantics, find the content that’s frequently viewed across similar queries, compare it with yours, and keep your equivalents fresh and well linked.

There’s a gentler version too. If you publish lots of similar pages (tutorials, guides), look at the query patterns people use to find them. A tutorial might rank for “how to” plus a task, but also for “can’t” plus the desired action. Those are different moments and different emotions, and the page should acknowledge both. The PAA and Related searches data from layer 2 shows you the common follow-ups within a session. More on this in query context and session context, and the competitor side is covered in SERP competitor analysis with machine learning.

Since I gave this talk: context has become the whole game in AI search. Conversational interfaces carry context across turns by design, which is exactly the session context we’ve always had to infer. I liked this idea enough to build a whole talk around it in 2026, “Context (not content) is king”.

Layer 5: Search intent, beyond the four buckets

Informational, navigational, commercial, transactional. Useful, but broad. I split intent into two levels:

• Macro intent: the broad objective. Good for high-level content strategy and head terms.
• Micro intent: the specific need within it. Looking for a particular tutorial, a particular page on a site, a specific product model, or a comparison of two named products. Good for long-tail content that actually converts.

Even macro intent looks different depending on who’s searching. Transactional intent in ecommerce sounds like “buy”, “price”, “coupon”, “near me”. In SaaS it’s “pricing”, “plans”, “demo”, “free trial”, “API key”. In services it’s “call”, “book”, “portfolio”, “testimonials”. It changes by persona too: an affluent searcher, a bargain hunter and an early adopter all want to buy, but they type very different things.

Classify intent as facets, not a single label

The biggest change I’d push on any team is to stop giving each keyword one intent label and start classifying it on several facets. In the talk I showed rule-based formulas (regular expressions in Google Sheets) for six of them:

• Main intent category: informational, navigational, transactional, commercial.
• Explicit desired content format: video, infographic, template, document, webinar, podcast, ebook.
• Explicit knowledge level: beginner, intermediate, expert.
• Explicit user persona: student or educator, professional, casual user, affluent, budget-conscious, luxury, innovator.
• Implicit problem or struggle: “how to fix”, “can’t”, “won’t”, “not working”, “error”.
• Context: urgency (“now”, “today”, “asap”) and mood (“frustrated”, “overwhelmed”, and yes, the swear words).

A keyword that’s informational, video-format, beginner-level and frustrated needs a very different piece of content from one that’s informational, template-format and expert-level. The single label hides that. The full formulas and how to reduce the “unclassified” pile are in how to structure and categorise keyword data.

When rules aren’t enough: two ML approaches

Rules are transparent and cheap, but they only catch what you anticipated. So I showed two hybrid approaches:

1. Transformer-based: rule-based patterns on the query, then logic-based classification on the SERP features Google shows for it, then a transformer model for the final label.
2. Semantic parsing (entity-based): extract entities from each keyword, analyse them, then combine the entity data with heuristics to produce a label. Entities become weighted signals within the query’s wider meaning, not just flags.

Both approaches, with their pros and cons, are explained in search intent classification with machine learning, and the theory is in the search intent guide. Each approach also has its own notebook in the MLforSEO resource library.

Since I gave this talk: zero-shot classification with an LLM has become cheap enough that people skip straight to it. My advice hasn’t changed: rules first, because they’re auditable and free; an ML or LLM step for the residue the rules can’t place; then a sample checked by a human. If intent labels drive which pages you build, they’re mission-critical, and mission-critical tasks need consistent, explainable results.

Putting it together: from keyword list to semantic keyword universe

The five layers aren’t separate projects. They’re columns in the same database. By the end of the process, your workbook should hold, at a minimum: categorised keywords, topics with descriptions, keywords mapped to topics, keyword suggestions, entity data, entity relationships, SERP feature analysis, Google and YouTube autosuggest terms, query paths from PAA and suggested topics, and bulk SERP data for competitor analysis.

Why so much? Because the point isn’t the spreadsheet. It’s being able to produce semantically informed content briefs at scale, and a brief is only as good as the data behind it.

I closed the talk with a six-step starter plan, and I still stand by it:

1. Create the most comprehensive list of keywords you can: exports, entity extractions, programmatic EAV combinations, synthetic fan-outs.
2. Break them down and categorise them on as many facets as you can.
3. Group them per topic, per entity, per platform, per persona and per context.
4. Turn them into briefs, with different briefs for different platforms.
5. Assess competition, your content production capability, user needs and fit to brand.
6. Choose the ones that serve you most, supporting the complete user journey across platforms.

Steps 5 and 6 are where most teams cut corners, and they’re the ones that stop you from creating content on everything, everywhere. If you’re wondering what “good” looks like when you hand this over to a client or a content team, I wrote what a good semantic keyword universe looks like and how to turn it into an actionable SEO workflow. There’s also a free checklist for moving from a traditional to a semantic keyword universe and a content brief template that maps each brief section to its data source.

What I’d say differently in 2026

A year on, the framework holds. The emphasis would shift in three places.

Synthetic queries are no longer the exciting bonus section. In October 2025 I presented them as the newest layer. Now they’re the core of the method. If I gave the talk again, layer 3 would come first, because it’s the layer that tells you what AI systems will search for on your behalf.

Search volume is an even weaker guide than it was. The num=100 change shook up impression data in Search Console and rank trackers. AI Mode traffic is counted inside the Web search type in Search Console rather than reported separately (Google). Fan-out queries have no volume at all. Use volume to size demand, never to decide what’s worth understanding. On the bright side, Search Console now gives you a branded vs non-branded filter and query groups for free, so your first-party data got better even as third-party data got noisier.

Don’t hand the whole thing to an LLM. It’s tempting to paste a seed keyword into a chatbot and ask for “a semantic keyword strategy”. You’ll get something that looks plausible and is impossible to audit. Use generative AI where you want breadth (fan-outs, attribute brainstorming, brief drafts). Use purpose-built models and rules where you need precision (entity extraction, classification, clustering). And keep a human deciding what’s worth producing.

Want to go deeper?

I had a confession at the end of the talk: everything above is only part of it. Information gain scoring, knowledge graph entries and expansion, ML keyword clustering, the patents behind all of this, and dozens of practicals that get the process working at scale with APIs didn’t fit into a conference slot.

They’re all in my Semantic ML-enabled Keyword Research course on the MLforSEO Academy: 8+ hours of lessons with labs for the Natural Language API, Autocomplete, SERP collection, intent classification and automated content briefs, plus a bonus module on queries in the AI era (synthetic queries, query expansion and AI user prompts). If this post made you want to rebuild your keyword research process, that’s the place to do it, step by step.

Frequently asked questions

What is semantic keyword research?

It’s keyword research that focuses on meaning and behaviour rather than strings and metrics. Instead of only asking “how many people search this phrase?”, it asks which entities and attributes people search for, how their queries connect within and across sessions, what context surrounds them, and what intent (macro and micro) sits behind them. The output is a semantic keyword universe you can turn into content briefs for different platforms.

Do I need to code to do this?

No. Entity extraction, n-gram analysis, fuzzy matching and rule-based intent classification can all be done in Google Sheets with templates. Code (usually a Google Colab notebook) helps once you want to work at scale, for example with SERP APIs, embeddings or transformer models. I always recommend starting with the no-code version so you understand what the code is doing.

How is this different from keyword clustering?

Clustering is one step inside it. Grouping keywords into topics is useful, but a cluster doesn’t tell you which entity attributes matter, which follow-up questions come next, which format a searcher expects or how frustrated they are. Semantic keyword research categorises each keyword on several facets first, then groups them by topic, entity, platform, persona and context.

What is query fan-out, and why does it matter for keyword research?

Query fan-out is when an AI search system breaks one question into many sub-queries, runs them, and combines the results into a single answer. Google has confirmed that AI Mode and AI Overviews may use it. Those sub-queries are synthetic: nobody typed them and they have no search volume. Researching likely fan-outs for your core topics shows you which angles and entities your content needs to cover to be retrieved.

Want me to give this talk at your event? Book me to speak.