How to decide which prompts to track for a brand, and how to measure what changed

Example brand: Porsche. Porsche is not a scaile client. All examples in this article are illustrative.

What we are doing

When we talk about GEO, it is many things, but not standardized. Generative Engine Optimization is a very diverse and individual field; every brand needs individual content. Hence, we need to really understand our clients before we build something for them. We must deeply identify what a single client sells, to whom, and in which context. And all of this we need to test.

Therefore, we came up with a playbook that answers one question for each client: which prompts should we track, and how much do they matter? It is the same process for every brand, but the result is individual for every client.

We can divide this playbook into four phases: collecting the questions, planning the test, measuring, and checking what our content changed. While this article goes through all four phases, it answers seven questions.

1. What are the building blocks of a buying question?

In the beginning, we define all the offerings of a brand. Additionally, we make one group for questions about the brand in general.

For each offering, we list its first keywords in a fixed framework: variants, product types, attributes, needs and pain points, target groups (ICPs) and context.

For the 911 this looks like this: variants are Carrera, 4, S, 4S, Turbo and Turbo S. Product types are Coupé, Cabriolet and Targa. Attributes are sports car, acceleration, price, safety and fuel consumption. Needs are driving fast, showing wealth, spending a bonus or track days. Target groups are wealthy people, car enthusiasts, and commuters. Context is new, used or leased.

Why this framework? Our goal is to find all the questions people could ask. When someone has a buying question about a product, they must describe it in some form: the model, the variant, a property, a need, who they are or their buying situation. Each of these descriptions is a value in one of our categories. So, every relevant prompt contains at least one of our keywords, or a combination of several. This follows faceted classification, where classes are built from building blocks (Hjørland, ISKO). It makes our task easier: instead of finding all prompts, we have to find all building blocks.

2. How do you find real prompts?

We combine the keywords from the framework and search for real prompts with them. We start with short combinations and make them longer step by step. We do this once branded (with Porsche or 911 in it) and once non-branded. For every combination we search where people really ask: chat logs (WildChat, LMSYS), Reddit, forums, Google results and internal data from sales calls and the client.

Then we classify every text we find. First: is it a real prompt or a search query? Second: how much purchase intent does it have: none, light, mid or clear? We only keep real prompts with mid or clear intent that fit the offering.

Example:

• With “Cabriolet” (product type) we found “cabriolet daily driver”. This is a search query. Out.
• With “Cabriolet + daily driver” (product type + need) we found “I drive 40 km to work every day. Which Cabriolet is fun but still comfortable enough for that?” This is a prompt with mid intent. Stays.
• With “911 GTS” (offering + variant) we found “How fast is a 911 GTS?” This is a prompt, but without purchase intent. Out.
• With “911 GTS + first Porsche buyer” (offering + variant + ICP) we found “First Porsche, budget around 150k. Should I buy a 911 GTS or a Carrera S?” This is a prompt with clear intent. Stays.

3. How do we know that we found enough?

The first keywords are only the start. In every round we find new keywords, for example from “People also ask” and “People also search for”, and search again for real buying prompts with them. Keywords are the bait, buying prompts are the catch. We stop when a round adds fewer than 5% or fewer than 3 new buying prompts.

We tested this loop on 10 offerings with 4,009 buying prompts. The starting keywords alone found only 53% of the buying prompts the whole loop found. Most prompts come early: 85% after two rounds. Rounds after the stop would have added only 0.2%.

We also compared our method with free search by independent researchers, with the same budget (60 searches per offering), in a text archive both searched in full (Reddit, forums, chat logs). Our method found 97% of the buying prompts free search found (2,101 of 2,176), plus at least 291 that free search missed.

Keywords from the framework that never appear in a real prompt go on a gap list. It shows where buyers haven’t asked yet, or where we haven’t found them yet.

4. How do you group the prompts?

For each offering we group the prompts by meaning into buying intents. Every prompt becomes a vector, and prompts with a similar meaning are close to each other. One cluster is one buying situation, for example “daily driver” or “buying used”. Later we can see in which situations a brand is visible and in which not.

The threshold is 0.93 similarity. We tested several thresholds against the AI answers: how similar are the answers to prompts in the same cluster? From about 0.92 to 0.94 on, finer clusters hardly make the answers more similar. But we get many more clusters with only one prompt, and the clusters become less stable. At 0.93 the answers inside a cluster were about as similar as at 0.95 (answer distance 0.046 vs. 0.044), and the clusters were more stable (0.94 vs. 0.91).

One finding surprised us: even two very similar real questions get more different answers than a question and its rewording. So, a cluster is a group with the same buying intent, not a group with the same answer.

5. How many prompts do you need?

There is no general number for a company, but there is one for a precision. The error of an average gets smaller with more prompts: SEM = σ / √N. σ is the spread between prompts. From our tests it is about 0.25, 0.5 is the worst case. With 95% certainty the true value lies within about ±2 × SEM. So we need N = (σ / SEM)² prompts.

• SEM 0.05: ±10 points, 25 prompts (σ = 0.25), 100 prompts (σ = 0.5)
• SEM 0.04: ±8 points, 40 / 157
• SEM 0.03: ±6 points, 70 / 278
• SEM 0.025: ±5 points, 100 / 400
• SEM 0.02: ±4 points, 157 / 625
• SEM 0.01: ±2 points, 625 / 2,500

So for ±10 points we need 25 real prompts per offering, for ±5 points already 100. Before we test, we decide how precise each offering has to be.

6. How do you weight the results?

Weights make important situations count more, so the score shows what buyers actually ask. We use them twice. Clusters → offering: each cluster counts by its search volume (DataForSEO). Offerings → brand: each offering counts by its market weight and an individual client weight.

• Score per cluster = share of answers that name the brand.
• Score per offering = cluster scores multiplied by their weights, then added up.
• Score for the brand = offering scores multiplied by their weights, then added up. Once with the client weights, once with the market weights.

Branded prompts get their own score and are never mixed into the main score.

The weights also decide how many prompts each cluster gets. Example 911, 25 prompts for ±10 points:

• Daily driver: 50% of the search volume, 50% × 25 = 12.5, tested 13
• Buying used: 30%, 7.5, tested 8
• Track days: 20%, 5, tested 5
• Total: 26 prompts

For more certainty we first pick the number of prompts per cluster by weight and then add up, not the other way round. Taking the same number of prompts for every cluster and then applying the weights would increase the error.

We only test real prompts, no rewordings. In our tests a rewording added only about a quarter of the information of a new real question. And sometimes it measured something else: a real question about a blood pressure monitor that sends its results to an app named the brand in 20 of 20 runs, the rewording “Recommend me a blood pressure monitor with Bluetooth” only in 5 of 20. If a cluster has fewer real prompts than it needs, we test it with what it has, report a wider margin and search again for this buying situation.

7. How do you measure what changed?

We measure through an OpenAI endpoint where the model writes its own web searches (fan-out). Every prompt runs 5 times. A mention counts when the brand name is in the answer or the brand’s domain is among the cited sources.

Before the content work starts, we write down which clusters we work on. After the content work we measure again, with exactly the same prompts. Then we compare per prompt: after minus before, and per cluster the average change with its error. Clusters we did not work on are the control. If they move too, it is probably a model update and not our content.

To see a change of about 5 points, we need about 64 prompts with 5 runs per group of clusters. The change of a single prompt between two measurements spreads by about 0.20 and was never above 0.25 (378 prompts, measured three times).

We also tested if the measurement shows changes that did not happen. We measured the same 378 prompts three times without any content change (5,646 answers). The brand rate was 0.202, 0.192 and 0.192. Only 2 of 195 cluster comparisons looked significant, chance alone gives about 5%, and none after correction for multiple tests. So the method does not invent changes.

How it helps

• The right prompts. The prompt set reflects what buyers of this brand actually ask, not what we assume they ask.
• Weights that matter. Buying situations with more demand count more.
• Known precision. Every number comes with its margin, and we know how many prompts we need for it.
• Visible gaps. Keywords without real prompts show where buyers don’t ask yet, or where we haven’t found them yet.

Why it is cool

• Real questions instead of made-up prompts. Keywords only show where to look. What we track comes from real buyers.
• The same process for every client, an individual result. A sports car brand and a medical device maker end up with very different prompt sets.
• A fixed set. Because the set doesn’t change, we can compare measurements over time and see what our content changed.

Outcome

At the end, every client has:

• a map of everything the brand sells, with keywords in six categories and two weights per offering,
• a prompt set built from real buyer questions, grouped into weighted buying situations,
• the number of prompts needed for the precision we want, and how they are split across clusters,
• a gap list of keywords and situations without enough real prompts,
• a before and after measurement that shows what our content changed.

What we don’t know yet:

• There is no public data on how often a given prompt is asked. We use search volume as an approximation of demand.
• At 0.93, a blind rating found 2 of 10 to 12 clusters that mixed different buying intents. We read every cluster and mark mixed ones.
• The threshold test and the repeat measurements ran on one brand so far.
• We know the method does not invent changes. Whether it catches every real change still needs a test with a known content change.
• The answers come from an API, not from the ChatGPT app. Model updates can change answers, so we repeat measurements and record the model version.