The anatomy of an LLM prompt response | STAT Search Analytics

The anatomy of an LLM prompt response

LLM prompt responses are not like traditional search results. We break down their structure, what powers them, and the shifts in tactics and measurement they require.

SEOs have been dissecting traditional search results for many years, and have formulated well-established strategies to appear in them. LLM prompt responses, however, don’t follow the same playbook.

While prompt responses bear some similarities to a standard SERP, there are also stark differences.

In this post, we dig into the anatomy of a prompt response and examine how it impacts AI search tactics and measurement.

How are people searching?

Before we even get to responses, prompts themselves tend to look quite different to traditional keywords. Searchers are unlikely to use AI tools such as ChatGPT, Gemini, or AI Mode in the same way as they would Google Search.

Prompts are typically very specific, long-tail queries and often have a conversational tone, whereas keywords can be comparatively shorter and broader in nature. Of course, this is not a hard-and-fast rule, but it reflects the general behaviour.

While there are infinite possibilities, here are some examples of what a prompt could look like:

Compared to possible keyword alternatives:

Since many keywords are used repeatedly by searchers, search volume can help determine which ones should be targeted and tracked. However, the majority of prompts are likely to be unique and, in that case, would have a search volume of one — so volume is not currently a useful metric in AI search measurement.

How is the response generated?

LLM prompt responses are returned in one of two forms — “in-model” or “out-of-model” responses.

“In-model” responses rely solely on an LLM’s original training data. These responses are often generated from “zero-shot prompting," which is when a prompt contains instructions without specific examples, and requires the LLM to respond independently. An example of this would be when an LLM is prompted to translate or summarize text.

However, for more complex prompts, an in-model response may not be sufficient and, depending on the model, training data could be outdated at the time of your query.

“Out-of-model” responses are generated when LLMs use grounding queries — that is, when they perform one or more (usually Google) searches to construct an answer.

Grounding queries generally allow for more diverse, topical content to appear in a response. For example, Google uses a “query fan-out” method that breaks a complex, long-tail query into multiple related web searches, then merges the results to create a response that addresses the full scope of the original query. These additional queries can provide insight into the types of prompts you should track.

Using a similar methodology, our prompt generation tool assists with prompt research by breaking out chosen topics into suggested prompts to track.

What does a response look like?

Prompt responses are often text heavy and may include multiple sections of text before an image or another visual element is shown.

In a recent study, we looked at the features that currently appear on AI Mode SERPs. We found that the paragraph feature is by far the most prevalent, and shows up on almost 100% of SERPs.

Here’s an example of the paragraph feature in AI Mode:

The second most common feature, lists, is also text-based:

Cards are the most organic-like result on AI Mode SERPs, and are the third most common result type — appearing on 62.57% of the SERPs in our dataset:

Like their organic equivalent, cards present an opportunity for SEOs to target specific results that appear in search. However, that opportunity is limited by their tendency to show up at the lower end of a response.

One crucial distinction is the difference between mentions and citations in prompt responses.

Mentions are when your brand or brand terms appear in the response, while citations are when your content is used to help generate the response, and your site appears as a clickable link.

For the time being, traffic potential from an LLM response is lower than from a standard SERP, making brand visibility even more important. As such, contrary to traditional search — where the goal is to bring traffic to your site — your primary objective in AI search is to have your brand mentioned. Earning that citation is currently a secondary goal.

While being cited in a response is beneficial, the cited URL doesn’t necessarily have to be from your own site — as long as your brand is mentioned in a meaningful, positive way.

What tactics can you use to appear in responses?

When it comes to GEO tactics for appearing in prompt responses, a lot of what SEOs have already been doing is still relevant, but the priorities have shifted.

Since brand mentions are the primary focus, your tactics need to be properly aligned to achieve them.

Barnacle SEO — the practice of attaching yourself to bigger sites with greater potential to rank — is a key part of this. While you should still optimize your own site for appearing in responses, you should also focus on profiles and content under your direct control on other sites.

Another area to adjust your focus in is technical SEO. Work to improve internal linking, discovery, and indexing is still important, but technical SEO also needs to be a hygiene factor in GEO. This means getting back to basics and ensuring your site’s content can be easily ingested by AI agents — increasing your chances of featuring in LLM prompt responses.

How can you measure brand performance?

As we outlined earlier, prompts are not like keywords — so you shouldn’t track the same search terms. Instead, track plausible, long-tail prompts that are topically relevant to your brand.

The metrics for measuring your brand’s visibility in prompt responses also need to be fit for purpose. In our AI brand visibility tracking tool, two metrics we surface are the average depth and presence of brand terms. These help you understand how early and how often you’re showing up.

Start digging into prompt response data

Now that you’ve got a better understanding of prompt responses and the tactics and measurement practices that go along with them, it’s time to begin tracking your own prompts and analyzing their responses.