AI query expansion for better retrieval

Customers rarely use the same wording as your help center. A customer asks how to "end my plan," while the article is titled "Cancel a subscription." Query expansion gives the retriever more than one way to look for that article.

It is useful, but it is not a blanket accuracy upgrade. A weak rewrite can broaden a precise question until the wrong document looks relevant. The right goal is simple: improve retrieval on a fixed evaluation set without increasing unsupported answers.

In brief

• Query expansion creates a small set of alternate searches for one question.
• Each rewrite must preserve the original intent, entities, dates, codes, and account terms.
• Search every version, merge the ranked lists, then let the answer model use only the retrieved sources.
• Compare the expanded retriever with your current one on labeled support questions before releasing it.

What query expansion changes

A basic retriever searches once:

Customer question -> search -> relevant passages -> answer

An expanded retriever adds a rewrite step:

Customer question -> original query + careful rewrites
 -> search each query
 -> merge and rerank passages
 -> answer from the selected sources

For this question:

How do I end my plan before renewal?

A useful expansion might be:

cancel subscription before renewal
stop plan from renewing
subscription cancellation policy

Keep the original query in the search set. It often contains the most important exact term.

Preserve details that change the answer

The rewrite step should not replace identifiers or invent context. Protect these values:

• Product and plan names
• Error codes and version numbers
• Dates, amounts, and locations
• Negation, such as "do not cancel"
• User intent, such as a policy question versus a request to take action

"Why was invoice INV-1048 charged twice?" should still contain INV-1048 in every search that needs it. A general rewrite such as "billing problem" throws away the best retrieval clue.

A small implementation

Start with two or three rewrites, not a long list. More searches add cost and can pull in unrelated passages.

type SearchResult = {
 id: string
 score: number
 text: string
}

async function retrieve(question: string) {
 const rewrites = await rewriteQuestion(question, {
 count: 3,
 preserveEntities: true,
 })

 const queries = [question,...rewrites]
 const resultLists = await Promise.all(queries.map(query => searchKnowledgeBase(query, { limit: 8 })))

 return reciprocalRankFusion(resultLists).slice(0, 8)
}

The names are illustrative. The important choices are visible in the code: retain the original question, cap the number of rewrites, search in parallel, and limit the final context.

Merge results by rank

Scores from keyword and vector search are not directly comparable. Reciprocal Rank Fusion, or RRF, combines result positions instead of raw scores.

function reciprocalRankFusion(lists: SearchResult[][], k = 60) {
 const scores = new Map<string, { result: SearchResult; score: number }>()

 for (const list of lists) {
 list.forEach((result, index) => {
 const current = scores.get(result.id)
 const score = (current?.score? 0) + 1 / (k + index + 1)
 scores.set(result.id, { result, score })
 })
 }

 return [...scores.values()]
.sort((a, b) => b.score - a.score)
.map(item => item.result)
}

RRF is a practical starting point because one unusually large provider score cannot dominate the merged list. You still need to tune and test it on your content.

Test retrieval before answer quality

Create a set of real or representative support questions. For each one, label the passage that contains the answer. Include:

• Conversational wording that differs from the docs
• Exact error codes, SKUs, and policy names
• Short, ambiguous questions
• Questions with negation
• Questions that should return no answer

Run the set through the current retriever and the expanded version. Measure whether the labeled passage appears in the top results. Then inspect false positives. A higher hit rate is not useful if precise queries now retrieve broad but incorrect policy pages.

After retrieval passes, evaluate the complete answer. Check citation support, refusal behavior, and handoff rules. The answer model cannot repair a bad source set reliably.

When not to expand

Skip or narrow expansion when the query already contains a strong exact match, such as an error code or order identifier. It can also be the wrong choice for voice flows with a strict response-time budget, or for a small knowledge base where plain hybrid search already finds the correct passage.

Do not expand a request into actions. "Can I cancel?" is a policy question. "Cancel my account" is an action request that needs authentication, confirmation, and an approved tool.

A sensible release sequence

1. Build a labeled retrieval set from your support topics.
2. Add a small number of intent-preserving rewrites.
3. Merge the results and keep the source limit fixed.
4. Compare retrieval and answer quality with the current system.
5. Release to a small traffic sample and log the original query, rewrites, selected sources, and handoff result.
6. Disable expansion for query classes where it makes results worse.

Query expansion earns its place when it fixes a measured wording problem. It should not become another opaque model call in the path to every answer.

For the adjacent retrieval decisions, read hybrid search explained and how to prevent unsupported answers.

Frequently asked questions

What is AI query expansion?

Query expansion creates alternate searches that preserve the meaning of the customer's question. The retriever searches the original and rewritten versions, then merges the results.

How many rewrites should I generate?

Start with two or three. Test more only if your labeled retrieval set shows a clear benefit that justifies the extra latency and noise.

Does query expansion stop hallucinations?

No. It can improve the source set, but the answer still needs grounding, citation checks, refusal rules, and a human handoff for unsupported requests.

Should every question be expanded?

No. Exact identifiers, short latency budgets, and already reliable searches are good reasons to keep the original query only.