Sign in Get started

RAG in Production · Retrieval That Finds the Right Text

Embedding Search That Misses Exact Terms

Vector search finds meaning but misses error codes and product names; hybrid search with keyword scoring and rank fusion catches both.

Create a free account to save this page, take notes and track your progress. Sign in

Scenario

Example company: a developer-tools company builds a docs assistant with vector search only. Questions like "how do I rotate my API key?" work well.

What you see in production

Questions with exact identifiers fail. "What does error E4012 mean?" returns articles about errors in general, never the page for E4012. Product names and configuration flags (--max-retries) have the same problem. Support tickets about these are the most common kind.

The naive code

def search(question, k=5):
    return vector_index.query(embed(question), top_k=k)           # meaning only

Reproduce it

A toy semantic score, a keyword score and reciprocal rank fusion of both:

Semantic search, keyword search and reciprocal rank fusionTry it yourself
docs = {
    "errors-overview": "Understanding errors: what to do when a request fails, retries and support.",
    "e4012": "E4012 quota exceeded for the project. Raise the quota or wait for the reset.",
    "rotate-keys": "Rotate API keys safely: create a new key, deploy it, then revoke the old key.",
}
CONCEPTS = {"error": "failure", "errors": "failure", "fails": "failure", "mean": "explain",
            "understanding": "explain", "key": "credential", "keys": "credential"}

def concepts(text):
    return {CONCEPTS.get(w, w) for w in text.lower().replace(":", " ").replace(".", " ").replace("?", " ").split()}

def semantic_rank(q):           # stands in for embeddings: overlap of concepts, not exact tokens
    return sorted(docs, key=lambda d: -len(concepts(q) & concepts(docs[d])))

def keyword_rank(q):            # stands in for BM25: only documents containing the rare exact terms
    terms = [t for t in q.lower().replace("?", "").split() if any(c.isdigit() for c in t) or len(t) > 6]
    hits = {d: sum(docs[d].lower().count(t) for t in terms) for d in docs}
    return sorted((d for d in hits if hits[d]), key=lambda d: -hits[d])

def rrf(*rankings, k=60):
    scores = {}
    for ranking in rankings:
        for pos, d in enumerate(ranking, 1):
            scores[d] = scores.get(d, 0) + 1 / (k + pos)
    return sorted(scores, key=lambda d: -scores[d])

q = "What does error E4012 mean?"
print("semantic:", semantic_rank(q))
print("keyword :", keyword_rank(q))
print("hybrid  :", rrf(semantic_rank(q), keyword_rank(q)))

Output

semantic: ['errors-overview', 'e4012', 'rotate-keys']
keyword : ['e4012']
hybrid  : ['e4012', 'errors-overview', 'rotate-keys']

Root cause

Embeddings represent meaning, and an identifier like E4012 has little meaning on its own: it looks similar to other error codes and to the word "error". Keyword methods such as BM25 score exact rare terms highly, which is exactly what identifiers need, but they miss paraphrases. Each method fails where the other succeeds.

The fix

Run both searches and fuse the results:

-- PostgreSQL with pgvector: semantic and full-text candidates, fused with reciprocal rank fusion
WITH semantic AS (
  SELECT id, row_number() OVER (ORDER BY embedding <=> :query_embedding) AS rank
  FROM chunks ORDER BY embedding <=> :query_embedding LIMIT 50
), keyword AS (
  SELECT id, row_number() OVER (ORDER BY ts_rank_cd(tsv, query) DESC) AS rank
  FROM chunks, websearch_to_tsquery('english', :question) query
  WHERE tsv @@ query ORDER BY ts_rank_cd(tsv, query) DESC LIMIT 50
)
SELECT id, sum(1.0 / (60 + rank)) AS score
FROM (SELECT * FROM semantic UNION ALL SELECT * FROM keyword) both_ranks
GROUP BY id ORDER BY score DESC LIMIT 20;
  • Reciprocal rank fusion combines rankings without having to normalise incompatible scores.
  • Keep identifiers intact in the keyword index (do not let the tokenizer split E4012 or drop --max-retries).
  • Pass the fused candidates to a reranker (next lesson but one) before the model.

Trade-offs

  • Two indexes to build and keep in sync; many databases and search engines now support both in one place.
  • Fusion weights can be tuned per query type, but tuning without an evaluation set is guesswork.
  • Keyword search needs language-aware configuration (stemming, stop words) for each language you support.

Detect it in production

  • Failed-answer review: questions containing codes, names or flags that retrieved nothing relevant.
  • Retrieval recall on an evaluation set split by query type (identifier versus natural language).

Key points

  • Vector search misses exact identifiers; keyword search misses paraphrases.
  • Hybrid search with reciprocal rank fusion gets the strengths of both.
  • Protect identifiers from tokenisation and evaluate by query type.

Further reading: Okapi BM25 · Reciprocal rank fusion (paper) · pgvector · PostgreSQL full-text search

Practice questions

Why does pure vector search struggle with error codes and product names?

Answer

Embeddings capture meaning, and identifiers carry little meaning, so their vectors are close to similar-looking codes or the generic topic. Exact-term methods like BM25 rank them correctly.

What is reciprocal rank fusion and why is it convenient?

Answer

It scores each document by summing 1/(k + rank) across result lists. It uses ranks only, so it combines searches whose raw scores are on incompatible scales.

What must you check in the keyword index for technical content?

Answer

That the tokenizer keeps identifiers such as error codes, flags and version numbers intact, and that language settings (stemming, stop words) suit the content.

Open in the app to mark lessons complete, take notes and track your progress.

Was this page helpful?