Sign in Get started

Forward Deployed Engineering in Practice · AI Deployments in the Field

Confidently Wrong Answers From the Wiki

An internal assistant cited a policy that was replaced two years ago. Fix retrieval: curate sources, add metadata filters, rank fresh documents, and say 'I don't know'.

Create a free account to save this page, take notes and track your progress. Sign in

Scenario

Example deployment: an FDE builds a retrieval-augmented assistant over a company's intranet: 40,000 pages including HR policies, IT guides and team wikis. The pilot with 50 employees goes well, so it rolls out to 6,000.

What you see in production

An employee asks about parental leave and gets a confident, well-written answer with a citation, from a 2021 policy that was replaced in 2024. Others get answers mixing two country policies, and a contractor sees a summary of an HR page they shouldn't have access to. HR asks for the assistant to be switched off.

The naive code

chunks = split_every_500_tokens(all_intranet_pages)       # no metadata: no dates, countries, permissions
index.add(embed(chunks))

def answer(question):
    context = index.search(embed(question), k=5)          # top-5 by similarity only
    return llm(f"Answer using this context:\n{context}\n\nQuestion: {question}")

Reproduce it

Similarity alone can't tell old from new, or one country from another; metadata can:

Retrieval with and without metadata filtersTry it yourself
docs = [
    {"id": "hr-leave-2021", "text": "parental leave is 12 weeks paid", "country": "DE", "year": 2021, "status": "superseded", "acl": "all"},
    {"id": "hr-leave-2024", "text": "parental leave is 16 weeks paid plus 4 weeks flexible", "country": "DE", "year": 2024, "status": "current", "acl": "all"},
    {"id": "hr-leave-us", "text": "parental leave is 8 weeks paid", "country": "US", "year": 2025, "status": "current", "acl": "all"},
    {"id": "hr-cases", "text": "parental leave disputes and individual cases", "country": "DE", "year": 2025, "status": "current", "acl": "hr-only"},
]

def similarity(query, text):
    q, t = set(query.split()), set(text.split())
    return len(q & t) / len(q | t)

def retrieve(query, user, use_metadata):
    candidates = docs
    if use_metadata:
        candidates = [d for d in docs if d["status"] == "current" and d["country"] == user["country"]
                      and (d["acl"] == "all" or d["acl"] in user["groups"])]
    return sorted(candidates, key=lambda d: similarity(query, d["text"]), reverse=True)[:2]

user = {"country": "DE", "groups": ["employees", "contractors"]}
for use_metadata in [False, True]:
    hits = [d["id"] for d in retrieve("how many weeks parental leave paid", user, use_metadata)]
    print(f"metadata filters {'on ' if use_metadata else 'off'}: {hits}")

Output

metadata filters off: ['hr-leave-2021', 'hr-leave-us']
metadata filters on : ['hr-leave-2024']

Root cause

The system indexed everything without metadata (status, effective date, country, audience, permissions), so retrieval ranked passages only by similarity. Superseded and current documents look alike, similar policies for different countries compete, and permissions weren't enforced at retrieval time. The model did what it was asked: answer fluently from the context it got. Nobody measured retrieval quality separately from answer quality.

The fix

  1. Curate sources with the content owners: which spaces are authoritative, which are archives; exclude or down-rank archives.
  2. Index with metadata (owner, effective date, status, country, audience, ACLs) and filter by it: current documents, the user's country, and only what the user may see (permission-aware retrieval).
  3. Rank for freshness and authority in addition to similarity (hybrid keyword + vector search, re-ranking).
  4. Answer with citations and dates, and say "I couldn't find a current policy" when retrieval confidence is low.
  5. Evaluate retrieval with a question set and the documents that should be found (hit rate, precision), separately from answer grading.

Trade-offs

  • Metadata needs content-owner involvement and ongoing hygiene; it's cheaper than lost trust.
  • Strict filters can hide useful context; show the user which filters applied.
  • Permission-aware retrieval must stay in sync with the source system's ACLs; sync them, don't copy them once.

Detect it in production

  • Feedback buttons with reasons (outdated, wrong, not allowed to see); review them weekly with content owners.
  • Retrieval evaluation in CI on a curated question set.
  • Alerts when answers cite documents older than a threshold or marked superseded.

Key points

  • Most RAG failures are retrieval failures: stale, duplicated, wrong-audience or unauthorised documents.
  • Index with metadata and filter by status, freshness, audience and permissions.
  • Evaluate retrieval separately and let the assistant say it doesn't know.

Practice questions

A RAG assistant gives fluent answers from outdated documents. What do you fix first?

Answer

Retrieval: curate sources, add metadata such as status and effective date, filter or down-rank superseded content, and evaluate retrieval hit rate separately from answer quality.

How do you stop an internal assistant from revealing documents a user can't access?

Answer

Enforce permissions in retrieval: index ACLs from the source systems, keep them in sync, and only retrieve chunks the requesting user may see. Never rely on the prompt.

Why might adding more retrieved chunks make answers worse?

Answer

Irrelevant or conflicting context distracts the model, and models use information in the middle of long contexts less reliably; precise retrieval beats more retrieval.

Open in the app to mark lessons complete, take notes and track your progress.

Was this page helpful?