Forward Deployed Engineering Tutorial · AI Deployments · Lesson 19 of 24
Retrieval-Augmented Prototypes
Build a retrieval-augmented generation prototype on customer documents: chunking, ranking, citations and freshness.
Why retrieval
Models don't know a customer's policies, products or history. Retrieval-augmented generation (RAG) finds the most relevant passages from the customer's documents and gives them to the model with the question, so answers are grounded and can cite sources.
The pipeline
- Ingest: collect documents with metadata (source, owner, last updated, permissions).
- Chunk: split into passages of a few hundred words, keeping headings.
- Index: embeddings in a vector store, often combined with keyword search (hybrid).
- Retrieve and rank the best passages for a question, filtered by permissions and freshness.
- Generate an answer that cites the passages, and say "I don't know" when nothing relevant is found.
import math
import re
from collections import Counter
docs = {
"travel-policy-2026": "Employees may book economy flights. Business class is allowed for flights over 8 hours.",
"travel-policy-2021": "All flights must be economy class. No exceptions.",
"expenses-faq": "Submit receipts within 30 days. Hotel limit is 180 EUR per night in most cities.",
}
updated = {"travel-policy-2026": 2026, "travel-policy-2021": 2021, "expenses-faq": 2025}
def tokens(text):
return re.findall(r"[a-z0-9]+", text.lower())
df = Counter(t for d in docs.values() for t in set(tokens(d)))
def score(query, text):
tf = Counter(tokens(text))
return sum(tf[t] * math.log(1 + len(docs) / df[t]) for t in tokens(query) if t in tf)
query = "can I fly business class on a 10 hour flight"
ranked = sorted(docs, key=lambda d: score(query, docs[d]), reverse=True)
print("ranked:", ranked)
current = [d for d in ranked if updated[d] >= 2025]
print("after freshness filter, cite:", current[0])
Output
ranked: ['travel-policy-2026', 'travel-policy-2021', 'expenses-faq'] after freshness filter, cite: travel-policy-2026
Most RAG failures are retrieval failures (stale, duplicate or missing documents), not model failures; measure retrieval separately.
Test yourself
A RAG assistant quotes an outdated policy. What's the most likely fix?
Use a bigger modelImprove retrieval: remove or filter stale documents using metadataIncrease temperatureShorten the prompt
Show answer
Answer: Improve retrieval: remove or filter stale documents using metadata
The model answered from what it was given; fix what retrieval returns.