Free course · 13 lessons · Advanced
RAG in Production
Build retrieval-augmented generation that finds the right text and answers faithfully: structure-aware chunking, hybrid search with rank fusion, permission filters and tenant isolation, reranking, grounded answers with verified citations, context assembly, conversational query rewriting, index freshness, evaluation, prompt injection through documents, and cost and latency. Every lesson starts from a real failure and has a runnable Python simulation.
1. Retrieval That Finds the Right Text
Most bad RAG answers are retrieval failures: how documents are chunked, searched, filtered and ranked.
- Chunking: Why the Answer Was Split in Half — Fixed-size chunks cut sections, tables and lists apart; structure-aware chunks with context keep answers whole.
- Embedding Search That Misses Exact Terms — Vector search finds meaning but misses error codes and product names; hybrid search with keyword scoring and rank fusion catches both.
- Filters, Permissions and Tenant Isolation — Filtering after retrieval leaks other customers' documents or returns nothing; filters belong inside the search.
- Reranking: Top-k Is Not Top Quality — The best passage sits at rank 14 and never reaches the model; retrieving wide and reranking narrow fixes it.
2. Grounded Answers from Retrieved Text
Turn retrieved passages into answers users can trust: citations, context assembly, conversational queries and fresh data.
- Grounded Answers and Verified Citations — The model mixes retrieved facts with its own memory; quote-based citations you check in code keep answers honest.
- Lost in the Middle: Assembling the Context — Forty near-duplicate chunks in random order bury the answer; deduplicate, order, label and budget the context.
- Conversational Queries: "What About the Pro Plan?" — Follow-up questions retrieve nothing on their own; rewriting them into standalone queries restores retrieval.
- Stale Answers: Keeping the Index Fresh — Re-indexing everything nightly is slow, and deleted pages keep answering; incremental sync with content hashes keeps the index honest.
3. Operating RAG Systems
Measure retrieval and answer quality, defend against injected documents, control cost and latency, and know when RAG is the wrong tool.
- Measuring Retrieval and Answer Quality — "It seems better" is not a metric; an evaluation set with recall@k, MRR and faithfulness checks makes changes safe.
- Prompt Injection Through Retrieved Documents — Anyone who can write a document your system indexes can write instructions to your model; treat retrieved text as data.
- Cost and Latency of a RAG Pipeline — Embedding calls, oversized contexts and an uncached prompt make each answer slow and costly; measure each stage and fix the biggest.
- When RAG Is the Wrong Tool — "How many orders did we refund last month?" cannot be answered from five chunks; route aggregate and structured questions to tools.
4. Capstone
Design a production RAG system end to end.
- Capstone: A Support Knowledge Assistant for 40,000 Articles — Design a multi-tenant RAG assistant over help-centre articles and customer-specific documents, with evaluation and safety.