Interview Q&A · 39 questions · 10 answered publicly
RAG Interview Questions
Ordered by how often they're asked. Sign in free to see every answer, rate your confidence and run timed mock interviews.
-
Most asked RAG leaf: How do you measure retrieval quality independently of generation?
Summary
Use labeled query->doc sets and compute recall@k, MRR, nDCG before looking at answer quality.
Longer Answer
Use labeled query->doc sets and compute recall@k, MRR, nDCG before looking at answer quality. Track citation hit rate separately from fluency.
Key Points
- Use labeled query->doc sets and compute recall@k, MRR, nDCG before looking at answer quality
- Track citation hit rate separately from fluency
- Separates retrieval vs generation failures
- Use the answer to connect the concept back to a realistic production or interview scenario for 'RAG leaf: How do you measure retrieval quality independently of generation?'
Comparison / Tradeoffs
Aspect Notes Core idea Use labeled query->doc sets and compute recall@k, MRR, nDCG before looking at answer quality. Practical tradeoff Separates retrieval vs generation failures. -
Most asked RAG leaf: Explain query rewriting and multi-query retrieval.
Summary
LLM expands/clarifies the user query or generates multiple reformulations; retrieve per query and fuse (RRF).
Longer Answer
LLM expands/clarifies the user query or generates multiple reformulations; retrieve per query and fuse (RRF). Improves recall for vague questions.
Key Points
- LLM expands/clarifies the user query or generates multiple reformulations; retrieve per query and fuse (RRF)
- Improves recall for vague questions
- Common recall booster
- Use the answer to connect the concept back to a realistic production or interview scenario for 'RAG leaf: Explain query rewriting and multi-query retrieval.'
Comparison / Tradeoffs
Aspect Notes Core idea LLM expands/clarifies the user query or generates multiple reformulations; retrieve per query and fuse (RRF). Practical tradeoff Common recall booster. -
Most asked RAG leaf: How do you prevent prompt injection via retrieved documents?
Summary
Treat docs as untrusted, delimit content, instruct model to ignore embedded instructions, and never grant tools based on doc text alone.
Longer Answer
Treat docs as untrusted, delimit content, instruct model to ignore embedded instructions, and never grant tools based on doc text alone. Security for RAG.
Key Points
- Treat docs as untrusted, delimit content, instruct model to ignore embedded instructions, and never grant tools based on doc text alone
- Security for RAG
- Relevant areas: Ai, Rag, Security
- Use the answer to connect the concept back to a realistic production or interview scenario for 'RAG leaf: How do you prevent prompt injection via retrieved documents?'
Comparison / Tradeoffs
Aspect Notes Core idea Treat docs as untrusted, delimit content, instruct model to ignore embedded instructions, and never grant tools based on doc text alone. Practical tradeoff Security for RAG. -
Most asked RAG leaf: How do you evaluate faithfulness of answers?
Summary
NLI/LLM-as-judge against retrieved context, citation coverage checks, and human spot audits.
Longer Answer
NLI/LLM-as-judge against retrieved context, citation coverage checks, and human spot audits. Fail closed when unsupported.
Key Points
- NLI/LLM-as-judge against retrieved context, citation coverage checks, and human spot audits
- Fail closed when unsupported
- Groundedness evaluation
- Use the answer to connect the concept back to a realistic production or interview scenario for 'RAG leaf: How do you evaluate faithfulness of answers?'
Comparison / Tradeoffs
Aspect Notes Core idea NLI/LLM-as-judge against retrieved context, citation coverage checks, and human spot audits. Practical tradeoff Groundedness evaluation. -
Most asked How do you choose a chunking strategy for RAG?
Summary
Chunk along document structure (headings, paragraphs, lists) and prefix each chunk with its title and heading path; keep tables and code whole; consider contextual retrieval or small-to-big retrieval when chunks lack context; and choose between options using retrieval recall on a labelled set of real questions.
Longer Answer
Fixed-size chunks split facts from their headings, so the best-matching chunk may lack the answer. There is no universal chunk size; measure.
Key Points
- Structure-aware chunks with heading prefixes.
- Keep tables and code whole.
- Contextual or small-to-big when needed.
- Decide with recall@k.
Deep dive: Chunking: Why the Answer Was Split in Half
-
Most asked Why use hybrid search in RAG, and how do you combine the results?
Summary
Vector search finds meaning but misses exact identifiers (error codes, product names, flags); keyword search (BM25) finds exact terms but misses paraphrases. Running both and merging rankings with reciprocal rank fusion gets both strengths without normalising incompatible scores.
Longer Answer
RRF scores each document by summing 1/(k + rank) across result lists (k is often 60). Keep identifiers intact in the keyword index, and evaluate by query type.
Key Points
- Vectors miss identifiers; keywords miss paraphrases.
- Run both.
- Reciprocal rank fusion.
- Evaluate per query type.
Deep dive: Embedding Search That Misses Exact Terms
-
Most asked How do you enforce permissions and tenant isolation in RAG?
Summary
Filter by tenant and access rights inside the search query (metadata filters or SQL WHERE clauses), not after retrieving top-k, and back it with isolation below the application (per-tenant indexes or namespaces, or row-level security) so a missing filter fails closed. Sync access lists at indexing time and test cross-tenant isolation automatically.
Longer Answer
Post-filtering both leaks data when a code path forgets the filter and returns nothing for small tenants whose documents never reach the top k. Check recall of filtered queries with approximate indexes.
Key Points
- Filter inside the query.
- Isolation below the app.
- Sync document permissions.
- Automated cross-tenant tests.
Deep dive: Filters, Permissions and Tenant Isolation
-
Most asked How do you make RAG answers grounded and verifiable?
Summary
Instruct the model to answer only from the provided documents and to say when they don't contain the answer, require a source and an exact quote (or use a citations feature) for each claim, verify the quotes in code, and show users only verified citations.
Longer Answer
Retrieved text doesn't override what the model learned in training; frequently seen but outdated facts can leak into answers. Make "I can't find that" an acceptable and measured outcome.
Key Points
- Answer only from documents.
- Per-claim citations with quotes.
- Verify in code.
- 'Not found' is valid.
Deep dive: Grounded Answers and Verified Citations
-
Most asked How do you evaluate a RAG system?
Summary
Separately for retrieval and generation, on a labelled set of real questions: retrieval with recall@k and MRR against gold passages; generation with correctness against key facts, faithfulness (claims supported by retrieved text) and citation validity, using code graders and calibrated model graders. Compare per question to find regressions, and gate changes in CI.
Longer Answer
Production signals (negative feedback, 'not found' rate, escalations) feed new cases into the set.
Key Points
- Retrieval: recall@k, MRR.
- Generation: correctness, faithfulness, citations.
- Per-question comparison.
- CI gates and production feedback.
Deep dive: Measuring Retrieval and Answer Quality
-
Most asked RAG leaf: How do you choose top-k and context budget?
Summary
Tune k via retrieval metrics then constrain tokens for the generator.
Longer Answer
Tune k via retrieval metrics then constrain tokens for the generator. Rerank to keep fewer, better chunks. Monitor truncation rate.
Key Points
- Tune k via retrieval metrics then constrain tokens for the generator
- Rerank to keep fewer, better chunks
- Monitor truncation rate
- Practical RAG tuning
Comparison / Tradeoffs
Aspect Notes Core idea Tune k via retrieval metrics then constrain tokens for the generator. Practical tradeoff Practical RAG tuning. - RAG leaf: How do citations should be mapped back to sources?
- RAG leaf: What metadata filters are essential in enterprise RAG?
- RAG leaf: When is parent-document retrieval preferable to flat chunks?
- RAG leaf: How do you handle stale documents in a knowledge base?
- RAG leaf: What is late chunking / contextual retrieval?
- RAG leaf: Compare dense retrieval vs sparse (BM25) for code search.
- RAG leaf: How would you RAG over tables and structured data?
- RAG leaf: What caching layers help RAG latency?
- RAG leaf: Explain corrective RAG / self-reflective retrieval loops.
- RAG leaf: How do you ingest PDFs with complex layouts?
- RAG leaf: How do you A/B test a new embedding model?
- RAG leaf: Describe a failure mode when context is contradictory.
- RAG leaf: How does multimodal RAG differ from text-only?
- RAG leaf: What SLOs would you set for a customer-facing RAG assistant?
- What is reranking, and when does it help?
- How do you handle follow-up questions in conversational RAG?
- How do you keep a RAG index fresh?
- How can documents in a RAG index attack your system, and how do you defend?
- When is RAG the wrong tool?
- RAG leaf: When should you use graph RAG?
- How do you assemble retrieved chunks into a good prompt?
- What is contextual retrieval?
- Where does the time and money go in a RAG pipeline, and how do you reduce it?
- How would you choose a vector store for a new RAG project?
- Why must you re-embed everything when changing embedding models?
- Explain how a typical RAG pipeline works end to end.
- How do you choose chunk size and chunking strategy for RAG?
- What causes hallucinations in RAG systems, and how do you reduce them?
- How do you handle access control and multi-tenancy in a RAG knowledge base?