RAG interview question · Most asked
How do you choose a chunking strategy for RAG?
Summary
Chunk along document structure (headings, paragraphs, lists) and prefix each chunk with its title and heading path; keep tables and code whole; consider contextual retrieval or small-to-big retrieval when chunks lack context; and choose between options using retrieval recall on a labelled set of real questions.
Longer Answer
Fixed-size chunks split facts from their headings, so the best-matching chunk may lack the answer. There is no universal chunk size; measure.
Key Points
- Structure-aware chunks with heading prefixes.
- Keep tables and code whole.
- Contextual or small-to-big when needed.
- Decide with recall@k.
Deep dive: Chunking: Why the Answer Was Split in Half
More most-asked RAG questions
- RAG leaf: Explain query rewriting and multi-query retrieval.
- How do you enforce permissions and tenant isolation in RAG?
- How do you make RAG answers grounded and verifiable?
- How do you evaluate a RAG system?
- RAG leaf: How do you measure retrieval quality independently of generation?
- RAG leaf: How do you prevent prompt injection via retrieved documents?
- Why use hybrid search in RAG, and how do you combine the results?
- RAG leaf: How do you evaluate faithfulness of answers?