AI Engineering Tutorial · Retrieval (RAG) · Lesson 18 of 25
Vector Search From Scratch
Store chunk vectors, embed the question, return the top-k most similar chunks; then see why keyword search still matters.
Top-k search
- At indexing time, embed every chunk and store the vector with the chunk.
- At question time, embed the question and compute similarity with every stored vector.
- Return the top k chunks (for example k = 5).
Real systems use a vector index (pgvector, a vector database) to make step 2 fast for millions of chunks.
import math
def cosine(a, b):
dot = sum(x * y for x, y in zip(a, b))
return dot / (math.hypot(*a) * math.hypot(*b))
index = [("Reset your password from the login page", [0.9, 0.1, 0.1]),
("Change your billing address in Settings", [0.1, 0.9, 0.2]),
("Two-factor authentication setup", [0.8, 0.0, 0.3]),
("Download invoices as PDF", [0.0, 0.8, 0.1])]
def top_k(query_vec, k=2):
scored = sorted(index, key=lambda item: -cosine(query_vec, item[1]))
return [(round(cosine(query_vec, v), 3), text) for text, v in scored[:k]]
print(top_k([0.85, 0.05, 0.2])) # a login/security question
print(top_k([0.05, 0.9, 0.1])) # a billing question
Output
[(0.991, 'Reset your password from the login page'), (0.99, 'Two-factor authentication setup')] [(0.998, 'Download invoices as PDF'), (0.993, 'Change your billing address in Settings')]
Don't forget keywords
Embeddings are weak on exact identifiers (error codes, product names). Hybrid search combines vector search with keyword search (BM25) and merges the rankings.
Test yourself
Which question type does pure vector search handle worst?
Paraphrased questionsQuestions with exact identifiers such as error code E4012Short questionsQuestions in English
Show answer
Answer: Questions with exact identifiers such as error code E4012
Identifiers carry little meaning; keyword search finds them exactly.