AI Engineering Tutorial · Retrieval (RAG) · Lesson 16 of 25
Embeddings and Similarity
Embeddings turn text into vectors so that similar meanings are close together; cosine similarity measures how close.
What an embedding is
An embedding model turns a piece of text into a list of numbers (a vector, often hundreds or thousands of numbers long). Texts with similar meaning get vectors pointing in similar directions, even when they share no words: "reset my password" is close to "I forgot my login".
Measuring closeness
Cosine similarity compares the direction of two vectors: 1 means the same direction, 0 unrelated. Search works by embedding the question and finding the stored texts with the highest similarity.
import math
def cosine(a, b):
dot = sum(x * y for x, y in zip(a, b))
return dot / (math.sqrt(sum(x * x for x in a)) * math.sqrt(sum(y * y for y in b)))
# made-up 3-dimensional vectors: [account/login, billing, shipping]
docs = {"How to reset your password": [0.9, 0.1, 0.0],
"Update your card details": [0.2, 0.9, 0.1],
"Track your parcel": [0.0, 0.1, 0.95]}
question = [0.85, 0.15, 0.05] # "I forgot my login"
for title, vec in sorted(docs.items(), key=lambda kv: -cosine(question, kv[1])):
print(f"{cosine(question, vec):.3f} {title}")
Output
0.996 How to reset your password 0.387 Update your card details 0.076 Track your parcel
Real embeddings come from an embedding API or an open model; the tiny 3-number vectors here only show the arithmetic. See Embeddings.
Test yourself
Two texts share no words but mean the same thing. What do their embeddings look like?
Identical vectorsSimilar vectors with high cosine similarityOpposite vectorsEmbeddings can't represent them
Show answer
Answer: Similar vectors with high cosine similarity
Embeddings capture meaning, not exact words.