Area 02 · RAG · Concept 01 of 20

Embeddings

Turn text into a list of numbers (a vector) so similar meanings end up close together.

01–10 · The basePhase 2 · Weeks 7–10≈ 1 hour

The idea

An embedding model reads a piece of text and outputs a fixed-length vector, for example 384 or 1,024 numbers. It is trained so that texts with similar meaning get vectors that point in similar directions, even when they share no words: "cancel my subscription" lands near "how do I stop paying".

This is what makes semantic search possible. Embed every chunk of your documents once and store the vectors. At question time, embed the question and find the stored vectors closest to it.

Embedding models are separate from chat models: smaller, cheaper and faster. You must use the same embedding model for documents and queries; vectors from different models are not comparable. Changing models means re-embedding everything.

Why it matters

Embeddings are the foundation of RAG, semantic caching, deduplication, clustering and recommendations.

Picture it

Words to know

  • VectorAn ordered list of numbers.
  • DimensionsThe length of the vector, fixed per model.
  • Embedding modelA model that maps text to vectors.

Try it without code

Ask a chat app to group sentences

  1. Give a chat app 10 short sentences on 3 hidden themes.
  2. Ask it to group them by meaning.
  3. That grouping is what embeddings do numerically.

Use it in code

Embed sentences locally and compare them.

python
# pip install sentence-transformers
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("BAAI/bge-small-en-v1.5")   # 384 dimensions
texts = ["cancel my plan", "stop paying monthly", "banana bread recipe"]
vecs = model.encode(texts, normalize_embeddings=True)
print(vecs.shape)
sims = vecs @ vecs.T
for i, t in enumerate(texts):
    print(t, [round(float(s), 2) for s in sims[i]])
PRACTICE · TICK IT AFTER THIS

Embed 50 of your own notes or FAQ entries and print the 3 nearest neighbours for 5 test questions.

Common mistakes

  • Mixing vectors from two embedding models.
  • Embedding huge documents as one vector; meaning gets averaged away.

Go deeper