The idea
An embedding model reads a piece of text and outputs a fixed-length vector, for example 384 or 1,024 numbers. It is trained so that texts with similar meaning get vectors that point in similar directions, even when they share no words: "cancel my subscription" lands near "how do I stop paying".
This is what makes semantic search possible. Embed every chunk of your documents once and store the vectors. At question time, embed the question and find the stored vectors closest to it.
Embedding models are separate from chat models: smaller, cheaper and faster. You must use the same embedding model for documents and queries; vectors from different models are not comparable. Changing models means re-embedding everything.
Embeddings are the foundation of RAG, semantic caching, deduplication, clustering and recommendations.
Picture it
Words to know
- VectorAn ordered list of numbers.
- DimensionsThe length of the vector, fixed per model.
- Embedding modelA model that maps text to vectors.
Try it without code
Ask a chat app to group sentences
- Give a chat app 10 short sentences on 3 hidden themes.
- Ask it to group them by meaning.
- That grouping is what embeddings do numerically.
Use it in code
Embed sentences locally and compare them.
# pip install sentence-transformers
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("BAAI/bge-small-en-v1.5") # 384 dimensions
texts = ["cancel my plan", "stop paying monthly", "banana bread recipe"]
vecs = model.encode(texts, normalize_embeddings=True)
print(vecs.shape)
sims = vecs @ vecs.T
for i, t in enumerate(texts):
print(t, [round(float(s), 2) for s in sims[i]])Embed 50 of your own notes or FAQ entries and print the 3 nearest neighbours for 5 test questions.
Common mistakes
- Mixing vectors from two embedding models.
- Embedding huge documents as one vector; meaning gets averaged away.