Area 02 · RAG · Concept 04 of 20

Vector Databases

Databases built to store vectors and find the nearest ones fast (Qdrant, Pinecone, Weaviate, Chroma).

01–10 · The basePhase 2 · Weeks 7–10≈ 1 hour

The idea

Comparing a query with every stored vector works for thousands of chunks, not millions. Vector databases store embeddings with their metadata and build an index that finds approximate nearest neighbours in milliseconds.

They also handle filtering by metadata, updates and deletes, replication and multi-tenancy. Options range from embedded libraries (Chroma, LanceDB) to self-hosted servers (Qdrant, Weaviate, Milvus) and managed services (Pinecone), and extensions to databases you already have (pgvector for Postgres).

Choose on scale, filtering needs, operations burden and whether you want vectors next to your relational data. For most products under tens of millions of chunks, Postgres with pgvector is a sensible default.

Why it matters

It is the storage layer of every RAG system and the main operational decision in it.

Words to know

  • ANNApproximate nearest neighbour search: fast, very slightly imperfect.
  • Collection / indexA named set of vectors with the same dimensions.

Try it without code

Compare three options

  1. Read the quick-start of Chroma, Qdrant and pgvector.
  2. Write down for each: how it runs, how filtering works, and how you would back it up.

Use it in code

An in-process vector store with Chroma in a few lines.

python
# pip install chromadb
import chromadb

client = chromadb.PersistentClient(path="./chroma")
col = client.get_or_create_collection("faq", metadata={"hnsw:space": "cosine"})
col.add(ids=["1", "2", "3"],
        documents=["Refunds take 5 days.", "Reset your password in Settings.", "We ship to the EU."],
        metadatas=[{"topic": "billing"}, {"topic": "account"}, {"topic": "shipping"}])
res = col.query(query_texts=["how long until I get my money back"], n_results=2)
print(res["documents"], res["distances"])
PRACTICE · TICK IT AFTER THIS

Load 500 chunks into Chroma and into pgvector, and compare query speed and how filters are written.

Common mistakes

  • Picking a managed vector DB before you know your scale.
  • Keeping vectors in one system and permissions in another with no sync.

Go deeper