The idea
A chunk like "Revenue grew 3% over the previous quarter" is useless alone: which company, which quarter? Contextual retrieval fixes this at indexing time. For each chunk, an LLM reads the whole document and writes one or two sentences of context, which are prepended to the chunk before embedding and keyword indexing.
Anthropic reported large reductions in retrieval failures from contextual embeddings plus contextual BM25, and more when combined with reranking.
It costs one LLM call per chunk at indexing time. Prompt caching makes this affordable, since the whole document is the same prefix for every chunk in it.
It attacks the most common RAG failure (chunks that lost their context) at the source, once, offline.
Words to know
- Contextualised chunkA chunk with a short generated description of where it belongs.
Try it without code
Write context by hand
- Pick 3 chunks from a report.
- Write one sentence for each saying which document and section they come from.
- Imagine searching for them with and without that sentence.
Use it in code
Generate context per chunk with the document cached.
import anthropic
client = anthropic.Anthropic()
def contextualise(document, chunk):
msg = client.messages.create(
model="claude-haiku-5-5", max_tokens=120,
system=[{"type": "text", "text": f"<document>{document}</document>",
"cache_control": {"type": "ephemeral"}}],
messages=[{"role": "user", "content":
f"<chunk>{chunk}</chunk>\nWrite 1-2 sentences situating this chunk within the document, "
"to improve search retrieval. Reply with the context only."}])
return msg.content[0].text.strip() + "\n\n" + chunk
# index contextualise(doc, c) instead of c, for both embeddings and BM25Re-index one document set with contextual chunks and compare Recall@5 with the plain index.
Common mistakes
- Doing it without prompt caching and paying full price per chunk.
- Showing the generated context as if it were a quote from the source.