Area 02 · RAG · Concept 12 of 20

HyDE

Have the LLM write a hypothetical answer, embed that answer, and search with it.

11–20 · Production levelPhase 2 · Weeks 7–10≈ 45 min

The idea

Questions and answers are phrased differently. "Why does my pod keep restarting?" looks unlike the documentation paragraph about liveness probes. HyDE (Hypothetical Document Embeddings) asks the LLM to write a plausible answer first, then embeds that fake answer and searches with it.

The fake answer may be wrong in details, but it uses the vocabulary and style of real answers, so it lands closer to the right chunks in embedding space. The real chunks then ground the final answer.

It costs an extra generation and helps most when questions are short and documents are technical. Measure it; on some datasets it does little.

Why it matters

It is a cheap trick to close the question-answer vocabulary gap without retraining embeddings.

Picture it

Words to know

  • Hypothetical documentA model-written answer used only as a search query.

Try it without code

Compare query texts

  1. Ask a chat app to write a 3-sentence answer to a question about your docs.
  2. Search your docs with both the question and that answer.
  3. Compare results.

Use it in code

HyDE as a function that returns the vector to search with.

python
import anthropic
from sentence_transformers import SentenceTransformer

client = anthropic.Anthropic()
embedder = SentenceTransformer("BAAI/bge-small-en-v1.5")

def hyde_vector(question):
    msg = client.messages.create(model="claude-haiku-5-5", max_tokens=200,
        messages=[{"role": "user", "content": f"Write a short documentation paragraph that answers: {question}"}])
    fake = msg.content[0].text
    return embedder.encode(fake, normalize_embeddings=True)

vec = hyde_vector("why does my pod keep restarting?")
PRACTICE · TICK IT AFTER THIS

Run your 30 test questions with plain query embeddings and with HyDE; compare Recall@5.

Common mistakes

  • Showing the hypothetical answer to users.
  • Using HyDE where questions already look like documents.

Go deeper