The idea
Questions and answers are phrased differently. "Why does my pod keep restarting?" looks unlike the documentation paragraph about liveness probes. HyDE (Hypothetical Document Embeddings) asks the LLM to write a plausible answer first, then embeds that fake answer and searches with it.
The fake answer may be wrong in details, but it uses the vocabulary and style of real answers, so it lands closer to the right chunks in embedding space. The real chunks then ground the final answer.
It costs an extra generation and helps most when questions are short and documents are technical. Measure it; on some datasets it does little.
It is a cheap trick to close the question-answer vocabulary gap without retraining embeddings.
Picture it
Words to know
- Hypothetical documentA model-written answer used only as a search query.
Try it without code
Compare query texts
- Ask a chat app to write a 3-sentence answer to a question about your docs.
- Search your docs with both the question and that answer.
- Compare results.
Use it in code
HyDE as a function that returns the vector to search with.
import anthropic
from sentence_transformers import SentenceTransformer
client = anthropic.Anthropic()
embedder = SentenceTransformer("BAAI/bge-small-en-v1.5")
def hyde_vector(question):
msg = client.messages.create(model="claude-haiku-5-5", max_tokens=200,
messages=[{"role": "user", "content": f"Write a short documentation paragraph that answers: {question}"}])
fake = msg.content[0].text
return embedder.encode(fake, normalize_embeddings=True)
vec = hyde_vector("why does my pod keep restarting?")Run your 30 test questions with plain query embeddings and with HyDE; compare Recall@5.
Common mistakes
- Showing the hypothetical answer to users.
- Using HyDE where questions already look like documents.