The idea
Long agent runs and long chats eventually approach the context limit. Compaction replaces old parts of the history with a condensed version: a summary of decisions, findings and open questions, while recent turns stay verbatim.
Trigger it at a threshold (for example 70% of the window). Ask the model to summarise with a specific template: goal, decisions made, facts found with sources, files changed, next steps. Then restart the history as [system, summary, recent turns].
Cheaper variants: drop old tool results but keep the calls, or clear tool results older than N steps. Some APIs offer built-in context editing for this.
It lets agents work on tasks longer than one context window without forgetting the goal.
Words to know
- CompactionReplacing old context with a shorter representation.
Try it without code
Summarise a long chat
- Take a long chat with a model.
- Ask it to summarise using: goal, decisions, facts, open questions.
- Start a new chat with only that summary and continue.
Use it in code
Compact history when input tokens pass a threshold.
import anthropic
client = anthropic.Anthropic()
TEMPLATE = "Summarise this work so far as: Goal / Decisions / Facts (with sources) / Open questions / Next step."
def maybe_compact(system, messages, last_input_tokens, limit=150_000, keep=6):
if last_input_tokens < 0.7 * limit or len(messages) <= keep:
return messages
old, recent = messages[:-keep], messages[-keep:]
transcript = "\n".join(f"{m['role']}: {m['content'] if isinstance(m['content'], str) else '[tool activity]'}" for m in old)
summary = client.messages.create(model="claude-haiku-5-5", max_tokens=1000,
messages=[{"role": "user", "content": f"{TEMPLATE}\n\n{transcript}"}]).content[0].text
return [{"role": "user", "content": f"<summary_of_earlier_work>{summary}</summary_of_earlier_work>"},
{"role": "assistant", "content": "Understood. Continuing."}] + recentRun a 30-step task with and without compaction and compare total tokens and whether the goal was met.
Common mistakes
- Cutting between a tool call and its result.
- Summaries that drop sources and IDs.