Area 03 · AI Agents · Concept 18 of 20

Context Compaction

Summarise or drop old history when the context gets long, and keep what still matters.

11–20 · Production levelPhase 3 · Weeks 11–14≈ 1 hour

The idea

Long agent runs and long chats eventually approach the context limit. Compaction replaces old parts of the history with a condensed version: a summary of decisions, findings and open questions, while recent turns stay verbatim.

Trigger it at a threshold (for example 70% of the window). Ask the model to summarise with a specific template: goal, decisions made, facts found with sources, files changed, next steps. Then restart the history as [system, summary, recent turns].

Cheaper variants: drop old tool results but keep the calls, or clear tool results older than N steps. Some APIs offer built-in context editing for this.

Why it matters

It lets agents work on tasks longer than one context window without forgetting the goal.

Words to know

  • CompactionReplacing old context with a shorter representation.

Try it without code

Summarise a long chat

  1. Take a long chat with a model.
  2. Ask it to summarise using: goal, decisions, facts, open questions.
  3. Start a new chat with only that summary and continue.

Use it in code

Compact history when input tokens pass a threshold.

python
import anthropic
client = anthropic.Anthropic()
TEMPLATE = "Summarise this work so far as: Goal / Decisions / Facts (with sources) / Open questions / Next step."

def maybe_compact(system, messages, last_input_tokens, limit=150_000, keep=6):
    if last_input_tokens < 0.7 * limit or len(messages) <= keep:
        return messages
    old, recent = messages[:-keep], messages[-keep:]
    transcript = "\n".join(f"{m['role']}: {m['content'] if isinstance(m['content'], str) else '[tool activity]'}" for m in old)
    summary = client.messages.create(model="claude-haiku-5-5", max_tokens=1000,
        messages=[{"role": "user", "content": f"{TEMPLATE}\n\n{transcript}"}]).content[0].text
    return [{"role": "user", "content": f"<summary_of_earlier_work>{summary}</summary_of_earlier_work>"},
            {"role": "assistant", "content": "Understood. Continuing."}] + recent
PRACTICE · TICK IT AFTER THIS

Run a 30-step task with and without compaction and compare total tokens and whether the goal was met.

Common mistakes

  • Cutting between a tool call and its result.
  • Summaries that drop sources and IDs.

Go deeper