Compaction is lossy compression you control
Every long-running agent hits the same wall: the transcript outgrows the window. Compaction is the answer — replace the old part of the conversation with a summary of it and keep going. What nobody says out loud is that compaction is lossy compression, and you are the one choosing the loss function. Leave the choice to a default and the model loses the thing you most needed it to keep.
Two operations get conflated here and they are not the same. Clearing removes tool results while leaving the record that the call happened. Compaction summarizes the whole transcript and drops everything before the summary. Clearing is a mechanical edit with no inference cost; compaction costs a summarizer call and is where information actually dies.
That difference sets the order of operations. Clear first, compact later. In most agents tool results are the bulk of the window and the cheapest thing to lose, because they are re-fetchable. A decision made in turn three is not.
Clear tool results before you summarize anything
Anthropic's context editing does the clearing step server-side. The edit type is clear_tool_uses_20250919, under the context-management-2025-06-27 beta, and it has four knobs worth knowing: trigger (the input-token threshold that starts clearing, default 100K), keep (how many recent tool uses survive, default 3), clear_at_least, and exclude_tools.
clear_at_least is the one people skip and the one that decides whether this saves money. Clearing rewrites the prompt prefix, which invalidates your cached prefix, so every firing costs a cache write. Clearing 500 tokens to pay a full cache write is a net loss. The floor makes each firing earn its keep.
The measured shape, from Anthropic's own research-agent run over eight documents of roughly 40K tokens each with trigger at 30K and keep at 4: four clearing events, peak context 173,137 tokens against 335,279 for the unmanaged baseline, seven of eight file reads cleared by the end. Note the cost they name honestly — the agent re-reads what was cleared, so against a slow or rate-limited tool a token saving converts into latency.
context_management = {
"edits": [
{
"type": "clear_tool_uses_20250919",
"trigger": {"type": "input_tokens", "value": 30_000},
"keep": {"type": "tool_uses", "value": 4},
"clear_at_least": {"type": "input_tokens", "value": 10_000},
"exclude_tools": ["memory"], # never clear the memory tool
}
]
}
# Beta header: context-management-2025-06-27
# After the call, read what it actually did:
for edit in resp.context_management.applied_edits:
print(edit.cleared_tool_uses, edit.cleared_input_tokens)
# peak context 173,137 tokens vs 335,279 unmanaged (4 firings)What a summary keeps, and what it quietly drops
Compaction proper is a different edit type: compact_20260112, under the compact-2026-01-12 beta, with a server-side trigger defaulting to 150K input tokens and a floor of 50K. When it fires, the transcript becomes a summary and the history before it is dropped on the next request.
Anthropic published the probe everyone should be running against their own setup. With the trigger at 180K, one compaction at turn four produced a summary of roughly 2,783 tokens and held peak context to 169,164 tokens against the same 335,279 baseline. Then they asked the compacted agent to recall specific facts: three of three high-level facts preserved, zero of three obscure appendix values preserved.
That result is the entire post in one line. Summarization preserves the gist and destroys the specifics, and it is not malfunctioning when it does — that is what summarization means. It becomes a production incident only when the specific was a constraint.
Constraint amnesia, and how to detect it
Here is the failure mode worth naming. At turn two an agent is told the budget ceiling is 400 dollars, never to contact the customer directly, and to use the staging endpoint. Forty turns and one compaction later it books 600 dollars, emails the customer, and writes to production. Nothing raised. No tool errored. The model is behaving correctly on the context it actually has.
The reason this survives in codebases is that it does not present as a context bug. It presents as the model ignoring instructions, and the reflex is to write the rule more forcefully in the conversation — which puts the constraint in exactly the region compaction will summarize away. Louder phrasing does not survive compression any better than quiet phrasing.
The detection signal is mechanical, and you can run it over traces you already have. For each constraint stated in the conversation, check whether it was stated before the last compaction boundary and whether its key terms appear in the summary text. A constraint that crossed the boundary and is absent from the summary is a live incident waiting for the right turn. That is a grep, not a research project.
Promote decisions out of the transcript
The fix is not a better summarizer. Anything load-bearing must not live in the compactible region at all. Maintain a small structured ledger of decisions, constraints and identifiers, update it as they are made, and render it into the system prompt on every turn. Fixed context is paid on every call and is never compacted — that is precisely the property you want, and a few hundred tokens forever is a cheap price for closing a class of silent failure.
The discipline that keeps the ledger useful is being strict about what qualifies. A short declarative fact that changes what the agent is allowed to do goes in — the customer chose express shipping, the ceiling is 400 dollars, tenant is acme-prod. Process narration does not — the API returned 200, the search found twelve rows. If you cannot write it as one sentence that would still be true in an hour, it belongs in the transcript and it is allowed to die there.
from dataclasses import dataclass, field
@dataclass
class Ledger:
constraints: list[str] = field(default_factory=list) # hard limits
decisions: list[str] = field(default_factory=list) # user choices
ids: dict = field(default_factory=dict) # order_id, tenant
def render(self) -> str:
if not (self.constraints or self.decisions or self.ids):
return ""
return (
"## Established facts (authoritative; survive compaction)
"
+ "".join(f"- CONSTRAINT: {c}
" for c in self.constraints)
+ "".join(f"- DECIDED: {d}
" for d in self.decisions)
+ "".join(f"- {k}: {v}
" for k, v in self.ids.items())
)
# Re-rendered into the SYSTEM block on every turn, so compaction
# cannot reach it. ~200 tokens that make turn 40 behave like turn 2.
messages = [{"role": "system", "content": BASE_PROMPT + ledger.render()}, *history]Write the compaction prompt yourself
The default summarization prompt is generic because it has to be: it asks for a summary wrapped in summary tags and optimizes for narrative coherence. The instructions field does not append to it — it replaces it entirely, so you supply the whole framing including the wrapper.
Write it as a retention list rather than a style guide. Name the categories that must survive verbatim: numbers with their units, identifiers, the open-or-closed state of every subtask, and the constraints. Anthropic's own coding example is in that shape — preserve code snippets, variable names, technical decisions — and it reads like a schema because that is the useful register.
One mechanical detail bites everyone once. The API returns a typed compaction content block and you have to serialize it back into the history yourself. Forget it and the next request has neither the summary nor the history it replaced, which is the most expensive way to discover that compaction worked.
context_management = {
"edits": [{
"type": "compact_20260112",
"trigger": {"type": "input_tokens", "value": 150_000}, # min 50K
"instructions": (
"Summarize this agent transcript. Preserve VERBATIM: every "
"numeric limit with its unit, every identifier (order, tenant, "
"endpoint), every constraint the user stated, and the "
"open/closed state of each subtask. Narrative may be lossy; "
"the items above may not. Wrap in <summary></summary>."
),
}]
}
# The returned block MUST be written back into history, or the next
# request has no summary and no pre-summary history either.
for block in resp.content:
if block.type == "compaction":
history = [{"type": "compaction", "content": block.content}]The same pattern in the frameworks
LangMem exposes summarize_messages to call inside your own node, or a prebuilt SummarizationNode. The threshold is max_tokens_before_summary, which defaults to max_tokens, and summary length is capped by max_summary_tokens — set the summarizer model's own max_tokens to match it or your budget arithmetic is quietly wrong. The RunningSummary held in graph state is what stops the same messages being re-summarized on every turn.
The placement detail there is worth stealing wholesale: in a ReAct graph the summarization node sits before the model node, and after tool execution the graph routes back through it rather than straight to the model, so tool output is compressed on the way in rather than after it has already bloated the window.
The OpenAI Agents SDK starts from the other end. Sessions persist the full transcript by default — SessionSettings with limit of None fetches all stored items — and compaction is opt-in through OpenAIResponsesCompactionSession, which wraps another session and either fires after each turn or waits for your own run_compaction call. A limit on SessionSettings, or a session_input_callback, trims what gets sent without deleting what is stored. That split is the right default: the model sees the compacted list, and you keep the full transcript, because the full transcript is how you debug the compaction.
Test it like compression, because it is
You cannot read a summary and know what it lost. The only honest test is a probe set: plant a handful of facts early in a synthetic conversation, inflate it past the trigger, compact, then ask for each fact back. Score recall per category, not in aggregate — the aggregate hides the result that matters, which is the one Anthropic published and which you will reproduce: near-perfect on narrative, zero on buried specifics.
Run it as a regression test against your instructions string, because that string is production code. A one-word edit to a retention prompt is a behaviour change, and it is invisible in a diff review in a way that a code change is not.
The interview answer is two sentences. Compaction trades specificity for window space, so anything load-bearing — constraints, identifiers, decisions — gets promoted into state that compaction cannot reach rather than trusted to the summarizer. And the summarizer is measured by per-category recall against a probe set, not read once and approved.
PROBES = [
("constraint", "What is the spending ceiling?", "400"),
("identifier", "Which endpoint are we writing to?", "staging"),
("decision", "Which shipping did the customer pick?", "express"),
("number", "What was the measured p99, in ms?", "840"),
]
def probe(session, compacted_history):
out = {}
for category, question, expected in PROBES:
answer = session.ask(compacted_history, question)
out.setdefault(category, []).append(expected.lower() in answer.lower())
return {k: sum(v) / len(v) for k, v in out.items()}
# Typical first run, and the reason the ledger exists:
# {'constraint': 0.0, 'identifier': 0.0, 'decision': 1.0, 'number': 0.0}
assert probe(s, h)["constraint"] == 1.0, "constraint amnesia: promote it"
Work the same ideas as a full interview case study, with pressure questions and model answers.