Context Engineering: The Discipline That Replaced Prompt Engineering

The shift

Prompt engineering is about wording: how you phrase an instruction to get a better answer from one call. It is a real skill and it still matters at the margins.

But in an agent, the prompt is a small and shrinking fraction of what the model sees. By turn six the window holds a system prompt, thirty tool schemas, six rounds of tool results, retrieved documents and the whole conversation. Rewording the instruction changes perhaps five per cent of the input. The other ninety-five per cent is the result of decisions about what to include, in what order, at what fidelity.

Those decisions are the discipline. Context engineering is deciding what occupies the window on every turn — and the reason it matters more than wording is that the failures it causes are the ones that actually take production agents down.

Four failures that are all context failures

Degradation over a long session. The agent is sharp for five turns and vague by turn fifteen. Nothing changed in the prompt; the signal simply got diluted as accumulated tool output crowded the window.

Losing its own constraints. It was told not to promise delivery dates, and on turn twelve it promises one. The instruction is still technically present, buried under nine tool results.

Re-deriving what it already knows. It looks up the same customer three times because the resolved ID was never held anywhere durable, and each lookup is a turn and a chance to resolve differently.

Confidently answering from a partial set. A tool returned fifty of four thousand rows without saying so, and the model reasoned over fifty as though it were everything. That one is a return-shape decision, which is a context decision.

The three budgets

The technique that helps most is to stop treating the window as one pool and split it into three budgets with separate rules.

Fixed context is what appears on every turn: system prompt and tool schemas. It is the most expensive real estate you own because you pay for it on every single call, and it is where most waste hides. Trimming a bloated tool set here pays back on every turn of every conversation.

Retrieved context is what you fetch for this turn — documents, records, search results. It should be the most aggressively managed: relevance-filtered, truncated, and dropped once used. The mistake is treating retrieved content as permanent once it has entered the conversation.

Conversational context is the running history. It grows without bound unless you do something, and doing something means compaction — which the next post but one covers.

Assign each a share of the window and enforce it. The arithmetic is the subject of the next post; the point here is that having budgets at all changes how you build.

Placement matters as much as inclusion

Attention is not uniform across a long context. Material at the start and the end is weighted more heavily than material buried in the middle — the effect usually called lost in the middle — and it has direct architectural consequences.

Constraints that must hold go early, in the system prompt, and the most critical ones can be restated immediately before the final instruction on turns where they matter. Retrieved evidence goes close to the question it supports, not appended at the top where the model saw it eight turns ago.

And ordering matters inside a single response too: put the reasoning field before the conclusion in any structured output, because generation is sequential and a model cannot condition on something it has not written yet. That is the same principle at a different scale, and the structured-output post in this series covers it.

Curate rather than accumulate

The default agent loop appends and never removes, which means the window fills with things that were relevant six turns ago and are noise now. A better default is that everything entering the context has a lifetime.

In practice: summarise a tool result into the fact you needed rather than keeping the payload. Drop retrieved documents once the answer that used them is written. Promote durable facts — a resolved customer ID, a chosen order — into session state, where they cost a line instead of a payload and cannot be lost to compaction.

The mental shift is from a transcript to a working set. A transcript records what happened. A working set holds what is still needed.

# Accumulating: the full payload lives forever
messages.append(tool_result(raw_response))          # 4,000 tokens, forever

# Curating: keep the fact, drop the payload, promote what is durable
fact = summarise(raw_response)                      # ~40 tokens
messages.append(tool_result(fact))
session.state["customer_id"] = raw_response["id"]   # survives compaction

Why this is the interview question now

Ask someone how they would improve an agent and a weak answer reaches for the prompt. A strong one asks what is in the window on turn ten, and then talks about what to remove.

The specific things worth naming: a token budget split across fixed, retrieved and conversational context; a compaction strategy for long sessions; promotion of durable facts into state rather than history; and bounded, truncation-aware tool returns.

That set of answers signals someone who has operated an agent past the demo, because every one of them is a lesson you learn by watching a session degrade.