The idea
Prompt engineering is about wording. Context engineering is about selection: at each step, which instructions, which tool definitions, which retrieved data, which past messages and which memories should the model see? The context window is a limited budget, and every token competes for attention.
Good practice: keep the system prompt focused, load tools and skills only when relevant, retrieve data just in time instead of up front, trim tool outputs, and keep a short running summary of progress.
Treat it as an engineering problem with measurements: tokens per step, success rate, cost per task.
As agents get longer and more capable, what you leave out matters as much as what you put in.
Words to know
- Just-in-time retrievalFetch data when a step needs it, not all at the start.
Try it without code
Audit one step
- Dump the full request of one agent step to a file.
- Highlight every part the model did not need for that step.
- Estimate the tokens you could save.
Use it in code
Build each step’s context from parts with a budget.
def build_context(task, plan, notes, recent_messages, tools_for_step):
system = "\n\n".join([
"You are a research agent. Work through the plan; cite sources.",
f"<task>{task}</task>",
f"<plan>{plan}</plan>",
f"<notes>{notes[-3000:]}</notes>", # running summary, capped
])
messages = recent_messages[-6:] # only the last few turns verbatim
return system, messages, tools_for_step # only tools relevant to the current stepMeasure average tokens per step and success rate on 5 tasks before and after restructuring your agent’s context.
Common mistakes
- Dumping all documents into context "just in case".
- Loading 50 tool definitions when 5 are relevant.