Area 03 · AI Agents · Concept 06 of 20

Short-term Memory

What the agent holds in the current context: the conversation and recent tool results.

01–10 · The basePhase 3 · Weeks 11–14≈ 45 min

The idea

An LLM has no memory between calls. Short-term memory is simply everything you resend each time: the system prompt, the conversation so far, and the tool calls and results from this run.

It grows quickly. Tool results (web pages, file contents, query results) can be thousands of tokens each, and every step resends all of them. That drives up cost and latency and can push important details out of focus.

Manage it deliberately: truncate or summarise large tool results before appending, keep only what later steps need, and use a scratchpad (a notes tool or plan) for key facts.

Why it matters

Most agent cost and many agent failures come from an overstuffed context.

Words to know

  • Working memoryAnother name for what is in the context window right now.

Try it without code

Measure context growth

  1. Run an agent task with logging of input tokens per step.
  2. Plot or list them.
  3. Spot which tool results caused the biggest jumps.

Use it in code

Trim tool results before they enter the context.

python
def compact_result(text, limit=3000):
    if len(text) <= limit:
        return text
    head, tail = text[:limit * 2 // 3], text[-limit // 3:]
    return f"{head}\n...[{len(text) - limit} chars omitted]...\n{tail}"

# in the loop:
# results.append({"type": "tool_result", "tool_use_id": b.id, "content": compact_result(raw)})
# and log msg.usage.input_tokens after every step to watch growth
PRACTICE · TICK IT AFTER THIS

Log input tokens per step for 3 agent runs, then add result trimming and compare totals.

Common mistakes

  • Appending entire web pages as tool results.
  • Assuming the model "remembers" past sessions.

Go deeper