Write the Agent Loop Yourself: 80 Lines, No Framework

The loop is the whole idea

An agent is a language model in a while loop with access to functions. That is not a simplification for beginners — it is the actual architecture. LangGraph adds a state machine around it. Google's ADK adds a session service and a deployment target. CrewAI adds roles. Underneath all of them sits the same four steps: send the conversation to the model, look at what came back, if it asked for a tool then run it and append the result, otherwise stop.

Engineers who have only ever used a framework tend to describe agents in the framework's vocabulary — nodes, crews, runners. That vocabulary collapses the moment something breaks, because the bug is almost never in the abstraction. It is in the loop: the model called a tool that did not exist, or the tool result went back in a shape the model could not read, or nothing ever satisfied the stop condition and you burned forty turns. You cannot debug what you have never implemented.

So implement it once. The version below is about eighty lines, runs against any tool-calling model API, and contains every component that a production agent needs to have an opinion about.

Step one: the message list is the state

The single most important thing to internalise is that an agent has no memory beyond the list of messages you send it. Every turn is stateless from the model's point of view. What makes it feel continuous is that you keep appending to an array and re-sending the whole thing.

That array has four kinds of entry: the system prompt, user messages, assistant messages (which may contain tool calls), and tool results. The ordering rule that trips people up is that a tool result must immediately follow the assistant message that requested it, and every requested call must get a result. If the model asks for three tools and you return two, most APIs will reject the next request outright.

messages = [
    {"role": "system", "content": SYSTEM_PROMPT},
    {"role": "user", "content": "How many orders shipped late last month?"},
]

# After one round trip the array looks like this:
# [ system,
#   user,
#   assistant(tool_calls=[query_orders(sql=...)]),
#   tool(tool_call_id=..., content='{"rows": [...]}'),
#   assistant("142 orders shipped late in August.") ]

Step two: a tool is a function plus a schema

A tool has two halves that must stay in sync: the JSON schema you advertise to the model, and the Python callable you dispatch to. Keeping them in one place is worth doing from the first hour, because the classic production bug is a schema that says a parameter is optional and a function that requires it.

Note what the registry below does not do: it does not let the model call anything not explicitly registered. That is the entire authorization model at this layer, and it is why 'the model called a dangerous function' is not a real failure mode unless you registered a dangerous function.

TOOLS = {}

def tool(name: str, description: str, schema: dict):
    def wrap(fn):
        TOOLS[name] = {
            "spec": {
                "name": name,
                "description": description,
                "input_schema": schema,
            },
            "fn": fn,
        }
        return fn
    return wrap


@tool(
    "query_orders",
    "Run a read-only SQL SELECT against the orders table. "
    "Use for counts, aggregates and lookups over order history.",
    {
        "type": "object",
        "properties": {
            "sql": {"type": "string", "description": "A single SELECT statement."}
        },
        "required": ["sql"],
    },
)
def query_orders(sql: str) -> dict:
    if not sql.lstrip().upper().startswith("SELECT"):
        return {"ok": False, "error": "Only SELECT statements are allowed."}
    rows = db.execute(sql).fetchall()
    return {"ok": True, "row_count": len(rows), "rows": rows[:50]}

Step three: the loop itself

Here is the whole thing. Read it once for shape, then read the four annotations underneath, because each one corresponds to a decision that frameworks make for you silently.

The structure is deliberately boring: call the model, check whether it wants tools, run them, append, repeat. If it did not ask for tools, it produced an answer and we are done.

MAX_TURNS = 12

def run(user_input: str) -> str:
    messages = [{"role": "user", "content": user_input}]
    specs = [t["spec"] for t in TOOLS.values()]

    for turn in range(MAX_TURNS):
        reply = client.messages.create(
            model=MODEL,
            system=SYSTEM_PROMPT,
            tools=specs,
            messages=messages,
            max_tokens=4096,
        )
        messages.append({"role": "assistant", "content": reply.content})

        calls = [b for b in reply.content if b.type == "tool_use"]
        if not calls:                                     # (1) stop condition
            return "".join(b.text for b in reply.content if b.type == "text")

        results = []
        for call in calls:                                # (2) every call gets a result
            entry = TOOLS.get(call.name)
            if entry is None:                             # (3) hallucinated tool
                payload = {"ok": False,
                           "error": f"No tool named {call.name}.",
                           "available": list(TOOLS)}
            else:
                try:
                    payload = entry["fn"](**call.input)
                except Exception as exc:                  # (4) never raise past here
                    payload = {"ok": False,
                               "error": f"{type(exc).__name__}: {exc}"}
            results.append({
                "type": "tool_result",
                "tool_use_id": call.id,
                "content": json.dumps(payload)[:20_000],
            })
        messages.append({"role": "user", "content": results})

    return "Stopped: hit the turn limit without reaching an answer."

The four decisions hiding in that code

(1) The stop condition. This loop stops when the model returns text instead of a tool call — the model decides it is finished. That is the right default, but it is not the only option. A supervisor agent might stop when a specific tool is called (a submit_answer tool), which gives you a typed result instead of prose. Frameworks call this a terminal tool or an output schema, and it is worth reaching for whenever something downstream has to parse the answer.

(2) Every requested call gets a result, including the ones that failed. Skipping a result because the tool errored is the single most common way to produce an API 400 on the following turn. Return the error as content, not as an absent entry.

(3) Hallucinated tool names are normal, not exceptional. Models occasionally invent a plausible-sounding function, especially when the real tool set does not cover what the user asked for. Returning the list of available tools in the error turns a dead end into a recoverable turn — the model reads that, picks a real tool, and continues.

(4) The try/except is not defensive programming, it is the contract. An exception that escapes the loop kills the turn and loses everything the agent has done so far. A structured error goes back into the conversation and the model gets to respond to it. This is important enough that it deserves its own treatment, and it is the subject of the next post in this series.

The turn budget is a cost control, not a safety net

MAX_TURNS exists because agents loop. Not occasionally — regularly. The classic pattern is a tool that keeps returning something almost-but-not-quite useful, so the model tries a slight variation, gets an almost-useful result again, and repeats until something stops it. Without a budget that is an unbounded bill.

Twelve is a reasonable starting number for a single-purpose agent, but the number matters less than what you do when you hit it. Returning a bare 'turn limit exceeded' string to a user is a bad answer. Better is to make one final model call with the tools removed and an instruction to summarise what was learned and what remains unknown. You get a partial answer instead of a failure, and the transcript tells you whether the limit was genuinely too low or the agent was stuck.

Instrument this from day one. Log the turn count for every run and look at the distribution, not the average. A healthy single-purpose agent finishes most tasks in two to four turns. If your p95 is at the limit, you do not have a limit problem, you have a tool design problem.

What the frameworks add, now that you can see the seams

With the loop in your head, the framework feature lists become legible. Persistent state and checkpointing means the message array survives a process restart, so a long task can resume rather than start over. Graph-based control flow means you can route to different prompts and tool sets depending on state, instead of relying on one system prompt to cover every branch. Streaming means you surface tokens and tool progress as they happen rather than after the final turn. Observability means each iteration of that for loop becomes a span you can inspect a week later.

Every one of those is a real feature and most production systems want several of them. The point is not that frameworks are unnecessary — it is that you should be able to name which of these you are adopting a framework for. 'We use LangGraph' is not an architecture. 'We use LangGraph because the workflow has three branches and we need checkpointing across a multi-hour task' is.

This is also, almost verbatim, a very common interview question: describe how an agent works. The answer that lands is the loop — messages in, tool calls out, results appended, stop when the model stops asking. Then the follow-ups: how do you bound it, what happens when a tool fails, how do you know it worked. The rest of this series takes those one at a time.