When Not to Build an Agent (A Decision Tree)

What you are actually buying

An agent buys you one thing: the model chooses the sequence of steps at runtime instead of you choosing it at design time. That is the entire value proposition. Everything else — tool calling, retrieval, structured output — you can have without an agent.

It is worth being precise about what that costs, because the costs are not obvious until you are living with them. Non-determinism, so the same input can take a different path tomorrow. Variable latency and variable cost, because the number of turns is decided at runtime. A much harder debugging story, since a bad answer might come from the prompt, the tool set, a tool's return shape, or the model's choice among them. And an evaluation problem with no ground truth, because there is no single correct trajectory to compare against.

That is a real price. It is worth paying when the sequence genuinely cannot be known in advance. It is a bad trade when you knew the sequence all along.

The first question: can you draw the flowchart?

Sit down and draw the steps for the ten most common requests. If you can draw them — even with branches, even with a dozen boxes — you do not need an agent. You need a workflow, and a workflow will be faster, cheaper, testable, and debuggable at three in the morning.

Teams resist this because a workflow feels less sophisticated. But a deterministic pipeline with an LLM at two or three steps is not a lesser architecture; it is the correct architecture for a known process. Use the model where judgement is genuinely required — classification, extraction, generation — and use ordinary code for the control flow you already understand.

The honest version of the question is: are you reaching for an agent because the problem demands runtime planning, or because agents are what everyone is building?

Four cases that are not agent problems

One input, one answer, no tools. That is a prompt. Wrapping a loop around a single model call adds latency and failure modes to buy nothing.

Retrieve then answer. If the shape is always search-then-respond, that is a RAG endpoint. It has a fixed two-step flow, so a loop only adds the risk of a second unnecessary search.

A known multi-step pipeline. Extract fields, validate them, write to a database, send a confirmation. Every run does the same four things in the same order. That is a job, and it should look like one — with retries, idempotency and a queue, not a model deciding whether to do step three.

Bulk processing where every row gets identical treatment. Ten thousand documents, same extraction each time. You want a batch pipeline with concurrency control, not ten thousand agent loops each deciding afresh how to approach a document.

What a real agent problem looks like

The signal is that the next step depends on what the previous step returned, in a way you cannot enumerate. A support request where the right investigation depends on what the first lookup reveals. A research task where each finding suggests where to look next. A debugging session where the error determines the next probe.

A second signal is a long tail you cannot cover. If ninety per cent of requests fit five known shapes and the remaining ten per cent are all different, the sane architecture is often a workflow for the ninety and an agent for the tail — not an agent for everything.

A third is a large tool surface where selection itself is the hard part. Thirty tools with genuinely different applicability is a routing problem a model is good at and a decision tree is bad at.

Does the next step depend on what the last step returned,
in a way you cannot enumerate up front?
  │
  ├─ No ──► Can you draw the flowchart?
  │           ├─ Yes ──► Workflow. Put the model at the judgement steps.
  │           └─ No, because it is one step ──► Just a prompt.
  │
  └─ Yes ─► Is the variation in the long tail only?
              ├─ Yes ──► Workflow for the common path, agent for the tail.
              └─ No ───► Agent. Now go read the rest of this series.

The hybrid is usually the right answer

The framing that helps most is to stop treating this as a binary. Most production systems that work well are workflows with agentic steps, not agents all the way down.

A concrete example. An invoice pipeline: receive the document, extract fields with a model, validate against the purchase order in code, and route. All deterministic. But when validation fails in a way the rules do not cover — the amounts differ and the reason is not obvious — that one branch hands off to an agent that can query the PO system, check delivery records and read the vendor's notes to work out what happened.

Ninety-five per cent of invoices never touch the agent. The five per cent that do are exactly the cases where runtime planning earns its cost. You get deterministic throughput on the bulk and genuine flexibility where you need it — and the agent's scope is small enough that you can actually evaluate it.

How to answer this in an interview

"Would you use an agent here?" is a judgement question, and reaching straight for an agent is the answer that reads as inexperience. What lands is naming the trade explicitly: an agent buys runtime planning and costs determinism, latency predictability, and evaluability.

Then apply it to the case in front of you. If the interviewer describes a fixed pipeline, say so and propose the workflow. If they describe genuine branching, propose the agent and immediately talk about bounding it — turn budgets, tool scope, stop conditions.

The strongest version is proposing the hybrid unprompted, because it shows you have run one of these in production rather than only read about them.