A 44-part series on building, deploying, evaluating and securing AI agents — the loop itself, the frameworks around it, and everything that breaks once real users arrive.
Every agent framework you will ever use is a wrapper around one while loop. Build that loop by hand once — with real tool dispatch, a stop condition, and a turn budget — and the frameworks stop being magic. This is the version you can debug at 2am.
Most agent failures that look like reasoning failures are tool-selection failures. The model picked the wrong function, or invented arguments, or did not realise a tool applied. Almost all of it is fixable in the schema — and the schema is prompt engineering, not API plumbing.
In ordinary code an exception is a signal to a developer. In an agent, an uncaught exception kills the turn and throws away everything the model has figured out. The tool contract that makes agents recover instead of collapse is one rule long — and it changes how you write every tool.
Three mechanisms will force a model to return a schema, and teams pick between them by habit rather than by fit. They differ in guarantee strength, in latency, and in one under-discussed way: the strictest option can quietly make the content worse even as the format gets perfect.
An agent's system prompt is not a paragraph of encouragement. It is the only place you get to specify behaviour that holds across every turn, every tool result, and every input a user throws at it. Four blocks, in a fixed order, and one of them is the block nobody writes.
A large share of agent projects should have been a workflow, a retrieval endpoint, or a scheduled job. The tell is usually visible in week one and ignored until month three. Here is how to check before you commit a quarter to it.
The Agent Development Kit quickstart gets you a working agent in twenty lines. It also leaves out every decision that matters once the project is real — session state, tool ergonomics, and the deployment boundary that decides whether your agent runs at all.
Agent Engine will host your agent without you writing a server, managing sessions, or provisioning anything. Getting there means understanding four things the documentation mentions in passing and that account for most failed first deploys.
A postmortem. The deploy succeeds, the agent goes live, and every query returns nothing — or a type error from deep inside a framework you did not write. The cause is a serialisation boundary nobody told you existed, and the fix is two lines.
Feature tables do not help, because all three do the same core thing. What separates them is four constraints — where state lives, where it deploys, what you can see when it breaks, and what your team already knows. Pick on those and the choice makes itself.
Assigning agents roles and letting them converse is a genuinely useful abstraction for some problems and a determinism disaster for others. Meanwhile the most-cited framework in the category went into maintenance mode this year, which is its own lesson about what to depend on.
From an empty file to a server that Claude, Cursor and your own agent harness can all call. The code is short; the decisions that determine whether anyone can actually use it are the part worth slowing down for.
A server that works in the inspector and fails in practice usually fails for one of seven reasons. None of them are protocol bugs — they are interface decisions that only show up once a model, rather than a developer, is doing the calling.
Once an agent has tools, memory and retrieval, the job stops being how you word the instruction and becomes what occupies the window on every turn. That is a different skill, with different failure modes, and it is the one production work actually demands.
Split the window into fixed, retrieved and conversational budgets, put real numbers against each, and enforce them. The arithmetic is simple and almost nobody does it — which is why so many agents degrade at turn fifteen for reasons the team cannot name.
Vector search over data you could have queried with SQL is the most common architecture smell in agent systems. It is slower, less accurate, more expensive to maintain, and it fails in a way that is hard to detect. Here is the decision rule.
Fixed-size, semantic, and structure-aware chunking measured against real retrieval failures. The uncomfortable finding: your chunking choice barely moves whether the answer reaches the model, and hugely moves how much junk you pay for to get it there. Also, your chunk size is probably a library default nobody chose.
BM25 plus embeddings, then a cross-encoder over the top-k. The two stages buy different things and most teams cannot say which one they added. Hybrid search buys recall because lexical and vector retrieval fail on different queries; reranking buys precision because a cross-encoder reads the query and the document together. Here is what each one measurably adds, and how to tell when the reranker is buying you a rounding error.
Compaction is lossy compression and you are choosing the loss function. Anthropic's own probe of a compacted transcript recovered 3 of 3 high-level facts and 0 of 3 specific numbers — which is exactly what summarization is supposed to do, and exactly how an agent forgets the constraint it was given forty turns ago.
Microsoft measured GraphRAG beating vector RAG on 72-83% of sense-making questions. Microsoft then shipped LazyGraphRAG, which indexes at 0.1% of that cost. When the vendor builds the cheap version, the expensive version's price was the finding. Here is where a graph actually earns its keep, and the hybrid most teams land on instead.
Anthropic measured its orchestrator-worker research system beating a single agent by 90.2%, while burning roughly 15x the tokens of a chat. Cognition spent a year telling people not to build multi-agent systems, then shipped one under a single rule: writes stay single-threaded. Three topologies, the failure mode each one owns, and the one you should default to.
A handoff is not a message forward. Control does not return, so anything the caller knew and did not write down is gone the moment the transfer lands. OpenAI ships a filter to cut the trace, LangChain tells you to forward exactly two messages, and Google ADK isolates each subagent in its own session branch — three frameworks admitting the same thing. What travels across a handoff, and the three things that must never.
Fan-out looks like a speed decision and is actually a capacity decision. A sequential agent loop throttles itself — one request in flight, backpressure for free. The moment you dispatch fifty branches you have handed rate-limit management to the provider, and the provider manages it by refusing you. Three things to decide before you pick a library: how many run at once, what happens when one fails, and what you return when some succeed.
The three planning loops differ on one question: where does the plan live? ReAct holds a plan one step long, replans for free, and gets stuck. Plan-and-Execute commits to a graph up front, which buys parallelism and a cheaper executor and goes stale the moment reality disagrees. Reflexion wraps whole attempts in an outer loop and quietly depends on an oracle you probably do not have. Here is what each costs, with the numbers.
Which actions need a human, how to pause a running agent without losing its state, and why confirming everything trains people to click yes. The approval boundary is a claim about reversibility and blast radius, and the blast radius lives in the arguments, not the tool name. Includes the resume bug every framework has, and the aviation data that explains why your gate stops working in week three.
Checkpointing, idempotency keys and replay — what Temporal-style durability gives an agent that a retry loop cannot. The determinism rules that make ordinary Python illegal inside a workflow, why uuid4() is the wrong idempotency key, the 51,200-event ceiling that collides with your context budget, and the framework defaults that quietly hand the guarantee back.