What Multi-Agent Systems Actually Are

🧭 Most systems called multi-agent are just workflows

The term multi-agent gets stretched so far that it stops being useful. A team wires together several model calls, adds a planner prompt, maybe runs a reviewer in parallel, and the whole thing gets described as a multi-agent system. It sounds plausible because more than one model instance is involved. But that alone does not change the architecture. If the code still decides the route in advance, you do not have distributed decision-making. You have a workflow with multiple steps.

That distinction matters because control is the real design variable. A workflow can be complex, conditional, and even highly effective while still being centrally defined. The developer decides the graph. The model fills in content within that graph. In a true multi-agent system, at least one agent participates in deciding what happens next. It can delegate, route, hand off, or request critique based on what it sees in the current task state. Anthropic makes a closely related distinction in its guide to building effective agents, separating predefined workflows from more dynamic agent-directed processes.

Why this matters in practice is simple. You debug workflows by inspecting prompts, branch conditions, and tool results. You debug multi-agent systems by inspecting those things plus the reasoning around delegation. Why was a subtask created at all? Why was it assigned to that role? What context was included in the handoff, and what was silently omitted? Once those decisions happen at runtime, they become part of the system’s behavior, not just an implementation detail. Using the right label keeps the engineering problem visible.

🧩 A practical definition: pipeline, workflow, then coordination

The cleanest way to understand this space is to separate three levels of control. First comes the pipeline. A pipeline is fixed sequential execution. Step one runs, then step two, then step three. There is no meaningful branching and no runtime choice about structure. Pipelines are common in production because they are easy to reason about, easy to test, and often completely sufficient for narrow tasks like extraction, formatting, or classification followed by a templated response.

Next comes the workflow. A workflow is still predefined, but it can include richer control logic. You can branch on confidence, retry a failing tool call, run two tasks in parallel, or put an evaluator in an agent loop until a condition is met. This gives you flexibility without surrendering control. The important thing is that the graph still exists before the request arrives. The system reacts within the boundaries the developer already designed.

Then comes coordination. A multi-agent system begins when the next-step decision is not fully pinned down by code. An agent looks at the live state of the task and decides whether to create subtasks, which specialist should receive them, whether review is needed, or whether no delegation is necessary at all. OpenAI’s examples of agent handoffs in the Agents SDK quickstart are useful because the handoff is not just a fixed chain. It is a routing choice.

A concrete example helps. Imagine a support system for incoming tickets. In a pipeline, every ticket gets summarized, classified, and answered in the same order. In a workflow, high-risk tickets might trigger a review branch while simple tickets skip it. In a multi-agent design, a triage agent decides whether billing, technical troubleshooting, fraud review, or direct resolution should take over. That difference seems small on paper, but it changes who owns structure. Once the route is chosen dynamically, you are dealing with agent orchestration rather than mere sequencing.

🔄 What changes when delegation happens at runtime

Runtime delegation sounds like a small feature until you see what it does to the system boundary. In a pipeline or normal workflow, there is one explicit control graph. In a multi-agent setup, there are multiple interacting control loops. One agent can create work for another. A reviewer can reject output and request revision. A specialist can escalate back to a planner because the task was poorly scoped. These are not just more model calls. They are additional decision centers.

That changes agent reliability immediately. Every handoff creates a place where meaning can degrade. The planner may know the full objective, the budget constraint, the risk level, and the final output format. A specialist may receive only a short task summary. If the summary drops a critical constraint, the specialist can produce work that looks correct in isolation but is wrong for the overall goal. The failure is not in generation quality alone. It is in transfer quality.

This is also why runtime state becomes a first-class concern. OpenAI’s post on equipping the Responses API with a computer environment emphasizes persistent context and compact state for long-running agents. That matters even more in multi-agent settings because a delegating agent has to decide what to carry forward. Should it pass raw history, a summary, references to external artifacts, or only a strict task contract? Those are agent state management choices, not prompt tweaks.

The long-term impact is easy to miss at first. A single-agent system mostly needs good tool use and coherent memory. A multi-agent system needs those things plus stable delegation behavior. If delegation is inconsistent, the same request can take different paths across runs, making behavior harder to explain and harder to evaluate. That is why teams often underestimate the cost of just adding another agent. The real cost is not one extra model call. It is the introduction of another controller inside the system.

🛠️ The real design problem is responsibility boundaries

Titles like planner, worker, reviewer, and executor are useful shorthand, but they are not the real design. The real design is the boundary of responsibility attached to each role. Who may decompose the task? Who may access external systems? Who may spend expensive tokens or trigger risky actions? Who can declare the answer complete? If those boundaries are unclear, your architecture may still run, but it will be difficult to trust and even harder to debug.

A planner is not merely the model that thinks first. It owns decomposition. It decides whether subtasks should exist at all, how coarse or fine they should be, and what good completion looks like. A specialist is narrower by design. It might receive less context, have access to only one tool family, or follow stricter instructions because it is meant to perform one type of work consistently. An executor is different again. It is coupled to actions in an environment, such as making API requests, querying a database, or operating inside a sandbox. A reviewer has a separate kind of authority: it can reject work, request revision, or compare output against acceptance criteria.

These distinctions matter because they map directly to risk. If every role can do everything, the architecture becomes fuzzy. A planner starts calling tools it was not supposed to use. A reviewer begins rewriting content instead of evaluating it. A specialist silently changes shared state because no one said it could not. The result is not flexibility. It is loss of accountability. When something breaks, you cannot tell whether the problem was poor decomposition, poor execution, or poor review because your roles never had clean boundaries in the first place.

This is why patterns like supervisor worker agents and evaluator-optimizer are helpful. Anthropic uses these patterns to show that many systems with several model calls are still centrally controlled, which is often a good thing. LangGraph pushes a similar idea in its guidance on explicit and traceable control flow. Clear role boundaries do not make a system less intelligent. They make it understandable enough to operate in production.

📦 Handoffs are where architecture becomes a systems problem

Most writing about agent handoffs treats them like a convenience feature. In real systems, handoffs are one of the hardest parts of the design because they force you to package context under constraints. Every time one agent gives work to another, you have to decide what survives the transition. Does the receiving agent get the full history, a summary, a task brief with acceptance criteria, links to artifacts, or some combination of all four? Each option changes cost, latency, and failure behavior.

If you pass the whole history, token cost grows and signal quality often drops. Long conversations carry irrelevant details, stale assumptions, and previous attempts that may confuse the next role. If you pass only a summary, the system becomes cheaper and cleaner, but subtle constraints can disappear. If you depend on external artifacts, such as a shared memory store or a workspace document, then retrieval quality becomes part of the control loop. A good routing decision can still fail if the wrong artifact is retrieved or if the artifact is out of date.

A useful way to think about a handoff is as a contract. The contract should tell the next agent what the goal is, what constraints must not be violated, what inputs are authoritative, what outputs are expected, and what conditions require escalation. Without that contract, the next agent has to infer too much. Inference is sometimes necessary, but too much inference during transfer is where systems become brittle.

This matters for long-term system health because handoffs are where local intelligence meets global coordination. A specialist may be excellent at its narrow task and still fail repeatedly if incoming task packages are ambiguous. Teams often respond by tweaking the specialist prompt, when the real fix is upstream: the planner or router needs to produce better contracts. In practice, many agent quality problems are really handoff design problems wearing a different name.

🧠 Memory and state are part of the architecture, not a convenience

Developers often talk about memory as if it were a single feature, but AI agent memory becomes more complex once several agents interact. One role may need the latest conversational context, another may need durable facts from earlier sessions, and a third may only need a procedural checklist for how to complete a task safely. This is where short-term vs long-term memory in agents stops being a theoretical distinction and becomes an operational one.

Short-term memory helps an agent stay coherent within the current task. Long-term memory helps it preserve durable information across tasks or sessions. There is also procedural memory for agents, which captures repeatable patterns, working rules, or compact task strategies. If those memory types get mixed together carelessly, routing and delegation degrade. A worker may receive noisy conversation history when it only needs a structured task contract. A planner may miss a durable constraint because it was buried inside short-lived context.

This is why frameworks such as LangGraph emphasize memory design and persistent state, including patterns often discussed under LangGraph memory. For AI coding agents and other systems that operate over extended timelines, memory is inseparable from control flow. The question is not simply whether the system remembers. It is what each role remembers, how that memory is updated, and whether the memory boundary matches the responsibility boundary.

⚙️ Tool use and execution loops change the risk profile

Many modern agent systems rely on tool calling agents. That sounds straightforward until multiple agents can choose tools, retry tools, or escalate when tool output conflicts with the task objective. At that point, the agent loop is no longer just request, tool call, response. It becomes a control pattern with authority boundaries, budgets, and stopping conditions.

This matters because tool use is where model judgment touches external reality. A specialist that queries a database, edits a file, or invokes code execution introduces different risks than a planner that only decomposes work. For developers building Responses API agents or systems with the OpenAI Agents SDK, the practical question is not only whether tools are available. It is which role can use which tool, under what state conditions, and with what review requirements before the result is accepted.

Once you see tool use this way, agent architecture becomes less about prompts and more about controlled execution. That shift matters because many production failures are not language failures. They are action failures. The system did the wrong thing, at the wrong time, with the wrong context, while following an execution loop no one fully instrumented.

⚠️ More agents create new failure layers, not just more capacity

It is tempting to assume that more agents simply means more parallelism or more expertise. Sometimes it does. But it also creates new failure layers. A single-agent system can already fail by choosing the wrong tool, stopping too early, looping too long, or forgetting a constraint. A multi-agent system keeps all of those risks and adds coordination failures on top.

The first added layer is decomposition failure. The planner may split the task into the wrong subtasks, create unnecessary subtasks, or fail to notice that some work depends on another part being finished first. The second layer is assignment failure. Even with good decomposition, the router can send the work to the wrong specialist or assign multiple specialists overlapping jobs. The third layer is transfer failure, where the handoff omits key context. Only after that do you reach execution failure at the specialist level itself.

Then there is synthesis failure. Several subagents can each produce sensible partial outputs, yet the combined result can still be incoherent, contradictory, or misaligned with the original objective. Review failure is another separate category. The evaluator might miss the defect, approve low-quality work, or reject a correct result because its criteria are poorly framed. Looking at the system through these layers matters because it gives developers a practical way to localize blame instead of treating the entire architecture as an opaque black box.

Anthropic’s write-up on its multi-agent research system is especially useful here because it surfaces a non-obvious downside: subagents sometimes duplicated investigation. That is not a minor inconvenience. It means the architecture spent more money and more latency to learn the same thing twice. This is the broader lesson. Decomposition can improve breadth and speed, but if partitioning is weak, more agents simply multiply inefficiency. Capacity and complexity rise together.

🔍 Observability gets harder faster than people expect

Once control is distributed, agent observability becomes a design requirement rather than a nice extra. In a single-agent system, a trace often tells a fairly coherent story. You can inspect the input, the tool calls, the intermediate thoughts if available, the output, and the stopping decision. With multiple agents, that is no longer enough. You also need to understand how control moved between roles.

That means traces must capture parent-child run relationships, delegation edges, timestamps, handoff payloads, artifact references, reviewer decisions, and the synthesis path that produced the final answer. Without that structure, a poor result is very hard to diagnose. You might see five model calls and three tool calls, but still have no idea whether the root problem was bad routing, weak context transfer, redundant decomposition, or an overzealous reviewer.

The important shift is conceptual. Observability in multi-agent systems is not just logging text. It is logging control flow. You want to know not only what each agent said, but why it got involved, what authority it had, what context it received, and what outcome its participation changed. That is much closer to systems engineering than to ordinary prompt engineering.

This is also why agent evaluation and tracing matter together. Evaluation without traces tells you that the system failed. Traces tell you where and why it failed. OpenAI’s emphasis on orchestration and traces, along with LangGraph’s preference for explicit node boundaries, points in the same direction. Distributed control without strong observability produces impressive demos and weak operations. The more autonomy you introduce, the more your traces need to function like a map rather than a transcript.

⏳ Long-running agents need durability, not just memory

It is easy to think memory alone is enough for extended tasks, but long-running agents need something stricter: recoverable execution. If a process spans multiple tools, asynchronous events, reviews, or handoffs, the system has to preserve not only context but progress. This is where durable execution for agents becomes essential.

Why this matters is practical. A developer agent that analyzes a repository, opens files, plans edits, runs tests, and then waits for a callback cannot safely rely on one continuous prompt history. It needs resumable state, explicit checkpoints, and clear ownership of pending actions. Without that, a crash or timeout does not just interrupt the conversation. It can break the task structure itself.

For teams designing production systems, durability is part of agent state management, not an add-on. If handoffs, tool results, reviewer decisions, and memory updates cannot survive retries or restarts cleanly, reliability remains fragile no matter how good the prompts look in a demo.

💸 When not to use multi-agent systems

There is a common idea that multiple agents represent a more advanced architecture by default, as if a single-agent system is only a temporary stepping stone. In practice, that is often backwards. If the task is stable, the steps are well understood, and the cost of failure is high, a deterministic workflow is usually the better choice. It is cheaper to run, easier to evaluate, easier to secure, and easier to explain to the rest of the organization.

You should be especially skeptical when the proposed benefit is mostly cosmetic. If one well-instrumented agent can call the necessary tools, maintain enough state, and complete the task reliably, adding planners, critics, and specialists may only increase token usage and latency. It also expands the evaluation burden. You are no longer testing only final task completion. You are now testing routing quality, handoff quality, synthesis quality, and reviewer behavior, each of which can fail independently.

The key question is whether dynamic delegation creates real task-level value. It often does when multiple independent lines of investigation can run in parallel, when specialist isolation improves quality or safety, or when live reassignment is necessary because the task evolves unpredictably. If those gains are absent, the extra control loops are usually a liability rather than a sign of maturity.

This is why the gradual progression from simple workflows to more dynamic agents is such a useful framing. Complexity should be earned. A well-designed workflow is not primitive. It is disciplined. Teams that skip that discipline often end up wrapping an avoidable coordination problem around a task that never needed one.

🧪 A simple classification test developers can actually use

A practical test cuts through most of the marketing language. Ask yourself this: if I know in advance exactly which model calls will run and in what structure, would I still call this multi-agent? If the answer is yes, you are probably using the term too loosely. A genuine multi-agent system contains at least one meaningful runtime decision about delegation, handoff, reassignment, or critique that is not fully predetermined by code.

Consider a customer support assistant. If every case always runs classification, answer drafting, and review in the same order, that is a workflow. If a triage agent decides whether the case belongs with billing, technical troubleshooting, fraud review, or direct resolution, that is coordination. The difference is not the number of prompts or the number of model instances. The difference is who owns the next-step decision when the system is live.

You can push the test further. Suppose your code always launches three research subagents because the task template always has three known categories. That is parallel workflow execution, not necessarily a multi-agent system in the stronger sense. But if an agent decides whether there should be zero, one, or five subagents based on the ambiguity, scope, and current evidence in the task, then the structure itself is being chosen at runtime. That is the architectural threshold that matters.

The final takeaway is simple. Choose a pipeline when the route should never vary. Choose a workflow when the route can vary, but only within a graph you trust enough to define upfront. Choose multi-agent systems when live delegation materially improves the task enough to justify extra cost, extra observability needs, and extra failure modes. For AI agents for developers, that framing keeps the term honest, and more importantly, it keeps the system understandable to the people who have to build, debug, and maintain it.