How to Design a Single-Agent Workflow Before You Add Complexity

🧠 Most complexity enters the system before intelligence does

Many teams say they are building a multi-agent system when what they really have is a narrow workflow with unnecessary handoffs. One component extracts an ID, another decides whether to call a tool, another formats the result, and suddenly the architecture looks sophisticated. In practice, it is often just more places for the system to fail. The hard part is not inventing more agents. The hard part is designing a workflow whose boundaries are clear enough that failures can be observed, explained, and fixed.

This matters because every extra handoff introduces another layer of nondeterminism. One model has to interpret what another model meant. One context window has to be compressed into another. One trace becomes three traces stitched together after the fact. That usually increases latency, token usage, and debugging cost long before it improves quality. OpenAI’s current guidance on agent builder workflows makes this distinction explicit: a workflow is the combination of agents, tools, and control flow. That means the design unit is not just the model. It is the whole execution path.

The practical lesson is simple. Start by asking what the system must reliably do, not what kind of LLM agent architecture sounds advanced. If the task can be handled by deterministic validation, one bounded agent loop, and deterministic post-processing, then that is the right starting point. A narrow single-agent workflow is not a temporary compromise. It is often the cleanest form of the problem because it gives you a smaller system to inspect when reality disagrees with your design.

🎯 A single-agent workflow needs a real contract

The phrase “tight contract” sounds abstract until you turn it into fields the system can actually enforce. A strong single-agent workflow has a goal, a set of allowed tools, explicit forbidden actions, required inputs, termination conditions, and a fallback policy. That is not prompt style. That is system design. It gives you stable surfaces for testing and makes it much easier to answer a basic production question: what exactly was the agent allowed to do when it failed?

Consider an order-status assistant. Its goal might be to resolve order-status requests. Its allowed tools might be get_order_status and ask_for_order_id . Its forbidden actions might include refunds or address changes. Its required input is an order ID. Its stop conditions are either that status has been returned or that the user failed to provide an ID after one clarification turn. Its fallback is escalation to a human. This contract is small, but it already removes a lot of ambiguity. The model no longer has to infer the task boundary from prose alone.

Why does this matter? Because vague prompts create invisible policy. The model might choose to be helpful in ways your system cannot support. It might improvise steps, over-ask, or keep trying after it should stop. Once the contract is explicit, you can grade traces against it. You can ask whether the agent violated a forbidden action, stopped too late, or failed to request a required input. That is far more useful than debating whether the prompt felt clear.

agent = {
"goal": "resolve order status requests",
"allowed_tools": ["get_order_status", "ask_for_order_id"],
"forbidden_actions": ["modify_order", "issue_refund"],
"required_inputs": ["order_id"],
"termination_conditions": ["status_returned", "missing_id_after_1_retry"],
"fallback_policy": "escalate_to_human"
}

It helps to read this contract line by line. The goal narrows what good behavior means. Without it, the model may optimize for general helpfulness instead of task completion. The allowed_tools define the action space. This matters because tool choice errors often look like reasoning errors from the outside. The forbidden_actions tell the system what it must never do, even if the user asks or the model believes it would be useful. The required_inputs define the minimum viable state for progress. The termination_conditions prevent wandering loops. The fallback_policy determines what safe failure looks like.

From a developer’s perspective, the contract is also where interfaces become testable. You can write fixtures for missing IDs, malformed IDs, forbidden refund requests, or repeated clarification turns. That makes reliability measurable. Without a contract, you are not testing behavior against a spec. You are testing behavior against a hope. And hope is a poor dependency in production systems.

⚙️ Keep exact logic in code and fuzzy judgment in the model

A reliable single-agent workflow depends on a clean split between orchestration and model judgment. If a rule can be expressed exactly, it belongs in code. If a decision depends on messy language, ambiguity, or incomplete user input, that is where the model earns its keep. This is one of the most useful design heuristics in modern LLM agent architecture because it stops teams from hiding business logic inside prompts.

Retries are a good example. A prompt can tell the model to retry when appropriate, but that is not enough. Retry budgets, exponential backoff, idempotency checks, timeout handling, and permission boundaries should live in orchestration code. The model can decide whether the user is asking for billing or order status. The application should decide whether a failed lookup can be retried safely. That distinction sounds small, but it changes how failures behave under load.

Why this matters becomes obvious in real systems. Imagine that get_order_status times out. If retry behavior lives in the prompt, the model may call again with slightly different wording or keep asking the user to wait. If retry behavior lives in code, you can enforce one retry, log both attempts, preserve idempotency, and escalate when the budget is exhausted. The difference is not elegance. It is operational control.

Prompts are a weak place to store operational truth. They are hard to audit, easy to drift, and difficult to test in isolation. Code-based orchestration can be versioned, unit tested, replayed, and reasoned about with standard engineering tools. It also gives you safer resume behavior. If a tool call times out, you want to know whether replaying the step would duplicate a side effect or simply repeat a lookup. That kind of checkpointing logic cannot be delegated to the idea that the agent should be careful.

Once you adopt this split, the system becomes easier to trust. The model handles interpretation. The code enforces invariants. That is the difference between an agent that seems clever in a demo and one that survives contact with production traffic.

🧩 Narrow tool scope reduces the real failure surface

Developers often hear that narrow scope is good because it is simpler. That is true, but it undersells the point. Narrow scope matters because every extra tool expands the system’s failure surface in structural ways. It adds schema tokens to the prompt, increases overlap between tool descriptions, raises the chance of incorrect selection, and creates more paths that traces need to explain later. The issue is not elegance. The issue is error rate.

Many agent failures are not high-level reasoning failures at all. They are malformed arguments, missing fields, wrong identifiers, or tools called with half-extracted parameters. If you are resolving order status, the system does not benefit from exposing refund tools, profile tools, and broad web search just in case. It benefits from two or three sharply defined actions with short, discriminative descriptions and strict argument schemas. See OpenAI’s notes on correct function calling for the mechanics.

This is especially relevant for tool calling agents. When tool definitions overlap, the model has to choose between actions that look semantically similar, which raises the chance of bad calls and noisy traces. A smaller action space improves tool selection, keeps agent orchestration readable, and makes failures easier to diagnose.

There is also a cost reason. More tools mean larger repeated prefixes in the prompt. OpenAI’s prompt caching documentation makes clear that tool definitions and schemas contribute to the cacheable prefix. A stable, narrow workflow tends to produce better cache reuse. That lowers spend and improves latency. In other words, scope control is not just about cleanliness. It directly affects token economics.

A good operational tactic is dynamic tool filtering. If the user is unauthenticated, hide account-changing tools. If the conversation is still in triage, expose only lookup and clarification actions. This achieves many of the benefits people reach for with multiple agents, but without introducing handoffs. Very often, what looks like a need for agent specialization is really a need for smaller action spaces at the right moment.

🔍 The best first step is often a workflow, not an agent

The word agent has a way of making teams skip a simpler design that would work better. A lot of production tasks do not need open-ended decision loops. They need a mostly deterministic workflow with one or two judgment points. OpenAI’s evaluation examples show this clearly in customer support flows. A workflow version can deterministically extract an order ID, look up the record, and format the response. A single-agent version lets the model decide whether it already has enough information or needs to ask for clarification. The difference is meaningful, but still small.

This matters because the cleanest system is often the one that minimizes where nondeterminism enters. If extraction can be deterministic, make it deterministic. If routing can be done with a small classifier or explicit rules, do that. Then put the model where ambiguity actually lives. That could be deciding whether a user is really asking for shipping status versus cancellation policy, or synthesizing retrieved documentation into a grounded answer.

A documentation assistant is a good example. Query rewriting and retrieval can happen outside the agent, then the model reasons over the retrieved context and optionally calls a small set of tools. That is a strong pattern because retrieval quality, permissions, and indexing stay in the orchestration layer where they belong. The model is not burdened with pretending to be an information pipeline. It focuses on interpretation and synthesis.

Teams often reverse this. They hand one agent a wide prompt, broad retrieval access, and mutation tools, then wonder why the traces are noisy. The answer is usually not that they need more agents. The answer is that they allowed the model to do jobs the workflow should have done. Start with a workflow. Add an agent only where judgment is genuinely needed.

🛑 Stop conditions are where reliability becomes visible

A surprising number of agent failures are really stop-condition failures. The system keeps asking after it has enough information. It keeps calling a tool after the request has clearly failed. It loops because the prompt says be thorough but nowhere does the architecture define what done means. This is why explicit termination rules matter so much in a single-agent workflow. They convert a vague interaction into a bounded loop.

Useful stop conditions are concrete. Success criteria met. Required input missing after one clarification turn. Tool failure exceeded retry budget. Confidence too low, escalate. Policy requires human review. Maximum steps reached. These conditions should be enforced outside the prompt whenever possible, because they are part of control flow. The model may contribute a judgment such as I still do not have a valid order ID, but the workflow should decide that one clarification attempt is enough.

Why does this matter operationally? Because looping behavior is expensive and confusing. It inflates latency, burns tokens, and makes traces harder to interpret. It also creates poor user experience. A system that asks twice for the same identifier feels less intelligent, not more. In production, that frustration often gets misdiagnosed as a model-quality problem. In reality, it is usually an architecture problem: the workflow failed to define a hard stop.

This is also where durable execution for agents matters. If the system pauses for human review or a tool outage, you need to know what step was completed, what state is valid, and whether resuming will repeat a side effect. Clear stop and checkpoint boundaries make that possible. Without them, the agent loop becomes a blur. And blurry systems are almost impossible to debug under pressure.

🧠 Memory should support the workflow, not replace it

As teams build more capable AI coding agents and support assistants, they often assume memory will compensate for weak workflow design. It usually does not. Good AI agent memory helps the agent preserve useful context, but it cannot rescue unclear contracts, weak stop conditions, or poor tool schemas. Memory is support infrastructure, not architecture.

This is where short-term vs long-term memory in agents matters. Short-term memory holds the current task context: the latest user request, extracted identifiers, recent tool outputs, and the current stage of the workflow. Long-term memory stores reusable preferences, historical facts, or patterns worth carrying across sessions. If you mix these carelessly, the agent may use stale state, overfit to old interactions, or leak irrelevant context into the current task.

For a single-agent design, strong agent state management usually means keeping the active state small and explicit. The current order ID, retry count, latest tool result, and escalation flag should live in structured workflow state. Persistent memory should be added only when it improves a measured use case, such as remembering a user’s preferred environment or repository conventions in a developer tool.

This distinction matters because hidden memory often creates hidden bugs. A workflow with explicit state is easier to test, replay, and trace. A workflow that silently depends on recalled context is harder to reason about. If you later adopt frameworks that support persistent state, such as LangGraph memory, or platform patterns built around Responses API agents and the OpenAI Agents SDK, you still want the workflow contract to define what memory is allowed to influence. Otherwise memory becomes another source of nondeterminism rather than a source of useful continuity.

There is also a practical lesson for procedural memory for agents. If the system repeatedly learns a useful sequence, such as how to inspect a repository, run a test, and explain a failing assertion, that sequence should often become code or orchestration logic instead of free-form remembered behavior. The reason is simple: repeatable operations are stronger when they are explicit.

📊 Traces tell you whether you need a better workflow or another agent

Before adding a second agent, inspect traces. This is where a lot of architectural confusion disappears. OpenAI’s trace grading guidance emphasizes that traces reveal why a run succeeded or failed, not just whether the final answer looked acceptable. That distinction matters. A response can be correct for the wrong reason, and that usually breaks later at scale.

Imagine a support assistant that fails on 12 percent of runs. Without traces, the team might conclude the domain is too broad and split into a billing agent, shipping agent, and account agent. But trace inspection may show a different story. Perhaps the model repeatedly calls get_order_status with malformed IDs. Perhaps it asks for clarification even when the ID is already present. Perhaps it lacks a clear stop condition after one failed lookup. None of those problems require another agent. They require stricter schemas, better input validation, and clearer control flow.

A useful eval rubric for a single-agent workflow includes several trace-level questions: Did the agent choose a valid tool for the user intent? Were arguments complete and schema-compliant? Did it ask for missing information only when necessary? Did it stop at the right time? Did retries follow orchestration policy? Did the final answer reflect validated tool output rather than model invention? These are much better signals than a vague thumbs-up on final quality.

This is why agent evaluation and tracing sit at the center of reliable systems. Traces make agent observability concrete. They show where the workflow drifted, where the model overreached, and where the control logic failed to contain ambiguity.

Why this matters is simple. Multi-agent systems make trace interpretation harder by design. Now you need to understand which agent made the wrong judgment, whether the handoff dropped context, and whether the downstream agent failed because of its own prompt or inherited bad state. If the current trace already points to a fix inside one bounded loop, adding more loops is the wrong move.

🧪 Evals should earn complexity, not justify it after the fact

There is a common failure pattern in AI teams: architecture decisions get made by intuition, then evals are added later to defend them. That order should be reversed. A single-agent workflow should stay the default until measured evidence shows it has reached a real limit. OpenAI’s evaluation best practices explicitly warn that multi-agent systems add nondeterminism and complexity.

So what evidence should trigger escalation? One signal is tool selection quality degrading even after dynamic filtering and schema tightening. Another is a subtask that needs context heavy enough to pollute the main thread, such as large retrieval bundles or a specialist reasoning frame that hurts other steps. A third is meaningful parallelism. If two subtasks can run independently and doing so lowers latency or improves reliability, decomposition may be justified. If none of these are true, a single-agent design usually still has room to improve.

The developer benefit here is discipline. You stop using architecture as a substitute for diagnosis. If evals show that wrong arguments dominate failures, fix schema design. If traces show the model is overloaded by overlapping tools, reduce the action space. If stop behavior is unclear, encode termination outside the prompt. These are less glamorous fixes than drawing a supervisor worker agents diagram, but they are usually the fixes that make systems reliable.

In other words, complexity should be earned. Not because simplicity is morally better, but because every new agent adds another place where cost, latency, trace fragmentation, and coordination errors can hide. Measured pain justifies that trade. Architecture fashion does not.

🧵 A concrete design pattern for a strong first version

If you want a practical starting point, use a four-part layout. Begin with deterministic ingress validation. Confirm authentication state, normalize obvious fields, and reject malformed inputs before the model sees them. Then add an optional routing or classification step if the request type must be narrowed, such as distinguishing billing from order status. After that, call one agent with two to five tools at most. End with deterministic post-processing that validates tool outputs, formats the response, and records trace metadata.

This pattern works because it protects the agent from jobs it is bad at. It does not need to infer permissions from prose. It does not need to memorize retry budgets. It does not need to enforce JSON correctness after the fact. The workflow handles those things. The model handles the part that actually benefits from language understanding.

A realistic order-status flow might look like this in prose. The request enters. Code checks whether the session is authenticated. If the message appears to contain an order ID, code normalizes its format. A lightweight router decides whether the request is truly about order status or belongs to billing support. The single agent then has access only to ask_for_order_id and get_order_status . If a valid ID exists, it calls the lookup tool. If not, it asks once for clarification. Post-processing checks that the returned status is valid for display, then formats the final answer. If the lookup fails twice, the workflow escalates.

Why this pattern matters for long-term health is that each stage owns a different kind of uncertainty. Validation handles malformed input. Routing handles task narrowing. The agent handles language ambiguity. Post-processing handles output correctness and presentation. When a failure appears, you know where to look first. That shortens debugging loops and prevents the all-too-common habit of editing the prompt every time anything goes wrong.

Notice how little of this depends on a giant prompt. That is the point. A strong first version is not one where the model knows everything. It is one where the system is arranged so the model has fewer chances to guess wrong.

🚦When a single agent has actually reached its limit

There are real cases where one agent is no longer the right abstraction. The mistake is not moving to multi-agent systems. The mistake is moving there too early. A single agent has likely reached its limit when the action space remains confusing even after tool filtering, when one specialized task needs a very different context that consistently harms the main loop, or when parallel execution would materially reduce end-to-end latency.

Take an internal operations assistant that starts with incident triage. Early on, one agent may classify urgency, pull recent service status, and draft a response. Later, evals might show that log analysis requires a much larger toolset and context window, while communication drafting benefits from a separate prompt and policy frame. If the traces show context pollution and degraded tool choice inside one loop, that is a legitimate reason to split responsibilities.

Even then, the right move may not be many agents everywhere. It may be one coordinator with one specialized subagent only for the overloaded subtask. The principle still holds: introduce the minimum extra complexity that solves the measured problem. Do not use a team of agents to compensate for weak schemas, unclear stop logic, or prompt-buried business rules. That is not decomposition. That is avoidance.

If you do reach that point, be specific about the coordination pattern. Ask whether you truly need agent handoffs, whether a narrower specialist would help more than a broad coordinator, and whether the task really benefits from long-running agents with persistent state. Many failures blamed on scale are actually failures of scope and state management.

The reflective point here is that architecture should follow failure evidence. Not ambition, not demos, not hype. A single agent has reached its limit when the traces keep proving that bounded improvements inside one loop no longer fix the core issue. Until then, the shortest path to reliability is usually the narrow path.