"Agent" has become the word for any system with a language model and a tool in it, which makes it useless as a design term. The distinction that actually matters is narrow and testable: does your code decide what happens next, or does the model? Almost everything follows from that, including whether your system can be tested, budgeted and debugged.
The dividing line
A workflow orchestrates models and tools through predefined code paths. You wrote the sequence; the model does work inside it.
An agent lets the model direct its own process — choosing actions, ordering them, and deciding when the task is finished.
The question is not how sophisticated the system is. It is who owns the control flow.
Why the distinction is worth defending
It would be pedantry if the two had similar engineering properties. They do not.
| Workflow | Agent | |
|---|---|---|
| Steps | Known in advance | Decided at runtime |
| Cost per run | Predictable | Varies, sometimes wildly |
| Latency | Bounded | Unbounded without a cap |
| Testing | Step by step | Whole trajectories |
| Debugging | Find the failing step | Reconstruct a path that may not recur |
| Failure mode | A step errors | Loops, drifts, stops early |
| Handles the unforeseen | No | Yes — the whole point |
Every row except the last favours the workflow. That single row is why agents exist, and it is genuinely valuable — but you should be able to say out loud which unforeseen thing you are buying it for.
The building block underneath both
Neither is a starting point. Both are built from the same unit: a model call with something attached to it.
A great many production systems are one well-built augmented call and nothing more. That is not a failure to reach agenthood; it is the correct answer to a question that did not need more. Reach for structure only when a single call demonstrably cannot do the job.
Five workflow patterns
These cover the overwhelming majority of real systems. If your problem fits one, you do not need an agent.
1. Prompt chaining
Break a task into fixed steps, each consuming the last one's output. Use when the task decomposes cleanly and you would rather have several accurate steps than one call attempting everything.
The gate is the part people skip and then miss. A cheap validity check between steps stops you paying for three more calls on top of a bad first one.
2. Routing
Classify the input, then send it to a handler specialised for that class. Use when inputs fall into categories that genuinely want different treatment, and optimising one prompt for all of them makes each worse.
Routing is also the cheapest cost lever in this list: the easy 80% of traffic goes to a small fast model, and only the hard remainder reaches an expensive one.
3. Parallelisation
Two distinct shapes. Sectioning splits independent subtasks to run simultaneously. Voting runs the same task several times and aggregates, trading cost for confidence.
Voting suits problems where a false negative costs far more than a false positive — security review, safety filtering, anything where missing one is the expensive outcome.
4. Orchestrator-workers
A model breaks the task into subtasks, dispatches them to workers, and synthesises the results. This is where the line starts blurring — the subtasks are model-determined, but the orchestrate-dispatch-synthesise shape is fixed by you.
Use when you know the shape of the work but not its extent: "change this API across every file that uses it" has a predictable structure and an unpredictable file count.
5. Evaluator-optimiser
One model produces, another critiques, the first revises. Loop until the evaluator passes it or you hit a cap.
This works when you have clear evaluation criteria and iteration measurably helps — translation, writing to a brief, code that must pass tests. It fails when the evaluator cannot articulate what is wrong any better than the generator could, and you pay for rounds that change nothing.
💡 The always-cap rule
Every loop in this article needs a hard iteration limit, chosen by you, enforced in code. Not a prompt instruction to stop — a counter.
Loops that terminate only when a model agrees they should are the single most common source of runaway cost in LLM systems. The cap is not a safety net you hope never to hit; it is the thing that makes spend bounded.
What an agent actually is
An agent is the loop where the model decides both what to do and whether to continue.
Nothing in that loop says how many iterations it takes. That is the feature — and the cost. Three properties follow immediately:
- Feedback from the environment is what makes it work. An agent without real tool results is just a model talking to itself, and it will confidently proceed on imagined outcomes.
- Errors compound. A wrong step at iteration three shapes the context for every later one. Workflows reset at each stage; agents accumulate.
- You cannot bound the run without an explicit cap, because the model decides when it is finished.
⚠️ Autonomy is not a quality gradient
It is tempting to read workflow-to-agent as a progression from basic to advanced, and to feel that shipping a workflow means you did not finish. That is backwards.
Autonomy is a trade: you give up predictability, testability and bounded cost to buy the ability to handle situations you did not anticipate. If you did anticipate them, you have paid the price and bought nothing.
Deciding, honestly
Four questions, in order. Any "no" and you have your answer.
Question four is the one that gets waved through in planning and returns as an incident. "Variable cost" means a user can trigger a run costing fifty times the median. "Variable outcomes" means the same input can produce different results on Tuesday. Both are survivable — but only if you decided to accept them.
If you do build an agent
- Cap iterations, tokens and wall-clock time. All three, in code.
- Log the full trajectory — every decision, tool call and result. Without it, "it did something odd yesterday" is unanswerable.
- Keep the tool surface small. Each additional tool is another chance to pick the wrong one; fewer, well-described tools beat a large menu. Getting these descriptions right is most of the work, and tool schema design is where the leverage is.
- Make consequential actions require confirmation. An agent that can act irreversibly without a human is a prompt injection away from acting on someone else's instructions.
- Test trajectories, not outputs. The interesting failures are in how it got there.
- Start with the workflow anyway. Ship it, find the specific cases it cannot handle, and let those justify the loop. That list is also your evaluation set.
Building pipelines that pass data between steps?
Format, convert and validate JSON, CSV and YAML in your browser — nothing is uploaded to a server.
Open JSON Formatter →Summary
- Workflows follow code paths you wrote. Agents choose their own. That is the whole distinction.
- Every engineering property except adaptability favours the workflow.
- Both are built from the augmented LLM call, and often one call is the right answer.
- Five patterns cover most systems — chaining, routing, parallelisation, orchestrator-workers, evaluator-optimiser.
- An agent is a loop with model-controlled termination. Unbounded unless you bound it.
- If you can draw the flowchart, build the flowchart.
- Autonomy is a trade, not an upgrade — predictability sold for adaptability.
- Cap everything, log trajectories, keep the tool surface small.
Frequently Asked Questions
What is the difference between an AI agent and a workflow?
In a workflow, the sequence of steps is written in your code — the model does work at each step but does not choose what happens next. In an agent, the model itself decides which action to take, in what order, and when to stop. The dividing line is who controls the control flow: your code, or the model.
When should I use an agent instead of a workflow?
When you genuinely cannot predict the steps in advance — the number of actions varies by input, the path depends on what earlier steps discover, and enumerating the branches in code would be impractical. If you can draw the flowchart, you do not need an agent to walk it.
Why are agents harder to run in production?
Because their cost, latency and behaviour are all variable. An agent may take three tool calls or thirty for superficially similar inputs, so you cannot bound spend or response time in advance. They are also non-deterministic, which makes testing, reproducing bugs and reasoning about failures substantially harder than for a fixed pipeline.
What are the main LLM workflow patterns?
Five cover most real systems: prompt chaining, where each step feeds the next; routing, where an input is classified then sent to a specialised handler; parallelisation, running independent calls at once and combining them; orchestrator-workers, where a model splits a task into subtasks it dispatches; and evaluator-optimiser, where one model critiques another's output in a loop.
Is a single LLM call with tools an agent?
Not usually. If the model can call a tool and then answers, you have an augmented LLM call. It becomes an agent when the model runs in a loop, deciding after each result whether to act again and when it is finished — the loop with model-controlled termination is what makes it agentic.