Picture the demo. Your new AI agent takes a customer request and runs the whole thing end to end: it looks up the order, reads the refund policy, issues the refund, updates the customer record, and sends the confirmation email. Five steps, flawless, the room claps.
Three weeks later it is quietly issuing refunds against the wrong orders and abandoning a third of its sessions. The model did not get dumber. The demo just never told you the truth.
An AI agent is a computer program that uses a model, an AI system trained to produce text on command, to work through a task one step at a time: a step might be looking something up, updating a record, or asking another program for data, then it decides what to do next and repeats. That autonomy is what makes agents exciting. It is also exactly what makes them fragile in ways a demo will never show you.
The demo runs on the happy path
A demo is a controlled environment. The inputs are clean, the task is three to five steps, the APIs (the channels one program uses to ask another for data) are mocked or behaving, and it is one tidy turn. Production is none of that. Inputs arrive ambiguous and half-formed, the real task is fifteen to thirty steps, dependencies rate-limit you (cap how often you can call them) and change their schemas (the data formats they hand back) without warning, and a single session runs long enough that the agent forgets what it was told at the start. One analysis from Latitude found agents failing on roughly 63% of complex multi-step tasks in production.
They break in four recognizable ways: the agent drifts off the goal over a long session, it calls a tool (one action the agent can take, like looking up an order) with the wrong arguments, it loses the thread as the conversation overflows its memory, or it satisfies the literal instruction while missing the actual point. None of these is a failure of raw intelligence. They are what happens when clean logic meets a messy world.
Small cracks compound
Here is the part that surprises people. Say each step works 95% of the time. Sounds great, until the steps stack up: chain ten and the whole job lands only about 60% of the time, twenty and you are down near a third. Reliability multiplies, it does not average, so a strong model quietly becomes an unreliable agent as the task grows longer. And our refund agent that looked like five tidy steps on stage is closer to twenty in production, once you count the retries, the edge cases, and the checks it ought to be doing. So “our agent is 95% reliable” is not the reassurance it sounds like.
Worse, the failures hide. Our refund agent can make seven tool calls that each return a clean “200 OK” (the standard signal that a request succeeded) and still refund the wrong order, because step two grabbed the wrong order ID and step six refunded against it, and nothing ever raised an error. Some field reports put first-attempt completion for complex agent tasks as low as around 24%.
Why the benchmark lied
And the benchmark that sold you on the model? It scored one clean attempt on a tidy test set. Production is the opposite: messier inputs, so that 90% can slip to 70 or 80%, and what you actually want is an agent that works every time, not once in five tries. Long sessions make it worse, because models go vague on details buried in the middle of a sprawling conversation. The leaderboard number is a ceiling, not a floor.
The fix is boring engineering
The good news: none of the cure needs a smarter model.
Take fewer steps, since every one you remove is one less thing that can break. Check between steps, so an error gets caught at step three instead of surfacing, disguised, at step fifteen. And retry the safe ones, the highest-leverage move there is: one retry on each step of a five-step job lifts its success from about 77% to over 98%.
The rest is putting each safeguard where it belongs: strict tools so a bad call fails loudly, checks and retries between steps, a human gate on anything irreversible like moving money, and evals (automated tests that grade the agent) plus full logging wrapped around the whole run, so real failures come back as the next round of tests.
The pattern the best teams keep landing on is simple to say: separate thinking from doing. Let the model plan, let deterministic code (plain code that does the same thing every time) execute, and keep a human on the calls that cannot be undone. As one practitioner put it, the agents that survive in production are not the smartest, they are the most predictable.
So should you even build the agent?
Sometimes the most reliable agent is not an agent. If a task is well-defined and rule-shaped, a plain deterministic workflow beats an autonomous one every time, so reserve the agent for the genuinely ambiguous parts. When you do need one, keep it as simple as the job allows. The loudest current debate, whether to run one agent or a swarm, lands in the same place: Cognition argues for single-threaded agents (one agent working step by step) on tasks that change things, because parallel agents make conflicting decisions with no shared context (the background information each agent can see), while Anthropic uses an orchestrator with sub-agents (a lead agent handing pieces to helper agents) for research that fans out. Both agree the real hinge is context, and that multi-agent setups can cost roughly fifteen times as much to run. Add agents only where they earn it.
Where this leaves you
Gartner expects more than 40% of agentic (agent-style) AI projects to be cancelled by 2027, and today only about a third of companies have even one agent in production. The teams that make it are not the ones with the flashiest demo. They are the ones who treated reliability as the product.
That is the work we do at Gracient. We take the agent that dazzles on stage and build the parts nobody claps for: the tool contracts, the checks between steps, the retries, the human gates, the evals, and the tracing (a step-by-step record of what the agent did), so it holds up on the messy inputs and the twenty-step jobs. A great demo earns you a meeting. A reliable agent keeps you the customer. If yours shines in the demo and wobbles in production, the gap is not intelligence. It is engineering. Let’s talk.