AI agents in production: Lessons from 12 deployments

An agent that works in a demo and an agent that works on a Tuesday afternoon under real traffic are different systems, and the difference is almost entirely operational.

A prototype is judged on whether it can finish a task once, with a cooperative user and a clean input. A production agent is judged on the ambiguous ticket, the malformed order, and the API that times out mid-step.

Across clinic scheduling, order support and booking enquiries, the same lesson arrives. Model quality is rarely the constraint. What constrains an agent is how tightly its task is scoped, what it may touch, and what happens where it should stop.

Scoping a task an agent can finish

The commonest failure is a task defined by role rather than outcome. Handling support is not a task; issuing a refund below the automatic threshold, or handing the case over, is one.

A workable task has a concrete trigger, a bounded set of tools, a definition of done that is verifiable elsewhere, and an explicit exit. We test five conditions before building anything:

1. The trigger is a specific event, not a standing instruction to be helpful.
2. Every step the work needs is covered by the tools already given.
3. Success is verifiable from the system of record, not from the summary.
4. The worst realistic outcome is recoverable by a person the same day.
5. A volume estimate exists, so cost per run can be judged before launch.

Tasks that fail two or more are better shipped as a drafting step a person approves, which is often where the durable value sits.

Tool permissions and blast radius

We give agents narrow, purpose-built tools rather than general database or shell access. A refund tool that validates amount and order state is safer than a query runner behind a careful prompt, because the guard lives in code.

Writes are separated from reads and limited independently. They carry idempotency keys, so a retried step cannot refund twice, and destructive operations are unavailable or wrapped in a reversible soft delete.

The design question is not whether the agent will misbehave but what its worst available call costs. If the answer is unbounded, the tool needs a limit, not the prompt another sentence.

Escalation paths people trust

Every deployment needs a way to stop and hand over, and the handover has to carry context. An escalation that arrives as a bare transcript leaves the person slower than if they had started the case.

We pass a structured summary: what was asked, what the agent verified, which tools it called, and what is blocking. The correction that comes back is kept as an evaluation case.

Escalation rates need a floor as well as a ceiling. An agent that never escalates is usually guessing, and an unnaturally quiet queue is often the first sign of it.

Evaluation, cost and latency

Agents drift when prompts, tools, models or upstream data change, so each deployment carries measurements that run before a change ships and keep running afterwards.

• Golden cases: a fixed set of real conversations with expected tool calls, replayed on each change.
• Trace sampling: a share of live runs reviewed weekly by the team that owns the work.
• Cost per resolution: tokens and tool calls measured per completed task rather than per request.
• Latency budget: a ceiling per turn, met with a smaller model or a cached lookup.
• Loop guards: a hard cap on steps, so a stuck agent fails loudly instead of billing quietly.

Failure handling in the open

Agents fail in ways ordinary services do not. They succeed partially, report a failed step as a success, and can be confidently wrong in fluent prose. Logging the final answer hides all three.

We record the full trace and treat a partial run as an incident with an owner. The agents that stay in production are those where failure surfaces early enough to be boring.

• ai
• automation