Why offline benchmarks lie, and the observability you actually need once agents are live.
A benchmark suite tests the agent against questions someone anticipated. Production traffic is the questions nobody anticipated. An agent that scores well offline can still fail constantly on the long tail of real, slightly odd user input that never made it into the eval set.
Offline evals are still worth running — they catch regressions before deploy — but treating a high benchmark score as evidence the agent is production-ready is the most common and most expensive mistake teams make.
Every tool call, every intermediate reasoning step, and every fallback or retry needs to be logged and traceable back to the originating request. Without this, debugging a bad agent outcome means staring at a final answer with no way to see how it got there.
Tracking task completion rate, intervention rate (how often a human had to step in), and cost per successful task gives a far more honest picture than any single accuracy number.
Agents fail in production in ways static test sets rarely capture: looping on a tool call that returns an unexpected error, silently truncating a long context, or confidently completing a task using stale data fetched several steps earlier.
These are operational failures, not reasoning failures, and they need operational tooling — timeouts, step limits, and circuit breakers — rather than a better prompt.
Structured tracing of every agent step, alerting on abnormal tool-call patterns or runaway loops, and a sampling process where a human periodically reviews real transcripts cover most of what's needed to run agents in production with confidence.
None of this is exotic. It's the same discipline applied to any other distributed system — the only difference is that the component making decisions is non-deterministic, which makes the tracing more important, not less.