Measuring an agent workforce that is already doing the work
AI agent optimization treats agents as economic actors to be measured rather than software to be monitored: cost per completed outcome, decision quality against a reviewable trail, escalation rate, and where a gate is catching problems a model could not. Uptime tells you nothing useful about a fleet.
An agent fails plausibly
A service fails loudly: it returns an error, the alert fires, someone looks. An agent fails plausibly — it produces confident, well-formatted output that is wrong, and nothing in conventional monitoring notices. That is the entire reason agent observability is a different discipline rather than a dashboard variant.
What you watch for is disagreement between an agent’s confidence and its acceptance rate, tasks completed suspiciously fast, and outputs that stop being rejected because reviewers stopped reading them.
The metrics that transfer, and the ones that do not
Conventional observability answers whether the system is up. For an agent workforce that is necessary and close to worthless — the interesting failures happen while everything is nominally healthy. An agent can be running perfectly and producing work nobody should ship.
The metrics that matter look like labour metrics: cost per completed outcome rather than per token, first-pass acceptance rate, how often a human gate rejects, and how long a task waits before someone approves it. That last one is usually where the real bottleneck turns out to be, and it is a process problem rather than a model problem.
Decision audit as the substrate
You cannot optimise what you cannot reconstruct. If the record shows only that an agent acted, every review becomes an argument. If it shows what the agent saw, what it chose, and which policy allowed it, review becomes an ordinary management activity.
This is the practical reason the audit trail is a native property of the mesh rather than a logging add-on. The mesh once found a payout bug in its own operation and fixed it — that is only possible when the trail is complete enough to be read backwards.
Where the gains actually come from
In practice most improvement comes from three unglamorous places: narrowing what an agent is allowed to see so its context stays relevant, moving an approval gate to where it catches problems earliest rather than latest, and routing cheap tasks to cheap models. Model upgrades matter, but they are rarely the largest available gain in a fleet that has never been measured.