Every CFO we talk to is asking some version of the same question: we are spending real money on AI agents, so what did it buy?
Most engineering leaders can answer half of it. They know how many seats are active, how many tokens went through the gateway, and roughly what the bill was last month. That data usually arrives through OpenTelemetry, and it is genuinely useful.
But it does not answer the question. It tells you what the agents did. It does not tell you what work they touched, whether that work moved forward, or whether it shipped, got reworked or got reverted.
That gap is what agent mining closes. Agent mining applies process mining to agent work: it takes the events agents leave behind, connects them to the systems where work is tracked, and reconstructs how work actually flows, with agents and people side by side.
It is not one capability. It is a ladder. Here is how we think about the five levels, why each one matters, and what makes each one hard.
Level 1: Usage telemetry
The question: How much are we using, and what does it cost?
This is where everyone starts, and it is the right place to start. OpenTelemetry gives you a common transport and vocabulary across agents, frameworks and vendors. The GenAI semantic conventions describe model calls, tool calls and agent invocations the same way no matter who emits them. Coding agents and AI gateways export usage, tokens, cost and latency through it.
That is what budgets, chargeback and adoption tracking run on. If you do not have it, get it. We ingest it, and nobody should build a proprietary pipe to replace it.
Why it is not enough: vendor exports are built for usage and billing, which is exactly what they should be built for. Prompts and tool detail are off by default, and observability backends typically keep days of data, not months. The bigger gap is work context. Most exports say little or nothing about which repository or branch the agent was working in, so nothing ties the activity back to the work it served. That is correct for observability. It is not enough for mining.
Self-check: Can you see agent spend by team and by model, for every agent your engineers use?
Level 2: Session fidelity
The question: What did the agent actually do?
A usage report tells you somebody did something. A session tells you what happened: every turn, every tool call, what was proposed, what was accepted and what was thrown away. It shows subagents spinning up and handing work back. And it shows true cost. Cache tokens are 84% of real spend in our data, so any report built only on input and output tokens gets cost badly wrong.
This is where you start to see retries, loops, abandoned work and agents wandering outside the boundaries you set for them.
What makes it hard: every coding agent is different, and they all change almost weekly. Some agents' exports carry no repository or branch at all, and others send them only when extra settings are turned on. Getting that context means capturing it on the developer's machine, in a way the strictest security teams will approve. We scrub secrets and personal data on the machine before anything leaves it, and customers can drop whole fields, like prompts, agent responses or file diffs, if they choose. Capture is the entry fee, though. What it is worth depends on what you connect it to.
Self-check: For any agent session last week, could you replay what it did, turn by turn, including what it threw away?
Level 3: Work linkage
The question: What work did it touch?
Process mining needs a case: the change, the ticket, the release. A trace is not a case. One piece of work spans many sessions, several agents and a handful of people over days or weeks. A single event can touch several objects at once: a ticket, two pull requests and a build.
Standard telemetry carries trace and span ids. It does not carry ticket keys or pull request numbers. And the people in the process (reviewers, approvers, the on-call engineer) never emit a span at all. Their steps live in your work tracking, source control and CI/CD systems.
Until sessions are tied to the work they served, you cannot calculate the number that matters: cost per shipped change.
What makes it hard: real data is messy. Branches do not follow naming rules, pull requests reference the wrong ticket or none at all, and plenty of agent work, like admin and research, never had a ticket to begin with. Only part of agent work links on its own. The rest needs inference across systems, for example matching the code an agent wrote to the code that merged. In our data, 81% of sessions that edited code tie to a pull request. That stitching across systems is where most homegrown efforts stall.
Self-check: Pick a ticket that shipped last month. Can you list every agent session that worked on it and what it cost?
Level 4: Process and outcomes
The question: Did the work move forward, follow the process and ship?
Telemetry describes runtime operations. Leaders think in activities. Five tool calls might together mean "ran the regression suite and ignored a failure." The mapping from one to the other is a process ontology, and it is what turns a pile of events into a process you can measure.
This is also where outcomes come in. Merges, reverts, rework, incidents and approvals all arrive later, in other systems. Without them you can describe agent behavior, but you cannot judge it, and you cannot compare one way of working against another.
It is where Goodhart's law shows up. Throughput goes up, and rework goes up with it. The bottleneck moves from writing code to reviewing it, then to the build, then to the release.
What makes it hard: every organization's process is different and changes every quarter. A hand-built mapping is out of date before it is finished. The ontology has to build and maintain itself.
Self-check: Can you show rework rate and cycle time for agent-assisted work versus everything else, by team and by model?
Level 5: Closed loop
The question: What should agents do next?
The first four levels tell people how work went. Level five tells agents where work is right now.
When agents can read the process while they work, the data stops being a report and starts steering: routing work to the right model, flagging a skipped review before the merge, spotting two agents about to collide on the same change, and holding agent actions to policy in real time.
What makes it hard: an agent-first engineering organization writes events around the clock, at a scale far beyond what people generate. The graph has to stay current, and each agent needs the right slice of context, not all of it.
Self-check: Do your agents know what other agents and people are working on right now?
Where most teams are
Most enterprises we meet are at level one, with pockets of level two. That is not a criticism. Level one is the foundation, and the telemetry standards behind it are getting better every quarter.
The point is that each level answers a different question, and the CFO's question lives at level four.
Instrument with OpenTelemetry. Then ask what process those traces belong to, and what they bought.