AI agents
The tool call returned 200. The task still failed.
Follow the goal, state, tools, retries, and final result across the full run.
- Goal
- Exchange the order
- State
- intent = refund
- Tool call
- process_refund({order: 4821})
- Retries
- None
- Final result
- Refund started
What you can do
Three moves on every agent run
Test the whole run, not the final answer
Check whether the agent chose the right path, used the right tool, passed the right values, and reached the intended goal.
Find the step that changed the result
Loops, retries, wrong tool choices, stale state, and failed handoffs — grouped into the patterns that repeat across users, tasks, and versions.
Keep the failure as a test
Turn an important run into a scenario and check it before every release.
Inside a run
One agent run, opened up
The goal enters the run, every step and tool call is on the record, the mismatched step is isolated, and the failure leaves as a test that runs before every release.
Goal
exchange the order
Agent run
steps · tools · state
Steps that held
tools returned 200
Step 4 · mismatch
intent = refund
Pattern grouped
17 runs share it
Kept as a test
runs before release