Agent Observability
Every step the agent took, in one session view
Connect your AI and follow what happened across the session, model calls, retrieval, tools, state, timing, and final outcome.
- 01
- lookup_order #4821 · 142ms
- 02
- check_return_policy · 60ms
- 03
- process_refund → 200 OK · 210ms
Every span succeeded. The outcome still did not match the user goal.
What you can open
Four connected views of the same activity
Open the exact run that matters
Review the request, response, model, cost, timing, scores, findings, and available tool or retrieval details.
Traces
Read the full user journey
See how the conversation changed across turns and where context, intent, sentiment, or the final goal began to drift.
Sessions
Follow the agent step by step
Inspect decisions, tool calls, retries, handoffs, state changes, and outcomes when the integration supplies that information.
Agent runs
Know when quality changes
Review scores, metrics, alerts, and important changes without losing the trace behind the signal.
Scores and alerts
Production Intelligence
See the pattern, not just the event
Group what users ask, where they get stuck, which failures repeat, and what knowledge the AI is missing.
- Top need
- Order changes
- Repeated problem
- Wrong tool selected
- Knowledge gap
- Missing policy content
- Next priority
- Confirm intent before final actions
Illustrative view. Every figure in the product opens the sessions behind it.
The continuous loop
What continuous looks like
Every session. Not a sample.
48,210 sessions traced end to end in one month
Target 75 to 78. Jun 9: knowledge-base fix shipped. Jun 22: score crossed target
KB remediation wave shipped · Jun 978
Figures from a sample FinVault reliability report. Design-partner data, synthetic and illustrative of real deliverables.
12.7% to 8.4%. Two fixes. Same inflection point
Apr 6 → Jun 298.4%
PRISM weekly aggregates
You see not just what changed, but why
One trace, opened
Every span succeeded. The outcome still did not match.
This is what observability for AI has to do that observability for services does not. There is no failed span here, no error, no timeout, no elevated latency. The run is wrong anyway, and the only way to see it is to hold the goal next to the result.
Trace run_9f3a, 412ms, three spans, all green. The failure is in the fourth layer — the one a conventional tracer does not have.
Status codes cannot see this
Three spans, three successes, 412ms end to end. Every dashboard built on spans and status codes shows this session as healthy. It is the single most common shape of AI failure in production and it is invisible to the entire existing observability stack.
The fourth layer is the product
Session, spans and state are things a tracer can capture. Outcome — what the user actually wanted, next to what they actually got — is the layer that has to be added, and it is the only one that turns a trace into a finding.
One run becomes 446
The trace ends by naming the pattern it belongs to and how many other sessions share its signature. That is the step from “here is a bad session” to “here is a bug worth someone’s week”.
Guardrails
Safety that got tighter while the agent got more useful
Guardrail triggers fell 22% month over month while completion rate rose. Those two usually trade against each other — a stricter agent refuses more and completes less — so moving both the right way at once is the only evidence that the tuning was real rather than a threshold being loosened.
Every guardrail category fell while completion rate rose. Those normally trade against each other, so moving both the right way is the only proof the tuning was real and not a threshold quietly loosened.
- Policy-safe refusal311was 411
- Off-topic / KB-grounding168was 194
- Suitability check hold42was 61
- PII redaction43was 58
- Prompt injection blocked17was 22
Bar is June, the number beside it is May.
PRISM guardrail event log
Zero high-severity incidents for a third month, and one open problem: 146 refusals that should never have fired. Naming it is the point — a safety report with nothing outstanding is a safety report nobody checked.
- PII redactionApplied before the model call, never after
- Prompt injection100% block rate, zero bypass recorded
- KB groundingAnswers held to the retrieval corpus
- Policy refusal146 judged avoidable — tuning in backlog #4
- Suitability checksHeld on high-risk flows pending review
- High-severity incidentsZero, third consecutive month
What it feeds
Traces are the input. This is the output.
Observability that ends in a dashboard ends in nothing. 4,050 failed sessions narrow to 5 validated production changes, and every stage between them has a human gate before it.
4,050
Failures clustered
3,822
Root causes classified
14
Backlog items opened
9
Fixes proposed
7
Human approved
5
Shipped & validated
Nothing reaches production without explicit sign-off. Knowledge-base updates and guardrail rules change in-platform, prompt changes ship as tracked versions with diffs and rollback, and code changes ship as pull requests carrying the evidence set and the affected trace references in the description. PRISM never merges on its own.
Questions
Common questions
What is AI observability?
AI observability helps you understand how an AI app behaved across prompts, sessions, retrieval, state, tools, and outcomes. PRISM keeps those signals connected to the user journey.
Can I import historical conversations?
Yes, when the source and fields are supported. The depth of the first view depends on the information available in the import.
Do I need live tracing before I can use PRISM?
No. You can begin with a supported historical import or connect live activity.
What happens when credits reach zero?
New AI-powered analysis pauses. Existing traces, findings, recommendations, and reports stay readable. Raw trace capture may continue within plan limits.
Does PRISM store every field from my app?
The exact fields depend on the integration and configuration. The Security page and integration guide must explain what is collected and how it is handled.
Turn an important run into the next action
Connect your AI and follow what happened from request to result.