Glossary
AI Observability
Also known as: LLM observability, AI observability platform
Definition
AI observability is the practice of capturing every LLM call, scoring it for quality, and retaining the record for audit. It extends traditional application observability — logs, metrics, traces — to the model layer, including inputs, outputs, tool calls, agent reasoning steps, and guardrail decisions.
Why it matters
AI applications fail in ways traditional applications do not. They drift in quality silently, hallucinate facts, leak PII through prompts, and make decisions whose chain of reasoning is opaque to the people accountable for them. Without observability, debugging takes days, regulators cannot be answered, and silent regressions go unnoticed for months.
For regulated industries the bar is higher: the EU AI Act Article 12 requires automatic event logging for high-risk AI, NIST AI RMF MEASURE-2.7 expects continuous logging and metrics, and SR 11-7 demands ongoing model monitoring. AI observability is how those obligations get met operationally — not as a compliance afterthought, but as a working surface engineers and compliance officers both use.
In practice
Prism is an AI observability platform built around regulated-industry needs. Every LLM call is captured with model, latency, tokens, cost, and quality score. Guardrails redact PII at ingestion before storage. LLM-as-Judge evaluations score every trace across five quality dimensions. Agent trajectories record each step — LLM call, tool call, reasoning, memory access — so any decision can be reconstructed.
Related
More glossary terms
Start tracing in 5 minutes
One SDK. Five minutes. Full audit trails, PII redaction, and guardrail enforcement, from day one.