Prism Documentation

Overview

Prism is the AI agent observability platform built for regulated industries and any organization where AI decisions carry real consequences.

AI agents in production make thousands of decisions daily. Without observability, you have no visibility into what your agents are saying, whether they are following instructions, or if they are exposing sensitive data. In regulated industries, this creates compliance risk, customer harm, and audit exposure.

Supported models and integrations

• Large Language Models: OpenAI (GPT-4o, GPT-4), Anthropic (Claude), Google (Gemini), Mistral, Llama, and any custom or fine-tuned model.
• Cloud Platforms: Azure AI Foundry, AWS Bedrock, Snowflake Cortex, Databricks.
• Agent Frameworks: LangChain, LlamaIndex, CrewAI, custom Python/Node.js agents, or any system that makes LLM API calls.

Prerequisites

• A Prism account (you received an invite link via email)
• An AI agent that makes LLM calls (any language, any framework)
• Your LLM provider API key (OpenAI, Anthropic, Azure, etc.)
• Python 3.8+ or Node.js 16+ installed
• A .env file workflow for managing secrets (explained in this guide)

Four steps to full observability

1. 1Log in and explore your project dashboard
2. 2Connect your first AI agent (send your first trace)
3. 3Configure guardrails and alert rules
4. 4Monitor, analyze, and iterate

Step 1: Log In and Explore

Click the invite link in your email. This takes you to the signup page where you set your password. After login, you land on your project's Overview page.

Dashboard Areas

Recommended First Steps

1. 1View Overview to see your agent activity summary, top-level metrics, and recent traces at a glance.
2. 2Go to Settings > API Keys and copy your API key and Project ID. You will need these in Step 2.
3. 3Open Traces. If sample data is loaded, click any trace to see the full conversation, AI analysis, quality scores, and trajectory.

Team Management

Prism supports multiple projects under one organization. Use the project dropdown in the top-left sidebar to switch between projects. To invite team members, go to Settings > Team. You can invite users at the organization level (access to all projects) or project level (access to a specific project only).

Step 2: Connect Your First AI Agent

Get Your Credentials

API Reference

Send one trace after each AI agent response. Prism scores and analyzes every trace automatically.

{
  "project_id": "YOUR_PROJECT_ID",
  "api_key": "YOUR_API_KEY",
  "input_messages": [
    { "role": "user", "content": "What is my loan status?" }
  ],
  "output_message": "Your loan application #4521 is approved.",
  "model": "gpt-4o",
  "latency_ms": 1200,
  "token_count_input": 45,
  "token_count_output": 180,
  "metadata": {
    "agent_name": "Customer Service Agent",
    "agent_id": "cs-agent-001",
    "source": "production"
  }
}

Python Integration

1. Create a .env file

# .env -- NEVER commit this file
PRISMTRACE_API_KEY=pt_your-key-here
PRISMTRACE_PROJECT_ID=your-project-uuid-here
ANTHROPIC_API_KEY=sk-ant-your-key-here

2. Add .env to .gitignore

echo '.env' >> .gitignore

3. Install dependencies

pip install requests python-dotenv

4. Send traces with error handling

import os
import requests
from dotenv import load_dotenv

load_dotenv()

PRISMTRACE_URL = "https://prismtrace.blockconvey.com"
PROJECT_ID     = os.getenv("PRISMTRACE_PROJECT_ID")
API_KEY        = os.getenv("PRISMTRACE_API_KEY")

def send_trace(user_input, agent_output, model, latency_ms,
               tokens_in=0, tokens_out=0, agent_id="my-agent"):
    try:
        response = requests.post(
            f"{PRISMTRACE_URL}/api/traces",
            json={
                "project_id":         PROJECT_ID,
                "api_key":            API_KEY,
                "input_messages":     [{"role": "user", "content": user_input}],
                "output_message":     agent_output,
                "model":              model,
                "latency_ms":         latency_ms,
                "token_count_input":  tokens_in,
                "token_count_output": tokens_out,
                "metadata": {
                    "agent_name": "My Agent",
                    "agent_id":   agent_id,
                    "source":     "production",
                },
            },
            timeout=10,
        )
        if response.status_code != 200:
            print(f"Prism error: {response.status_code}")
    except requests.exceptions.Timeout:
        print("Prism: request timed out")
    except requests.exceptions.ConnectionError:
        print("Prism: connection failed")
    except Exception as e:
        print(f"Prism: unexpected error: {e}")

Node.js Integration

require("dotenv").config(); // npm install dotenv
const axios = require("axios"); // npm install axios

const PRISMTRACE_URL = "https://prismtrace.blockconvey.com";
const PROJECT_ID     = process.env.PRISMTRACE_PROJECT_ID;
const API_KEY        = process.env.PRISMTRACE_API_KEY;

async function sendTrace(userInput, agentOutput, model, latencyMs) {
  try {
    const response = await axios.post(
      `${PRISMTRACE_URL}/api/traces`,
      {
        project_id:     PROJECT_ID,
        api_key:        API_KEY,
        input_messages: [{ role: "user", content: userInput }],
        output_message: agentOutput,
        model:          model,
        latency_ms:     latencyMs,
        metadata: { agent_name: "My Agent", agent_id: "my-agent-001", source: "production" },
      },
      { timeout: 10000 }
    );
    console.log(`Trace sent: ${response.data.id}`);
  } catch (error) {
    console.error("Prism error:", error.message);
  }
}

cURL Example

Test from any terminal. HTTP 200 with a trace ID confirms your integration is working.

curl -X POST https://prismtrace.blockconvey.com/api/traces \
  -H "Content-Type: application/json" \
  -d '{
    "project_id":     "'$PRISMTRACE_PROJECT_ID'",
    "api_key":        "'$PRISMTRACE_API_KEY'",
    "input_messages": [{"role": "user", "content": "Test"}],
    "output_message": "This is a test trace.",
    "model":          "gpt-4o",
    "latency_ms":     500
  }'

Step 3: Configure Guardrails and Alerts

Guardrails are your first line of defense against AI agent failures. Navigate to Guardrails in the sidebar to configure rules. Rules take effect immediately on save with no deployment step required.

Guardrail Types

Recommended starting configuration

1. 1PII Detection > Flag: Flags traces containing personal data. Switch to Block in production once detection is confirmed working.
2. 2Prompt Injection > Block: Blocks all injection attempts. There is no legitimate reason for a user to override agent instructions.

Alert Rules

Navigate to Alerts and click + Create Alert Rule. Alerts watch for patterns and trends across your trace data, while guardrails protect individual traces.

Connectors and Model Inventory

Model Inventory

Navigate to Model Inventory > + Add Model to register each agent. This lets you filter traces by agent, compare per-agent performance and cost, and track which agents are active or degrading. Each registered agent card shows live trace counts, average latency, last activity, and provider information.

Prompt Library and Versioning

Navigate to Prompt Library in the sidebar to store your agent system prompts with full version history.

1. 1Create a prompt. Click + New Prompt, give it a name, paste your system prompt, and save. This becomes version 1.
2. 2Edit and version. Click the prompt, select Edit, make changes, write a change summary, and save as a new version. The previous version is preserved.
3. 3Compare versions. Click Compare on the version history page to see a side-by-side diff with red (removed) and green (added) highlighting.
4. 4Roll back. Click Activate on any older version to revert. Changes take effect immediately.

Step 4: Monitor, Analyze, and Iterate

Traces

Every agent interaction appears on the Traces page with a quality score, latency, cost, and model. Click any trace to see the full conversation, AI-generated analysis, trajectory steps, and guardrail flags. The Conversations view groups related traces into sessions, showing the full user journey across multiple interactions.

Scores and What They Mean

The Improvement Loop

1. 1Detect: Alerts and scores flag a quality drop or anomaly.
2. 2Investigate: Click through to the specific traces, review the conversation, and identify the root cause.
3. 3Fix: Update the prompt (using Prompt Library versioning), add a guardrail rule, or adjust model parameters.
4. 4Verify: Monitor scores after the change. Compare the new prompt version's performance against the previous one.
5. 5Repeat: Set tighter thresholds as your agents improve. Continuous improvement compounds.

Export trace data as CSV for compliance reporting via Data Export in the sidebar. For programmatic access, contact arun@blockconvey.com.

Security Best Practices

Prism is built for regulated industries where data security is non-negotiable.

• Store keys in environment variables: Use a .env file locally and your platform's secret manager in production. Never hardcode keys.
• Add .env to .gitignore immediately: Do this before your first commit. A single accidental push exposes your keys to anyone with repo access.
• Rotate keys periodically: Generate a new API key in Settings > API Keys, update your environment variables, then revoke the old key.
• Use separate keys per environment: Create distinct Prism projects for development, staging, and production. Each has its own API key.
• API keys are shown once: When you generate a new key, Prism displays it once. Copy it to your secret manager immediately.

Quick Reference