AI SDK
Observability Integrations

Patronus

Monitor, evaluate and debug your AI SDK application with Patronus

Patronus AI provides an end-to-end system to evaluate, monitor and improve performance of an LLM system, enabling developers to ship AI products safely and confidently. Learn more here.

When you build with the AI SDK, you can stream OpenTelemetry (OTEL) traces straight into Patronus and pair every generation with rich automatic evaluations.

Setup

OpenTelemetry

Patronus exposes a fully‑managed OTEL endpoint. Configure an OTLP exporter to point at it, pass your API key, and you’re done—Patronus will automatically convert LLM spans into prompt/response records you can explore and evaluate.

.env.local
OTEL_EXPORTER_OTLP_ENDPOINT=https://otel.patronus.ai/v1/traces
OTEL_EXPORTER_OTLP_HEADERS="x-api-key=<PATRONUS_API_KEY>"

With @vercel/otel

instrumentation.ts
import { registerOTel } from '@vercel/otel';
import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-http';
import { BatchSpanProcessor } from '@opentelemetry/sdk-trace-node';
import { registerTelemetry } from 'ai';
import { LegacyOpenTelemetry } from '@ai-sdk/otel';

export function register() {
  registerOTel({
    serviceName: 'next-app',
    additionalSpanProcessors: [
      new BatchSpanProcessor(
        new OTLPTraceExporter({
          url: process.env.OTEL_EXPORTER_OTLP_ENDPOINT,
          headers: {
            'x-api-key': process.env.PATRONUS_API_KEY!,
          },
        }),
      ),
    ],
  });
}

registerTelemetry(new LegacyOpenTelemetry());

If you need gRPC instead of HTTP, swap the exporter for @opentelemetry/exporter-trace-otlp-grpc and use https://otel.patronus.ai:4317.

Register the integration and enable telemetry

Once you've installed @ai-sdk/otel and added registerTelemetry to your instrumentation.ts, telemetry is captured automatically. Pass telemetry to attach metadata:

import { generateText } from 'ai';
import { openai } from '@ai-sdk/openai';

const result = await generateText({
  model: openai('gpt-6-astra'),
  prompt: 'Write a haiku about spring.',
  telemetry: {
    functionId: 'spring-haiku', // span name
    metadata: {
      userId: 'user-123', // custom attrs surface in Patronus UI
    },
  },
});

Every attribute inside metadata becomes an OTEL attribute and is indexed by Patronus for filtering.

Example — tracing and automated evaluation

app/api/chat/route.ts
import { trace } from '@opentelemetry/api';
import { generateText } from 'ai';
import { openai } from '@ai-sdk/openai';

export async function POST(req: Request) {
  const body = await req.json();
  const tracer = trace.getTracer('next-app');

  return await tracer.startActiveSpan('chat-evaluate', async span => {
    try {
      /* 1️⃣ generate answer */
      const answer = await generateText({
        model: openai('gpt-6-astra'),
        prompt: body.prompt,
        telemetry: { functionId: 'chat' },
      });

      /* 2️⃣ run Patronus evaluation inside the same trace */
      await fetch('https://api.patronus.ai/v1/evaluate', {
        method: 'POST',
        headers: {
          'X-API-Key': process.env.PATRONUS_API_KEY!,
          'Content-Type': 'application/json',
        },
        body: JSON.stringify({
          evaluators: [
            { evaluator: 'lynx', criteria: 'patronus:hallucination' },
          ],
          evaluated_model_input: body.prompt,
          evaluated_model_output: answer.text,
          trace_id: span.spanContext().traceId,
          span_id: span.spanContext().spanId,
        }),
      });

      return new Response(answer.text);
    } finally {
      span.end();
    }
  });
}

Result: a single trace containing the root HTTP request, the LLM generation span, and your evaluation span—all visible in Patronus with the hallucination score attached.

Once you've traced

  • If you're tracing an agent, Patronus's AI assistant Percival will assist with error analysis and prompt optimization. Learn more here
  • Get set up on production monitoring and alerting by viewing logs and traces on Patronus and configuring webhooks for alerting. Learn more here

Resources

PreviousMLflowNextPostHog