OpenInference Anthropic Instrumentation

JavaScript auto-instrumentation library for the Anthropic SDK

This package implements OpenInference tracing for the following Anthropic SDK methods:

• messages.create (including streaming)
• beta.messages.create (including server-side fallback and streaming)
• messages.countTokens and beta.messages.countTokens
• beta.messages.toolRunner (a CHAIN span parenting the LLM span of each request and a TOOL span for each tool run)

messages.stream() and beta.messages.stream() are traced through create. Clients from @anthropic-ai/bedrock-sdk, @anthropic-ai/vertex-sdk and @anthropic-ai/foundry-sdk are traced too, with llm.provider set to aws, google and azure.

These traces are fully OpenTelemetry compatible and can be sent to an OpenTelemetry collector for viewing, such as Arize Phoenix or Arize AX.

Spans of the Anthropic SDK

@anthropic-ai/sdk 0.134.0 and later create an OpenTelemetry span of their own for each API call (anthropic.messages.create and so on), unless it is turned off with openTelemetry: false or ANTHROPIC_OPEN_TELEMETRY=false. For a call that this instrumentation traces, the SDK's span is replaced by the OpenInference LLM span: the SDK records no span, its request carries the LLM span's trace context, and HTTP spans are children of the LLM span. Calls that this instrumentation does not trace, such as models.list(), keep the SDK's span. A beta.messages.toolRunner() run also keeps the SDK's tool runner and tool spans, which are not model calls.

Installation

npm install @arizeai/openinference-instrumentation-anthropic

Quickstart

Install required packages:

npm install @arizeai/openinference-instrumentation-anthropic @arizeai/openinference-semantic-conventions @anthropic-ai/sdk @opentelemetry/sdk-node @opentelemetry/exporter-trace-otlp-proto @opentelemetry/sdk-trace-node @opentelemetry/semantic-conventions

Set up instrumentation in your application:

import { NodeSDK, resources } from "@opentelemetry/sdk-node";
import { AnthropicInstrumentation } from "@arizeai/openinference-instrumentation-anthropic";
import { OTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-proto";
import { SimpleSpanProcessor } from "@opentelemetry/sdk-trace-node";
import { ATTR_SERVICE_NAME } from "@opentelemetry/semantic-conventions";
import { SEMRESATTRS_PROJECT_NAME } from "@arizeai/openinference-semantic-conventions";

import Anthropic from "@anthropic-ai/sdk";

// Configure the SDK with Anthropic instrumentation
const instrumentation = new AnthropicInstrumentation({
  // Optional: configure trace settings
  traceConfig: {
    // hideInputs: true,
    // hideOutputs: true,
  },
});
// necessary when instrumenting in an ESM environment
instrumentation.manuallyInstrument(Anthropic);
const sdk = new NodeSDK({
  spanProcessors: [
    new SimpleSpanProcessor(
      new OTLPTraceExporter({
        url: "http://localhost:6006/v1/traces",
      }),
    ),
  ],
  resource: resources.resourceFromAttributes({
    [ATTR_SERVICE_NAME]: "anthropic-service",
    [SEMRESATTRS_PROJECT_NAME]: "anthropic-service",
  }),
  instrumentations: [instrumentation],
});

// Initialize the SDK
sdk.start();

// Now use Anthropic as normal

const anthropic = new Anthropic({
  apiKey: process.env.ANTHROPIC_API_KEY,
});

const message = await anthropic.messages.create({
  model: "claude-3-5-haiku-latest",
  max_tokens: 1000,
  messages: [{ role: "user", content: "Hello, Claude!" }],
});

console.log(message.content);

sdk.shutdown();

Configuration

The AnthropicInstrumentation constructor accepts the following options:

• instrumentationConfig: Standard OpenTelemetry instrumentation configuration
• traceConfig: OpenInference-specific trace configuration for masking/redacting sensitive information
• tracerProvider: Optional custom tracer provider

Supported Features

• Messages API: Full support for anthropic.messages.create()
• Beta Messages API: Full support for anthropic.beta.messages.create()
• Server-Side Fallback: Captures requested and serving models across fallback boundaries, records the fallback handoff (declining model and refusal category) as message content, and takes token counts from the attempt that served the response
• Refusals: Records llm.finish_reason, so a classifier decline (stop_reason: "refusal") is visible on the span
• Streaming: Automatic handling of streaming responses
• Tool Use: Captures tool/function calling information
• Token Usage: Records input/output token counts when available
• Error Handling: Proper error recording and span status management

Semantic Conventions

This instrumentation follows the OpenInference semantic conventions for LLM observability, capturing:

• Model name and provider information
• Input/output messages and content
• Token usage statistics
• Tool calling details
• Invocation parameters

Examples

To run an example, run the following commands:

cd js/packages/openinference-instrumentation-anthropic
pnpm install
pnpm -r build
pnpx tsx examples/basic-usage.ts # or streaming.ts, tool-use.ts, etc

See the examples directory for more detailed usage examples.

To exercise server-side fallback:

export ANTHROPIC_API_KEY=your-api-key

# Real Anthropic beta request that triggers a reasoning-extraction fallback
pnpm exec tsx examples/server-side-fallback.ts

# Deterministic SSE fixture that guarantees a mid-stream fallback boundary
pnpm exec tsx examples/server-side-fallback-streaming.ts

License

Apache-2.0