Overview

@arizeai/ax-client is a TypeScript client for the Arize AX REST API. It is still under active development and is subject to change.

Arize is an AI engineering platform. It helps engineers develop, evaluate, and observe AI applications and agents.

Arize has both Enterprise and OSS products to support this goal:

• Arize AX — an enterprise AI engineering platform from development to production, with an embedded AI Copilot
• Phoenix — a lightweight, open-source project for tracing, prompt engineering, and evaluation
• OpenInference — an open-source instrumentation package to trace LLM applications across models and frameworks

Key Features

• Tracing - Trace your LLM application's runtime using OpenTelemetry-based instrumentation.
• Evaluation - Leverage LLMs to benchmark your application's performance using response and retrieval evals.
• Datasets - Create versioned datasets of examples for experimentation, evaluation, and fine-tuning.
• Experiments - Track and evaluate changes to prompts, LLMs, and retrieval.
• Playground- Optimize prompts, compare models, adjust parameters, and replay traced LLM calls.
• Prompt Management- Manage and test prompt changes systematically using version control, tagging, and experimentation.

Table of Contents

• Overview
• Key Features
• Installation
• Configuration
• Datasets
  • Listing datasets
  • Creating a dataset
  • Getting a dataset
  • Updating a dataset
  • Deleting a dataset
  • Listing dataset examples
  • Appending dataset examples
  • Updating dataset examples
  • Annotating dataset examples
  • Deleting dataset examples
• Experiments
  • Listing experiments
  • Creating an experiment
  • Getting an experiment
  • Deleting an experiment
  • Listing experiment runs
  • Appending experiment runs
  • Annotating experiment runs
• Prompts
  • Creating a prompt
  • Getting a prompt
  • Listing prompts
  • Updating a prompt
  • Deleting a prompt
  • Prompt versions
  • Prompt labels
• Evaluators
  • Listing evaluators
  • Creating a template evaluator
  • Creating a code evaluator
  • Creating a remote evaluator
  • Getting an evaluator
  • Updating an evaluator
  • Deleting an evaluator
  • Evaluator versions
  • Deleting evaluator versions
• Tasks
  • Listing tasks
  • Creating an evaluation task
  • Creating a run-experiment task
  • Getting a task
  • Updating and deleting a task
  • Triggering and monitoring task runs
• Spans
  • Listing spans
    • Column projection
  • Annotating spans
  • Deleting spans
• Traces
  • Listing traces
• Annotation Configs
  • Listing annotation configs
  • Creating an annotation config
  • Getting an annotation config
  • Updating an annotation config
  • Deleting an annotation config
• Annotation Queues
  • Listing annotation queues
  • Creating an annotation queue
  • Getting, updating, and deleting a queue
  • Managing queue records
• AI Integrations
  • Listing AI integrations
  • Creating an AI integration
  • Getting an AI integration
  • Updating an AI integration
  • Deleting an AI integration
• Integrations
  • Listing integrations
  • Creating an integration
  • Getting an integration
  • Updating an integration
  • Deleting an integration
• Projects
  • Creating a project
  • Getting a project
  • Listing projects
  • Updating a project
  • Deleting a project
• API Keys
  • Creating an API key
  • Listing API keys
  • Revoking an API key
  • Refreshing an API key
• Roles
  • Creating a role
  • Listing roles
  • Getting a role
  • Updating a role
  • Deleting a role
• Role bindings
  • Creating a role binding
  • Listing role bindings
  • Getting a role binding
  • Updating a role binding
  • Deleting a role binding
• Resource restrictions
• Organizations
  • Listing organizations
  • Getting an organization
  • Creating an organization
  • Updating an organization
  • Deleting an organization
• Organization memberships
  • Adding a user to an organization
  • Removing a user from an organization
• Users
  • Listing users
  • Getting a user
  • Creating a user
  • Updating a user
  • Deleting a user
  • Resending a user invitation
  • Resetting a user's password
  • Bulk deleting users
• Spaces
  • Listing spaces
  • Getting a space
  • Creating a space
  • Updating a space
  • Deleting a space
• Space memberships
  • Adding a user to a space
  • Removing a user from a space
• Webhooks
  • Listing webhooks
  • Creating a webhook
  • Getting, updating, testing, and deleting a webhook
  • Listing delivery attempts
• Webhook subscriptions
  • Subscribing a webhook to an event
  • Listing, getting, and deleting subscriptions
• REST endpoints

Installation

# or yarn, pnpm, bun, etc.
npm install @arizeai/ax-client

Configuration

The client will automatically read environment variables from your environment, if available.

The following environment variables are used:

• ARIZE_API_KEY - The API key to use for authentication.
• ARIZE_BASE_URL - The base URL of the Arize AX API.

Alternatively, you can pass configuration options to the client directly, and they will be prioritized over environment variables and default values.

Datasets

The @arizeai/ax-client package allows you to create and manage datasets and their examples.

Listing datasets

import { listDatasets } from "@arizeai/ax-client";

const datasets = await listDatasets({ space: "my-space" });
console.log(datasets);

Creating a dataset

Create a dataset by providing a space (name or ID), name, and array of examples (each containing at least one property).

import { createDataset } from "@arizeai/ax-client";

const dataset = await createDataset({
  name: "my-dataset",
  space: "my-space",
  examples: [{ question: "What is 2+2?", answer: "4", topic: "math" }],
});

Getting a dataset

import { getDataset } from "@arizeai/ax-client";

const dataset = await getDataset({ dataset: "my-dataset", space: "my-space" });
console.log(dataset);

Updating a dataset

import { updateDataset } from "@arizeai/ax-client";

const dataset = await updateDataset({
  dataset: "my-dataset",
  space: "my-space",
  name: "my-renamed-dataset",
});

Deleting a dataset

import { deleteDataset } from "@arizeai/ax-client";

await deleteDataset({ dataset: "my-dataset", space: "my-space" });

Listing dataset examples

Lists the examples of a dataset. filter is an optional SQL-like expression for more complex queries.

import { listDatasetExamples } from "@arizeai/ax-client";

const examples = await listDatasetExamples({
  dataset: "my-dataset",
  space: "my-space",
});
console.log(examples);

const filtered = await listDatasetExamples({
  dataset: "my-dataset",
  space: "my-space",
  filter: "question = 'What is 2+2?'",
});
console.log(filtered);

Appending dataset examples

import { appendExamples } from "@arizeai/ax-client";

const result = await appendExamples({
  dataset: "my-dataset",
  space: "my-space",
  examples: [{ question: "What is 2+2?", answer: "4", topic: "math" }],
});
console.log(result.exampleIds); // IDs of the inserted examples

Updating dataset examples

Updating examples creates a new version of the dataset.

import { updateExamples } from "@arizeai/ax-client";

const dataset = await updateExamples({
  dataset: "my-dataset",
  space: "my-space",
  examples: [{ id: "your_example_id", question: "What is 2+2?", answer: "4" }],
  newVersionName: "your_new_version_name",
});

Annotating dataset examples

import { annotateDatasetExamples } from "@arizeai/ax-client";

await annotateDatasetExamples({
  space: "my-space",
  dataset: "my-dataset",
  annotations: [
    {
      recordId: "example_id_abc123",
      values: [
        { name: "quality", score: 0.9 },
        { name: "topic", label: "science" },
      ],
    },
  ],
});

Deleting dataset examples

Examples are removed in place from the given version; no new version is created. The delete is partial-tolerant and idempotent — the result reports which IDs were deleted and which were not.

import { deleteDatasetExamples } from "@arizeai/ax-client";

const result = await deleteDatasetExamples({
  dataset: "my-dataset",
  space: "my-space",
  datasetVersionId: "your_dataset_version_id",
  examples: ["example_id_1", "example_id_2"],
});
console.log(
  result.completed,
  result.deletedExampleIds,
  result.notDeletedExampleIds,
);

Experiments

The @arizeai/ax-client package allows you to create and manage experiments and their runs.

Listing experiments

import { listExperiments } from "@arizeai/ax-client";

const experiments = await listExperiments({
  dataset: "my-dataset",
  space: "my-space",
});
console.log(experiments);

Creating an experiment

import { createExperiment } from "@arizeai/ax-client";

const experiment = await createExperiment({
  experimentName: "my-experiment",
  dataset: "my-dataset",
  space: "my-space",
  experimentRuns: [],
});
console.log(experiment);

Getting an experiment

import { getExperiment } from "@arizeai/ax-client";

// Using names
const experiment = await getExperiment({
  experiment: "my-experiment",
  dataset: "my-dataset",
  space: "my-space",
});

// Using an ID directly
const byId = await getExperiment({ experiment: "your_experiment_id" });

Deleting an experiment

import { deleteExperiment } from "@arizeai/ax-client";

await deleteExperiment({
  experiment: "my-experiment",
  dataset: "my-dataset",
  space: "my-space",
});

Listing experiment runs

List experiment runs by providing an experiment name or ID (space and dataset context are required when using names), optionally narrowed by an SQL-like filter over run columns, evaluations (eval.<name>.score), and annotations (annotation.<name>.*). Runs are returned in stable ID ascending order — pass pagination.nextCursor back as cursor to fetch the next page, keeping the filter unchanged while paging.

import { listExperimentRuns } from "@arizeai/ax-client";

const experimentRuns = await listExperimentRuns({
  experiment: "my-experiment",
  dataset: "my-dataset",
  space: "my-space",
});

const filtered = await listExperimentRuns({
  experiment: "my-experiment",
  dataset: "my-dataset",
  space: "my-space",
  filter: "eval.quality.score < 0.5",
  limit: 50,
});
console.log(filtered.data);

Appending experiment runs

Append between 1 and 1000 new runs to an existing experiment. Each run must include exampleId (the ID of an example from the experiment's dataset) and output. The response includes the updated experiment and the generated run IDs in input order.

import { appendExperimentRuns } from "@arizeai/ax-client";

const result = await appendExperimentRuns({
  space: "my-space",
  dataset: "my-dataset",
  experiment: "my-experiment",
  experimentRuns: [{ exampleId: "ex_abc123", output: "The answer is 42" }],
});
console.log(result.runIds); // IDs of the appended runs

Annotating experiment runs

import { annotateExperimentRuns } from "@arizeai/ax-client";

await annotateExperimentRuns({
  space: "my-space",
  dataset: "my-dataset",
  experiment: "my-experiment",
  annotations: [
    {
      recordId: "run_id_abc123",
      values: [
        { name: "accuracy", label: "correct", score: 1.0 },
        { name: "notes", text: "Well-structured output" },
      ],
    },
  ],
});

Prompts

The @arizeai/ax-client package allows you to create and manage prompts, their versions, and labels.

Creating a prompt

A prompt is created together with its initial version.

import { createPrompt } from "@arizeai/ax-client";

const prompt = await createPrompt({
  space: "my-space",
  name: "customer-support",
  description: "A prompt for customer support interactions",
  version: {
    commitMessage: "Initial version",
    inputVariableFormat: "F_STRING",
    provider: "OPEN_AI",
    model: "gpt-4",
    messages: [
      { role: "SYSTEM", content: "You are a helpful assistant." },
      { role: "USER", content: "Hello, {name}!" },
    ],
  },
});

Getting a prompt

By default the latest version is returned. Pass versionId or label to resolve a specific version instead.

import { getPrompt } from "@arizeai/ax-client";

// Latest version
const prompt = await getPrompt({
  prompt: "customer-support",
  space: "my-space",
});

// Version tagged with the "production" label
const productionPrompt = await getPrompt({
  prompt: "customer-support",
  space: "my-space",
  label: "production",
});

Listing prompts

import { listPrompts } from "@arizeai/ax-client";

const { data, pagination } = await listPrompts({ space: "my-space" });
console.log(data.map((p) => p.name));

Updating a prompt

import { updatePrompt } from "@arizeai/ax-client";

const updated = await updatePrompt({
  prompt: "customer-support",
  space: "my-space",
  description: "Updated description for the prompt",
});

Deleting a prompt

import { deletePrompt } from "@arizeai/ax-client";

await deletePrompt({ prompt: "customer-support", space: "my-space" });

Prompt versions

You can list all versions of a prompt, retrieve a specific version by its ID, and create new versions.

import {
  listPromptVersions,
  getPromptVersion,
  createPromptVersion,
} from "@arizeai/ax-client";

// List versions
const { data: versions } = await listPromptVersions({
  prompt: "customer-support",
  space: "my-space",
});

// Get a specific version by its ID (pure ID, no name resolution)
const version = await getPromptVersion({ versionId: "your-version-id" });

// Create a new version
const newVersion = await createPromptVersion({
  prompt: "customer-support",
  space: "my-space",
  commitMessage: "Updated system prompt",
  inputVariableFormat: "F_STRING",
  provider: "OPEN_AI",
  model: "gpt-4",
  messages: [
    { role: "SYSTEM", content: "You are a helpful assistant." },
    { role: "USER", content: "Hello, {name}!" },
  ],
});

Prompt labels

Labels let you tag a specific prompt version with a named alias (e.g. "production", "staging").

import {
  getPromptVersionByLabel,
  setPromptVersionLabels,
  deletePromptVersionLabel,
} from "@arizeai/ax-client";

// Resolve the version a label currently points to
const version = await getPromptVersionByLabel({
  prompt: "customer-support",
  space: "my-space",
  labelName: "production",
});

// Set labels on a version (replaces all existing labels)
const { labels } = await setPromptVersionLabels({
  versionId: "your-version-id",
  labels: ["production", "staging"],
});

// Remove a single label from a version
await deletePromptVersionLabel({
  versionId: "your-version-id",
  labelName: "staging",
});

Evaluators

The @arizeai/ax-client package allows you to create and manage LLM-as-a-judge (template), code, and remote evaluators, along with their versions.

Listing evaluators

import { listEvaluators } from "@arizeai/ax-client";

const evaluators = await listEvaluators({ space: "my-space" });
console.log(evaluators);

Creating a template evaluator

import { createTemplateEvaluator } from "@arizeai/ax-client";

const evaluator = await createTemplateEvaluator({
  name: "Relevance",
  space: "my-space",
  commitMessage: "Initial version",
  templateConfig: {
    name: "Relevance",
    template:
      "Is the response relevant?\nQuery: {{query}}\nResponse: {{response}}",
    includeExplanations: true,
    useFunctionCalling: true,
    classificationChoices: { relevant: 1, irrelevant: 0 },
    direction: "MAXIMIZE",
    llmConfig: {
      aiIntegrationId: "QUlJbnRlZ3JhdGlvbjphYmMxMjM=",
      modelName: "gpt-4o",
      invocationParameters: { temperature: 0 },
      providerParameters: {},
    },
  },
});

Creating a code evaluator

import { createCodeEvaluator } from "@arizeai/ax-client";

const evaluator = await createCodeEvaluator({
  name: "JSON Parseable",
  space: "my-space",
  commitMessage: "Initial version",
  codeConfig: {
    type: "MANAGED",
    name: "json_parseable",
    managedEvaluator: "JSON_PARSEABLE",
    variables: ["output"],
  },
});

Creating a remote evaluator

A remote evaluator calls a customer-hosted HTTP endpoint via an EVALUATOR integration. Requires the enableRemoteEvalTasks feature flag on the account.

import { createRemoteEvaluator } from "@arizeai/ax-client";

const evaluator = await createRemoteEvaluator({
  name: "My Remote Eval",
  space: "my-space",
  commitMessage: "Initial version",
  integrationId: "<evaluator-integration-id>",
});

Getting an evaluator

import { getEvaluator } from "@arizeai/ax-client";

const evaluator = await getEvaluator({
  evaluator: "Relevance",
  space: "my-space",
});
console.log(evaluator);

Updating an evaluator

import { updateEvaluator } from "@arizeai/ax-client";

const evaluator = await updateEvaluator({
  evaluator: "Relevance",
  space: "my-space",
  name: "Updated Evaluator Name",
});

Deleting an evaluator

import { deleteEvaluator } from "@arizeai/ax-client";

await deleteEvaluator({ evaluator: "Relevance", space: "my-space" });

Evaluator versions

You can list all versions of an evaluator, retrieve a specific version by its ID, and create new versions.

import {
  listEvaluatorVersions,
  getEvaluatorVersion,
  createTemplateEvaluatorVersion,
} from "@arizeai/ax-client";

// List versions
const versions = await listEvaluatorVersions({
  evaluator: "Relevance",
  space: "my-space",
});

// Get a specific version by its ID
const version = await getEvaluatorVersion({ versionId: "your-version-id" });

// Create a new template version (use createCodeEvaluatorVersion for code evaluators,
// or createRemoteEvaluatorVersion for remote evaluators)
const newVersion = await createTemplateEvaluatorVersion({
  evaluator: "Relevance",
  space: "my-space",
  commitMessage: "Updated prompt template",
  templateConfig: {
    name: "Relevance",
    template: "Rate the relevance.\nQuery: {{query}}\nResponse: {{response}}",
    includeExplanations: true,
    useFunctionCalling: true,
    classificationChoices: { relevant: 1, irrelevant: 0 },
    direction: "MAXIMIZE",
    llmConfig: {
      aiIntegrationId: "QUlJbnRlZ3JhdGlvbjphYmMxMjM=",
      modelName: "gpt-4o",
      invocationParameters: { temperature: 0 },
      providerParameters: {},
    },
  },
});

// Create a new remote version (switches to a different EVALUATOR integration)
import { createRemoteEvaluatorVersion } from "@arizeai/ax-client";

const remoteVersion = await createRemoteEvaluatorVersion({
  evaluator: "My Remote Eval",
  space: "my-space",
  commitMessage: "Switch endpoint",
  integrationId: "<evaluator-integration-id>",
});

Deleting evaluator versions

import { deleteEvaluatorVersions } from "@arizeai/ax-client";

await deleteEvaluatorVersions({
  evaluator: "Relevance",
  space: "my-space",
  versionIds: ["your_version_id_1", "your_version_id_2"],
});

Tasks

The @arizeai/ax-client package allows you to create and manage scheduled or on-demand tasks (server-side evaluations and experiments) and monitor their runs.

Tip: Prefer the narrowly-typed createEvaluationTask and createRunExperimentTask helpers shown below. A lower-level createTask is also exported if you need to construct a request payload directly.

Listing tasks

import { listTasks } from "@arizeai/ax-client";

const tasks = await listTasks({ space: "my-space", name: "prod" });
console.log(tasks);

Creating an evaluation task

Each evaluator is either a span-shape entry (with columnMappings / queryFilter) or a trace/session-shape entry (with queryMappings). Supply one shape per evaluator; they are mutually exclusive via the SpanEvaluatorInput | TraceOrSessionEvaluatorInput union.

Span-granularity task:

import { createEvaluationTask } from "@arizeai/ax-client";

const task = await createEvaluationTask({
  name: "Weekly Quality Check",
  type: "TEMPLATE_EVALUATION",
  space: "my-space",
  project: "my-project",
  queryFilter: "span_kind = 'LLM'",
  evaluators: [
    {
      evaluatorId: "your_evaluator_id",
      columnMappings: {
        input: "attributes.input.value",
        output: "attributes.output.value",
      },
    },
  ],
});

Trace/session-granularity task (multi-span query):

const task = await createEvaluationTask({
  name: "Trace Quality Check",
  type: "TEMPLATE_EVALUATION",
  space: "my-space",
  project: "my-project",
  queryFilters: {
    filters: [
      { id: "A", filter: "span_kind = 'LLM'" },
      { id: "B", filter: "span_kind = 'RETRIEVER'" },
    ],
    expression: "A AND B",
  },
  evaluators: [
    {
      evaluatorId: "your_evaluator_id",
      queryMappings: [
        {
          variableName: "input",
          queryIds: ["A"],
          attributePath: "attributes.input.value",
        },
        {
          variableName: "output",
          queryIds: ["B"],
          attributePath: "attributes.output.value",
        },
      ],
    },
  ],
});

Creating a run-experiment task

import { createRunExperimentTask } from "@arizeai/ax-client";

const task = await createRunExperimentTask({
  name: "GPT-4o Baseline Task",
  dataset: "my-dataset",
  space: "my-space",
  runConfiguration: {
    experiment_type: "LLM_GENERATION",
    aiIntegration: "my-openai-integration",
    model_name: "gpt-4o",
    input_variable_format: "F_STRING",
    messages: [
      { role: "SYSTEM", content: "You are a helpful assistant." },
      { role: "USER", content: "Answer: {question}" },
    ],
  },
});

Getting a task

import { getTask } from "@arizeai/ax-client";

// By ID
const task = await getTask({ task: "your_task_id" });

// By name (requires space)
const byName = await getTask({ task: "My Task", space: "my-space" });

Updating and deleting a task

import { updateTask, deleteTask } from "@arizeai/ax-client";

// Basic update — rename and adjust sampling rate
await updateTask({
  task: "your_task_id",
  name: "Renamed Task",
  samplingRate: 0.5,
});

// Switch to trace/session shape (multi-span query)
await updateTask({
  task: "your_task_id",
  queryFilters: {
    filters: [{ id: "A", filter: "span_kind = 'LLM'" }],
    expression: "A",
  },
  evaluators: [
    {
      evaluatorId: "your_evaluator_id",
      queryMappings: [
        {
          variableName: "input",
          queryIds: ["A"],
          attributePath: "attributes.input.value",
        },
      ],
    },
  ],
});

await deleteTask({ task: "your_task_id" });

Triggering and monitoring task runs

import {
  triggerTaskRun,
  waitForTaskRun,
  listTaskRuns,
  getTaskRun,
  cancelTaskRun,
} from "@arizeai/ax-client";

// Trigger a run, then poll until it finishes
const run = await triggerTaskRun({ task: "My Task", space: "my-space" });
const finalRun = await waitForTaskRun({
  runId: run.id,
  pollInterval: 3_000, // poll every 3 seconds
  timeout: 5 * 60_000, // give up after 5 minutes
});
console.log(finalRun.status); // "COMPLETED" | "FAILED" | "CANCELLED"

// List, inspect, or cancel runs
const { data: runs } = await listTaskRuns({
  task: "My Task",
  space: "my-space",
});
const single = await getTaskRun({ runId: run.id });
await cancelTaskRun({ runId: run.id });

Spans

The @arizeai/ax-client package allows you to list, annotate, and delete spans within a project.

Listing spans

import { listSpans } from "@arizeai/ax-client";

// By project ID
const spans = await listSpans({ project: "your_project_id" });

// By project name (requires space)
const byName = await listSpans({ project: "My Project", space: "my-space" });

Column projection

Set includedColumns to return only selected columns. Set excludedColumns to omit selected columns, such as a large embedding vector. Do not set both. Fixed span fields are always returned. Set truncateStringLength to shorten long string values inside each span's attributes object. Strings longer than the limit are cut and end with …, which counts toward the limit. Span IDs, names, status, timestamps, annotations, evaluations, and events are not truncated.

const spans = await listSpans({
  project: "your_project_id",
  excludedColumns: ["attributes.embedding.vectors"],
  truncateStringLength: 200,
});

Annotating spans

granularity selects what each recordId identifies: a span (SPAN, the default), a trace's root span (TRACE), or a session (SESSION, written to the root span of the session's earliest trace). Up to 1000 records may be annotated per request for SPAN/TRACE; up to 100 for SESSION.

import { annotateSpans } from "@arizeai/ax-client";

await annotateSpans({
  space: "my-space",
  project: "my-project",
  annotations: [
    {
      recordId: "c3Bhbl9pZF9hYmMxMjM=", // base64-encoded span ID
      values: [
        { name: "quality", score: 0.9 },
        { name: "topic", label: "science" },
      ],
    },
  ],
});

startTime/endTime bound the lookup window and are both optional: when omitted, endTime defaults to now and startTime defaults to 31 days before that (7 days for SESSION). If the record is older than the default window, pass startTime explicitly or the record won't be found.

// Session-granularity example
await annotateSpans({
  space: "my-space",
  project: "my-project",
  granularity: "SESSION",
  startTime: new Date(Date.now() - 7 * 24 * 60 * 60 * 1000),
  endTime: new Date(),
  annotations: [
    {
      recordId: "20144", // session ID
      values: [{ name: "quality", label: "good" }],
    },
  ],
});

Deleting spans

import { deleteSpans } from "@arizeai/ax-client";

// By project name (requires space)
await deleteSpans({
  project: "My Project",
  space: "my-space",
  spanIds: ["a1b2c3d4e5f6a7b8", "f8e7d6c5b4a39281"],
});

Traces

The @arizeai/ax-client package allows you to list traces within a project. Each trace carries its full (flat) list of spans plus lightweight roll-up metadata; reconstruct the trace tree client-side using each span's parentId.

Listing traces

import { listTraces } from "@arizeai/ax-client";

// By project ID
const { data: traces, pagination } = await listTraces({
  project: "your_project_id",
});

// By project name (requires space), with a filter and time range
const filtered = await listTraces({
  project: "My Project",
  space: "my-space",
  filter: "status_code = 'ERROR'",
  startTime: new Date("2024-01-01T00:00:00Z"),
  endTime: new Date("2024-01-02T00:00:00Z"),
  limit: 50,
});

// Page through results using the opaque cursor
const next = await listTraces({
  project: "your_project_id",
  cursor: pagination.nextCursor,
});

The filter uses the same SQL-like syntax as listSpans, but the semantics are trace-contains-match: a trace is returned when any of its spans matches the filter (the matching span is usually a child, not the root).

Each response page also has an overall span limit, so a trace's span list can be incomplete even when spansTruncated is false. Narrow the time window or call listSpans with a trace_id filter to retrieve the trace's spans directly.

Annotation Configs

The @arizeai/ax-client package allows you to create and manage annotation configs, which define the annotations that can be applied to spans and examples.

Listing annotation configs

import { listAnnotationConfigs } from "@arizeai/ax-client";

const annotationConfigs = await listAnnotationConfigs({ space: "my-space" });
console.log(annotationConfigs);

Creating an annotation config

import { createAnnotationConfig } from "@arizeai/ax-client";

const annotationConfig = await createAnnotationConfig({
  name: "Accuracy",
  space: "my-space",
  type: "CATEGORICAL",
  values: [
    { label: "accurate", score: 1 },
    { label: "inaccurate", score: 0 },
  ],
  optimizationDirection: "MAXIMIZE",
});

Getting an annotation config

import { getAnnotationConfig } from "@arizeai/ax-client";

const annotationConfig = await getAnnotationConfig({
  annotationConfig: "Accuracy",
  space: "my-space",
});

Updating an annotation config

There is a dedicated update function per annotation config type. Each one must be called with the stored config's type — a config's type is immutable and cannot be changed. Any fields you omit are left unchanged.

import { updateCategoricalAnnotationConfig } from "@arizeai/ax-client";

const annotationConfig = await updateCategoricalAnnotationConfig({
  annotationConfig: "Accuracy",
  space: "my-space",
  name: "Accuracy v2",
  values: [
    { label: "accurate", score: 1 },
    { label: "inaccurate", score: 0 },
  ],
  optimizationDirection: "MAXIMIZE",
});

import { updateContinuousAnnotationConfig } from "@arizeai/ax-client";

const annotationConfig = await updateContinuousAnnotationConfig({
  annotationConfig: "Accuracy",
  space: "my-space",
  name: "Accuracy v2",
  minimumScore: 0,
  maximumScore: 10,
  optimizationDirection: "MAXIMIZE",
});

import { updateFreeformAnnotationConfig } from "@arizeai/ax-client";

const annotationConfig = await updateFreeformAnnotationConfig({
  annotationConfig: "Notes",
  space: "my-space",
  name: "Notes v2",
});

Deleting an annotation config

import { deleteAnnotationConfig } from "@arizeai/ax-client";

await deleteAnnotationConfig({
  annotationConfig: "Accuracy",
  space: "my-space",
});

Annotation Queues

The @arizeai/ax-client package allows you to create and manage annotation queues, add records to them, and annotate or assign those records.

Listing annotation queues

import { listAnnotationQueues } from "@arizeai/ax-client";

const annotationQueues = await listAnnotationQueues({ space: "my-space" });
console.log(annotationQueues);

Creating an annotation queue

import { createAnnotationQueue } from "@arizeai/ax-client";

const queue = await createAnnotationQueue({
  name: "Quality Review Queue",
  spaceId: "your_space_id",
  annotationConfigIds: ["ac_abc123"],
  assignmentMethod: "ALL",
});

Getting, updating, and deleting a queue

import {
  getAnnotationQueue,
  updateAnnotationQueue,
  deleteAnnotationQueue,
} from "@arizeai/ax-client";

const queue = await getAnnotationQueue({
  annotationQueue: "my_queue",
  space: "my-space",
});
await updateAnnotationQueue({
  annotationQueue: "my_queue",
  space: "my-space",
  name: "Updated Queue Name",
});
await deleteAnnotationQueue({ annotationQueue: "my_queue", space: "my-space" });

Managing queue records

import {
  addAnnotationQueueRecords,
  listAnnotationQueueRecords,
  deleteAnnotationQueueRecords,
  annotateAnnotationQueueRecord,
  assignAnnotationQueueRecord,
} from "@arizeai/ax-client";

// Add records to the queue from span and session sources
await addAnnotationQueueRecords({
  annotationQueue: "my_queue",
  space: "my-space",
  recordSources: [
    {
      recordType: "SPAN",
      projectId: "proj_abc123",
      startTime: "2024-01-15T00:00:00Z",
      endTime: "2024-01-15T23:59:59Z",
      spanIds: ["span_abc123"],
    },
    {
      recordType: "SESSION",
      projectId: "proj_abc123",
      startTime: "2024-01-15T00:00:00Z",
      endTime: "2024-01-15T23:59:59Z",
      sessionIds: ["session_abc123"],
    },
  ],
});

// List records
const records = await listAnnotationQueueRecords({
  annotationQueue: "my_queue",
  space: "my-space",
});

// Assign a record to the current user
await assignAnnotationQueueRecord({
  annotationQueue: "my_queue",
  space: "my-space",
  annotationQueueRecordId: "aqr_abc123",
});

// Submit annotations for a record
await annotateAnnotationQueueRecord({
  annotationQueue: "my_queue",
  space: "my-space",
  annotationQueueRecordId: "aqr_abc123",
  annotations: [
    { name: "accuracy", label: "correct", score: 1.0 },
    { name: "quality", text: "Well-structured response" },
  ],
});

// Remove records
await deleteAnnotationQueueRecords({
  annotationQueue: "my_queue",
  space: "my-space",
  recordIds: ["aqr_abc123", "aqr_def456"],
});

AI Integrations

The @arizeai/ax-client package allows you to create and manage AI integrations (LLM provider credentials used by evaluators, tasks, and the playground).

Listing AI integrations

import { listAiIntegrations } from "@arizeai/ax-client";

const integrations = await listAiIntegrations({ space: "my-space" });
console.log(integrations);

Creating an AI integration

import { createAiIntegration } from "@arizeai/ax-client";

const integration = await createAiIntegration({
  name: "Production OpenAI",
  provider: "OPEN_AI",
  apiKey: "sk-...",
  modelNames: ["gpt-4o", "gpt-4o-mini"],
  enableDefaultModels: true,
});

Getting an AI integration

import { getAiIntegration } from "@arizeai/ax-client";

// By ID
const integration = await getAiIntegration({
  integration: "your_integration_id",
});

// By name (requires space)
const byName = await getAiIntegration({
  integration: "Production OpenAI",
  space: "my-space",
});

Updating an AI integration

import { updateAiIntegration } from "@arizeai/ax-client";

const integration = await updateAiIntegration({
  integration: "Production OpenAI",
  space: "my-space",
  name: "Updated OpenAI",
  modelNames: ["gpt-4o"],
});

Deleting an AI integration

import { deleteAiIntegration } from "@arizeai/ax-client";

await deleteAiIntegration({
  integration: "Production OpenAI",
  space: "my-space",
});

Integrations

The @arizeai/ax-client package allows you to create and manage integrations on the polymorphic /v2/integrations surface, covering both LLM (model-provider) and AGENT (customer-hosted endpoint) integrations. Integration names are unique per account and type, so type is required when resolving a name; an ID always works on its own. These functions are in alpha and emit a one-time pre-release warning.

Listing integrations

type is an optional filter. Omit it to list integrations of every type in one call; each item carries its type discriminator.

import { listIntegrations } from "@arizeai/ax-client";

// Only agent integrations
const agents = await listIntegrations({ type: "AGENT", space: "my-space" });

// Every type, discriminated by `integration.type`
const all = await listIntegrations({});

Creating an integration

The type field selects the config shape.

import { createIntegration } from "@arizeai/ax-client";

// LLM integration
const llm = await createIntegration({
  type: "LLM",
  name: "Production OpenAI",
  config: { provider: "OPEN_AI", apiKey: "sk-..." },
});

// Agent integration
const agent = await createIntegration({
  type: "AGENT",
  name: "My Support Agent",
  config: {
    endpoint: "https://agent.example.com/replay",
    inputSchema: { type: "object", properties: { input: { type: "string" } } },
    requestPresets: [{ name: "default", config: { input: "hello" } }],
  },
});

config.provider selects the LLM config shape. Secrets are write-only and never returned.

// Self-hosted LiteLLM proxy: the endpoint and virtual key are both required.
const litellm = await createIntegration({
  type: "LLM",
  name: "Team LiteLLM",
  config: {
    provider: "LITELLM",
    baseUrl: "https://litellm.internal:4000",
    apiKey: "sk-...",
  },
});

// Hosted providers such as Fireworks AI and Together AI need only the API key.
// Arize resolves the models the key can reach; list `modelNames` for
// fine-tunes or dedicated deployments that cannot be discovered.
const together = await createIntegration({
  type: "LLM",
  name: "Together AI",
  config: { provider: "TOGETHER_AI", apiKey: "..." },
});

Getting an integration

import { getIntegration } from "@arizeai/ax-client";

// By ID
const integration = await getIntegration({
  integration: "your_integration_id",
});

// By name (requires type)
const byName = await getIntegration({
  integration: "Production OpenAI",
  type: "LLM",
});

Updating an integration

type is required (it selects the update shape) and is immutable server-side. Provide at least one updatable field; omitted fields are preserved.

import { updateIntegration } from "@arizeai/ax-client";

const integration = await updateIntegration({
  integration: "Production OpenAI",
  type: "LLM",
  config: { provider: "OPEN_AI", apiKey: "sk-new-key" },
});

Deleting an integration

import { deleteIntegration } from "@arizeai/ax-client";

await deleteIntegration({
  integration: "My Support Agent",
  type: "AGENT",
});

Projects

The @arizeai/ax-client package allows you to create and manage projects.

Creating a project

import { createProject } from "@arizeai/ax-client";

const project = await createProject({
  space: "my-space",
  name: "my-project",
});
console.log(project);

Getting a project

import { getProject } from "@arizeai/ax-client";

const project = await getProject({
  project: "my-project",
  space: "my-space",
});
console.log(project);

Listing projects

import { listProjects } from "@arizeai/ax-client";

const { projects } = await listProjects({ space: "my-space" });
console.log(projects);

Updating a project

import { updateProject } from "@arizeai/ax-client";

const project = await updateProject({
  project: "my-project",
  space: "my-space",
  name: "my-renamed-project",
});
console.log(project);

Deleting a project

import { deleteProject } from "@arizeai/ax-client";

await deleteProject({ project: "my-project", space: "my-space" });

API Keys

The @arizeai/ax-client package allows you to create, list, delete, revoke, and refresh API keys.

Creating an API key

import { createApiKey } from "@arizeai/ax-client";

const apiKey = await createApiKey({ keyType: "USER", name: "CI pipeline key" });
// Store apiKey.key securely — the full key value is only returned once
console.log(apiKey.key);

Service keys are backed by a dedicated bot user scoped to one or more organizations and spaces:

const apiKey = await createApiKey({
  keyType: "SERVICE",
  name: "service-key",
  organizations: [
    {
      orgId: "T3JnMTIz",
      spaces: [{ spaceId: "U3BhY2UxMjM", role: { name: "MEMBER" } }],
    },
  ],
  expiresAt: new Date("2027-01-01"),
});
// apiKey.botUser — the bot user created for this key, with resolved role assignments

Listing API keys

import { listApiKeys } from "@arizeai/ax-client";

const { data } = await listApiKeys();
console.log(data.map((k) => k.name));

// Filter by key type or status
const { data: userKeys } = await listApiKeys({ keyType: "USER" });

// List service keys for a specific space
const { data: serviceKeys } = await listApiKeys({
  keyType: "SERVICE",
  spaceId: "U3BhY2U6MTIzNDU=",
});

// List service keys created by a specific user (any caller with space access)
const { data: byUser } = await listApiKeys({
  spaceId: "U3BhY2U6MTIzNDU=",
  userId: "VXNlcjoxMjM0NQ==",
});

// Each service key entry carries the same botUser detail createApiKey returns —
// use botUser.id as the userId when creating a role binding for the key.
for (const key of serviceKeys) {
  console.log(key.botUser?.id);
}

Revoking an API key

Sets the key's status to revoked and deactivates it immediately. This operation is irreversible; revoking an already-revoked key is a no-op and still succeeds.

import { revokeApiKey } from "@arizeai/ax-client";

await revokeApiKey({ apiKeyId: "your-api-key-id" });

Refreshing an API key

Atomically revokes an existing key and issues a replacement with the same metadata (name, description, key type). The old key is invalidated and the new key is activated in a single transaction.

import { refreshApiKey } from "@arizeai/ax-client";

const refreshed = await refreshApiKey({ apiKeyId: "your-api-key-id" });
// Store refreshed.key securely — the full key value is only returned once
console.log(refreshed.key);

Supply expiresAt to set an expiration on the replacement key:

const refreshed = await refreshApiKey({
  apiKeyId: "your-api-key-id",
  expiresAt: new Date("2027-01-01"),
});

Supply gracePeriodSeconds to keep the old key valid for a window after the refresh, giving your services time to adopt the new key before the old one is invalidated:

const refreshed = await refreshApiKey({
  apiKeyId: "your-api-key-id",
  gracePeriodSeconds: 3600, // old key stays valid for 1 hour
});
// Store refreshed.key securely — the full key value is only returned once
console.log(refreshed.key);

Roles

The @arizeai/ax-client package allows you to create and manage custom roles.

Creating a role

import { createRole } from "@arizeai/ax-client";

const role = await createRole({
  name: "AI Engineer",
  permissions: ["PROJECT_READ", "DATASET_READ", "DATASET_CREATE"],
  description: "Can read and create datasets and experiments.",
});

Listing roles

import { listRoles } from "@arizeai/ax-client";

const { data } = await listRoles();
console.log(data.map((r) => r.name));

// Filter to only predefined (system) roles
const { data: predefined } = await listRoles({ isPredefined: true });

Getting a role

import { getRole } from "@arizeai/ax-client";

const role = await getRole({ roleId: "your-role-id" });

Updating a role

import { updateRole } from "@arizeai/ax-client";

const role = await updateRole({
  roleId: "your-role-id",
  permissions: ["PROJECT_READ", "DATASET_READ"],
});

Deleting a role

import { deleteRole } from "@arizeai/ax-client";

await deleteRole({ roleId: "your-role-id" });

Role bindings

The @arizeai/ax-client package allows you to bind a role to a user on a specific resource (e.g. a project or space).

Creating a role binding

import { createRoleBinding } from "@arizeai/ax-client";

const binding = await createRoleBinding({
  userId: "VXNlcjoxMjM0NQ==",
  roleId: "Um9sZTphYmMxMjM=",
  resourceType: "PROJECT",
  resourceId: "UHJvamVjdDphYmMxMjM=",
});
console.log(binding);

userId is the ID of the user to bind the role to. To bind a service key, pass its bot user's ID instead of your own — it is returned as botUser.id from createApiKey (see Creating an API key above).

Listing role bindings

import { listRoleBindings } from "@arizeai/ax-client";

const bindings = await listRoleBindings({ resourceType: "SPACE" });
console.log(bindings);

Getting a role binding

import { getRoleBinding } from "@arizeai/ax-client";

const binding = await getRoleBinding({ bindingId: "Um9sZUJpbmRpbmc6YWJjMTIz" });

Updating a role binding

import { updateRoleBinding } from "@arizeai/ax-client";

const binding = await updateRoleBinding({
  bindingId: "Um9sZUJpbmRpbmc6YWJjMTIz",
  roleId: "Um9sZTphYmMxMjM=",
});

Deleting a role binding

import { deleteRoleBinding } from "@arizeai/ax-client";

await deleteRoleBinding({ bindingId: "Um9sZUJpbmRpbmc6YWJjMTIz" });

Resource restrictions

The @arizeai/ax-client package allows you to restrict access to a resource so that only explicitly bound users can access it, and to lift that restriction.

import { restrictResource, unrestrictResource } from "@arizeai/ax-client";

// Restrict a resource
const restriction = await restrictResource({ resourceId: "your_project_id" });
console.log(restriction);

// Lift the restriction
await unrestrictResource({ resourceId: "your_project_id" });

Organizations

The @arizeai/ax-client package allows you to manage organizations.

Listing organizations

import { listOrganizations } from "@arizeai/ax-client";

const result = await listOrganizations();
console.log(result.organizations);

Getting an organization

import { getOrganization } from "@arizeai/ax-client";

const org = await getOrganization({ organization: "your-organization-name" });
console.log(org);

Creating an organization

import { createOrganization } from "@arizeai/ax-client";

const org = await createOrganization({
  name: "your-organization-name",
  description: "Optional description",
});
console.log(org);

Updating an organization

import { updateOrganization } from "@arizeai/ax-client";

const org = await updateOrganization({
  organization: "your-organization-name",
  name: "your-organization-name-updated",
});
console.log(org);

Deleting an organization

Warning: This operation is irreversible and deletes the organization and all resources that belong to it, including all spaces and their contents.

import { deleteOrganization } from "@arizeai/ax-client";

await deleteOrganization({ organization: "your-organization-name" });

Organization memberships

Adding a user to an organization

import { addOrganizationUser } from "@arizeai/ax-client";

const membership = await addOrganizationUser({
  organizationId: "org_abc123",
  userId: "VXNlcjoxMjM0NQ==",
  role: { type: "PREDEFINED", name: "MEMBER" },
});
console.log(membership);

Removing a user from an organization

import { removeOrganizationUser } from "@arizeai/ax-client";

await removeOrganizationUser({
  organizationId: "org_abc123",
  userId: "VXNlcjoxMjM0NQ==",
});

Users

The @arizeai/ax-client package allows you to manage users.

Note: Unlike organizations, users are identified by opaque ID only — not by name. User display names are not unique within an account, so all functions require the user's ID.

Listing users

import { listUsers } from "@arizeai/ax-client";

const result = await listUsers();
console.log(result.data);

Getting a user

import { getUser } from "@arizeai/ax-client";

const user = await getUser({ userId: "VXNlcjoxMjM0NQ==" });
console.log(user);

Creating a user

The inviteMode parameter controls how the user is invited:

• "EMAIL_LINK" — sends the user an email with a verification link.
• "TEMPORARY_PASSWORD" — issues a one-time password returned in the response.
• "NONE" — pre-provisions an SSO user directly; no invitation is sent and the user is immediately active via the configured identity provider.

Returns a UserCreated object (HTTP 201) for a new user, or a User object (HTTP 200) when an existing invited user is returned as-is (idempotent — the invitation is not resent).

import { createUser } from "@arizeai/ax-client";

const user = await createUser({
  name: "Jane Smith",
  email: "[email protected]",
  role: { type: "PREDEFINED", name: "MEMBER" },
  inviteMode: "EMAIL_LINK",
});
console.log(user);

Updating a user

import { updateUser } from "@arizeai/ax-client";

const user = await updateUser({
  userId: "VXNlcjoxMjM0NQ==",
  name: "Jane Smith Updated",
  isDeveloper: true,
});
console.log(user);

Deleting a user

Warning: This operation permanently blocks the user from the account. Blocked users cannot be re-invited — inactive is a terminal state. The operation cascades to organization memberships, space memberships, API keys, and role bindings.

import { deleteUser } from "@arizeai/ax-client";

await deleteUser({ userId: "VXNlcjoxMjM0NQ==" });

Resending a user invitation

import { resendInvitation } from "@arizeai/ax-client";

await resendInvitation({ userId: "VXNlcjoxMjM0NQ==" });

Resetting a user's password

import { resetPassword } from "@arizeai/ax-client";

await resetPassword({ userId: "VXNlcjoxMjM0NQ==" });

Bulk deleting users

Delete multiple users by ID and/or email. Each entry's outcome is returned individually.

import { bulkDeleteUsers } from "@arizeai/ax-client";

const results = await bulkDeleteUsers({
  userIds: ["VXNlcjoxMjM0NQ=="],
  emails: ["[email protected]"],
});
for (const r of results) {
  console.log(r.userId, r.status, r.error ?? "");
}

Spaces

The @arizeai/ax-client package allows you to create and manage spaces.

Space responses from list, get, create, and update include organizationId, the ID of the parent organization, when the server supports this field. Older server versions may omit it.

Listing spaces

import { listSpaces } from "@arizeai/ax-client";

const spaces = await listSpaces({
  organizationId: "QWNjb3VudE9yZ2FuaXphdGlvbjox",
});
console.log(spaces);

Omit organizationId to list accessible spaces from all organizations. The SDK sends this value as the REST query parameter org_id.

Getting a space

import { getSpace } from "@arizeai/ax-client";

const space = await getSpace({ space: "my-space" });
console.log(space);

Creating a space

import { createSpace } from "@arizeai/ax-client";

const space = await createSpace({
  organizationId: "T3JnYW5pemF0aW9uOmFiYzEyMw==",
  name: "your_space_name",
  isPrivate: true, // omit or set false for a public space (default)
});

Private spaces are visible only to their members and account/org/space admins. A warning is logged when isPrivate: true to remind you to add members before the space becomes inaccessible to other users.

Updating a space

import { updateSpace } from "@arizeai/ax-client";

const space = await updateSpace({
  space: "my-space",
  name: "updated_space_name",
  isPrivate: false, // omit to preserve the current visibility
});

Deleting a space

import { deleteSpace } from "@arizeai/ax-client";

await deleteSpace({ space: "my-space" });

Space memberships

Adding a user to a space

import { addSpaceUser } from "@arizeai/ax-client";

const membership = await addSpaceUser({
  spaceId: "spc_abc123",
  userId: "VXNlcjoxMjM0NQ==",
  role: { type: "PREDEFINED", name: "MEMBER" },
});
console.log(membership);

Removing a user from a space

import { removeSpaceUser } from "@arizeai/ax-client";

await removeSpaceUser({
  spaceId: "spc_abc123",
  userId: "VXNlcjoxMjM0NQ==",
});

Webhooks

Webhooks are alpha: every call logs a one-time warning and the API may change. A webhook is an organization-owned HTTPS destination. Pass a webhook ID, or a name together with organization.

Listing webhooks

import { listWebhooks } from "@arizeai/ax-client";

const webhooks = await listWebhooks({ organization: "my-org" });
console.log(webhooks.data);

Creating a webhook

import { createWebhook } from "@arizeai/ax-client";

const webhook = await createWebhook({
  organization: "my-org",
  name: "deploy-notifier",
  url: "https://example.com/hooks/arize",
  authType: "HMAC_SHA256",
});
// Only returned here. Store it now; afterwards only signingSecretHint is readable.
console.log(webhook.signingSecret);

authToken and headers are write-only and never come back in any response.

Getting, updating, testing, and deleting a webhook

import {
  getWebhook,
  updateWebhook,
  testWebhook,
  deleteWebhook,
} from "@arizeai/ax-client";

const webhook = await getWebhook({
  webhook: "deploy-notifier",
  organization: "my-org",
});
// Omitted fields keep their value; pass `description: null` to clear it.
await updateWebhook({
  webhook: "deploy-notifier",
  organization: "my-org",
  timeoutMs: 10000,
});
const outcome = await testWebhook({
  webhook: "deploy-notifier",
  organization: "my-org",
});
console.log(outcome.statusCode, outcome.errorMessage);
await deleteWebhook({ webhook: "deploy-notifier", organization: "my-org" });

Listing delivery attempts

import { listWebhookDeliveryAttempts } from "@arizeai/ax-client";

const attempts = await listWebhookDeliveryAttempts({
  webhook: "deploy-notifier",
  organization: "my-org",
  limit: 100, // up to 500
});
console.log(attempts.data);

Webhook subscriptions

A subscription delivers one event from one prompt or evaluator to one webhook. Subscriptions have no name and are addressed by ID.

Subscribing a webhook to an event

import { createWebhookSubscription } from "@arizeai/ax-client";

const subscription = await createWebhookSubscription({
  webhook: "deploy-notifier",
  organization: "my-org",
  sourceType: "PROMPT",
  sourceId: "your_prompt_id",
  event: "PROMPT_VERSION_CREATED",
});

Listing, getting, and deleting subscriptions

import {
  listWebhookSubscriptions,
  getWebhookSubscription,
  deleteWebhookSubscription,
} from "@arizeai/ax-client";

// sourceType and sourceId go together; omit both to list every readable subscription.
const subscriptions = await listWebhookSubscriptions({
  sourceType: "PROMPT",
  sourceId: "your_prompt_id",
});
const subscription = await getWebhookSubscription({
  subscriptionId: "your_subscription_id",
});
await deleteWebhookSubscription({ subscriptionId: "your_subscription_id" });

REST endpoints

It is recommended to use the methods in this package. If more control is desired, you can use the client directly. The client provides a type-safe fetch for the entire Arize AX REST API.

import { createClient } from "@arizeai/ax-client";

const client = createClient();

// Get all datasets
const response = await client.GET("/v2/datasets");