@apicity/xai

X.AI / Grok provider for chat and search.

Runtime dependencies:

• ws@^8.21.0 — authenticated server WebSocket transport exported from @apicity/xai/realtime
• zod@^4.6.6 — request schemas attached to every POST endpoint as .schema

Installation

npm install @apicity/xai
# or
pnpm add @apicity/xai

Quick Start

import { createXai } from "@apicity/xai";

const xai = createXai({ apiKey: process.env.XAI_API_KEY! });

Real-world example: structured vision analysis with Grok-4

Hand Grok-4 a portrait, a system prompt that nails down the output schema, and text.format.type: "json_object" — get back a reproduction-ready JSON description with deterministic shot/pose vocabulary. The flow below is taken verbatim from tests/integration/xai-vision-json.test.ts and replays against tests/recordings/xai_3613880225/vision-analysis-json_243984103/recording.har, so the response shapes match what xAI actually returns.

import { readFile } from "node:fs/promises";
import { createXai } from "@apicity/xai";

const xai = createXai({ apiKey: process.env.XAI_API_KEY! });

// 1. Load the image and inline it as a data URL. xAI also accepts
//    https:// URLs, but inlining keeps the call self-contained and
//    works against private hosts.
const image = await readFile("./portrait.jpg");
const base64 = image.toString("base64");

// 2. The system prompt enumerates the legal vocabulary for `shot` and
//    constrains `pose` to body geometry only. Combined with
//    `text.format.type: "json_object"` this gives Grok no room to drift
//    off-schema — temperature 0 keeps the result reproducible.
const SYSTEM_PROMPT = [
  "You are an expert image-to-prompt analyst.",
  "Return only a JSON object with keys prompt, shot, and pose.",
  "prompt: a single-paragraph reproduction-ready image prompt, 1900 characters or fewer, with no line breaks.",
  'shot: exactly "<size>, <angle>" where size is one of extreme close-up, close-up, medium close-up, medium shot, medium long shot, long shot, or extreme long shot, and angle is one of eye-level, low-angle, high-angle, overhead, or dutch.',
  "pose: only body geometry for human figures, with no clothing, hair, background, or lighting details.",
].join(" ");

// 3. Multimodal Responses request: system turn + a user turn whose
//    content is an array of `input_image` + `input_text` parts.
const result = await xai.post.v1.responses({
  model: "grok-4",
  input: [
    { role: "system", content: SYSTEM_PROMPT },
    {
      role: "user",
      content: [
        {
          type: "input_image",
          image_url: `data:image/jpeg;base64,${base64}`,
          detail: "high",
        },
        {
          type: "input_text",
          text: 'Analyze this image and produce a reproduction-ready JSON description with keys "prompt", "shot", and "pose".',
        },
      ],
    },
  ],
  text: { format: { type: "json_object" } },
  store: false,
  temperature: 0,
  max_output_tokens: 300,
});

// 4. The Responses API wraps output in a typed item array. Find the
//    assistant message, then the first `output_text` part inside it.
//    Discriminated unions narrow `item.type === "message"` so
//    `item.content` is statically typed.
const message = result.output.find((item) => item.type === "message");
const outputText =
  message?.type === "message"
    ? message.content.find((part) => part.type === "output_text")?.text
    : undefined;

if (!outputText) throw new Error("Grok did not return output_text");

const analysis = JSON.parse(outputText) as {
  prompt: string;
  shot: string;
  pose: string;
};

console.log(analysis.shot);
// → "medium close-up, eye-level"

console.log(analysis.pose);
// → "upright torso facing forward, head straight and centered, shoulders squared, arms relaxed downward (implied)"

// 5. Reasoning-token accounting. Grok-4 spent 623 of its 728 output
//    tokens reasoning before emitting the 105-token JSON answer —
//    surfaced in `usage.output_tokens_details.reasoning_tokens`.
console.log(result.usage);
// → {
//     input_tokens: 2684,
//     input_tokens_details: { cached_tokens: 679 },
//     output_tokens: 728,
//     output_tokens_details: { reasoning_tokens: 623 },
//     total_tokens: 3412,
//   }

Notes

• store: false keeps the response off xAI's history surface. Flip to true to chain follow-ups via previous_response_id — useful for multi-turn refinement ("now describe the wardrobe") without re-uploading the image each time.
• The Responses output array also carries reasoning items and tool calls when present. Always discriminate on item.type before reading content; TypeScript's narrowing keeps you honest.
• For raw chat-style usage without the Responses wrapping, use xai.post.v1.chat.completions instead — same auth, same model catalog, just OpenAI-compatible request/response shapes.
• Errors surface as XaiError with status and the parsed body attached, so try { ... } catch (e) { if (e instanceof XaiError) ... } gives you the upstream error directly.

Imagine Files API integration

xAI Imagine image and video endpoints can reference private Files API assets directly and can persist generated assets back to Files storage.

• Inputs: anywhere Imagine accepts a public URL or base64 image/video, pass a stored file_id instead. Apicity accepts the raw REST shape (image: { file_id }, images: [{ file_id }], last_frame: { file_id }, video: { file_id }, reference_images: [{ file_id }]) plus convenience aliases (image_file_id, image_file_ids, last_frame_file_id, video_file_id, and reference_image_file_ids) that are normalized before the HTTP request.
• Outputs: pass storage_options with a required filename to persist the generated image or video. Omit public_url or set it to false for a private file; set public_url: true or public_url: { expires_after: 86400 } to create a shareable URL.
• Responses still include the default ephemeral imgen.x.ai or vidgen.x.ai generation URL. When storage is requested, the persistent Files metadata is returned as file_output on the generated image or completed video.

Reference-to-video, pinned frames and preset voices

xai.post.v1.videos.generations supports reference-to-video requests with up to 14 reference images on the grok-imagine-video-1.5 family (7 on grok-imagine-video), one to three preset voices, or both. Image references may use public HTTPS URLs, data URIs, or Files API IDs in any mixture. Preset voices use open, case-insensitive xAI voice identifiers; Apicity preserves each voice_id exactly and leaves voice recognition to xAI. Preset voices and pinned frames require the grok-imagine-video-1.5 family, including its -preview and dated aliases. Model-less requests with references, a last frame or keyframes default to the canonical grok-imagine-video-1.5 model.

const video = await xai.post.v1.videos.generations({
  prompt: "The person from <IMAGE_0> speaks the line with the voice from <AUDIO_0>",
  model: "grok-imagine-video-1.5",
  reference_images: [{ url: "https://example.com/character.png" }],
  reference_audios: [{ voice_id: "eve" }],
  duration: 15,
});

On the 1.5 family a request can also pin frames. image (or image_file_id) is the first frame, alone or beside references; last_frame (or last_frame_file_id) is the last frame; and keyframes pins up to four images inside the clip. Each keyframe's timestamp_s must be greater than 0 and less than duration (8 seconds when omitted). xAI snaps keyframes to a 1/3-second grid, so two keyframes that round to the same slot are rejected. prompt is optional whenever a frame is pinned; text-to-video and reference-only requests still need one. A pinned first frame takes <IMAGE_0>, and the reference images then start at <IMAGE_1>.

const timeLapse = await xai.post.v1.videos.generations({
  model: "grok-imagine-video-1.5",
  image: { url: "https://example.com/dawn.png" },
  keyframes: [
    { image: { url: "https://example.com/noon.png" }, timestamp_s: 3.5 },
    { image: { url: "https://example.com/dusk.png" }, timestamp_s: 6.5 },
  ],
  last_frame: { url: "https://example.com/night.png" },
  duration: 10,
  resolution: "720p",
});

The same callable accepts image-only and audio-only reference requests. Empty reference and keyframe arrays are inactive. A source video (video or video_file_id) excludes references and pinned frames on every model, and grok-imagine-video also refuses an image beside references. Prompt markers such as <IMAGE_0> and <AUDIO_0> are passed through unchanged.

Stored input field map

Use these fields when an image or video already lives in xAI Files storage. Apicity normalizes the convenience aliases into the REST file_id object shape before sending the request.

You can also pass the raw REST fields directly. images and reference_images entries can mix { file_id } and { url } items in the same request, which is useful when only some references are already private Files assets. Stored images must be PNG, JPEG, or WebP; stored videos must be MP4; and the file upload must be complete before it is referenced by an Imagine endpoint.

const gen = await xai.post.v1.images.generations({
  prompt: "A futuristic city skyline at night",
  model: "grok-imagine-image-2.0",
  quality: "low",
  storage_options: { filename: "city.jpg" },
});
const city = gen.data[0].file_output!.file_id!;

const edit = await xai.post.v1.images.edits({
  prompt: "Add neon signs to the buildings",
  model: "grok-imagine-image-2.0",
  quality: "low",
  image_file_id: city,
  storage_options: { filename: "city-neon.jpg" },
});
const neonCity = edit.data[0].file_output!.file_id!;

const video = await xai.post.v1.videos.generations({
  prompt: "A camera pulls back through the city",
  model: "grok-imagine-video",
  duration: 5,
  image_file_id: neonCity,
  storage_options: {
    filename: "city-loop.mp4",
    public_url: true,
  },
});

const done = await xai.get.v1.videos(video.request_id);
console.log(done.video?.url);
console.log(done.video?.file_output?.public_url);

See xAI's Imagine Files API integration, Referencing Files as Input, Persisting Generated Output, Managing Files, and Files Public URLs docs for uploads, expiration, and public URL lifecycle details.

Files Public URLs

Files uploaded to xAI storage are private by default. Use xai.post.v1.files.publicUrl(fileId) to create a shareable xAI CDN URL for an existing file, then revoke that URL independently with xai.post.v1.files.publicUrl.revoke(fileId) when sharing should stop. Revoking the public URL leaves the private file intact.

const file = await xai.post.v1.files(
  new Blob(["diagram"], { type: "image/png" }),
  "diagram.png",
  "assistants"
);

const created = await xai.post.v1.files.publicUrl(file.id, {
  expires_after: 86400,
});
console.log(created.public_url);
console.log(created.expires_at);

const withPublicUrls = await xai.get.v1.files({
  filter: "public_url != null",
});
console.log(withPublicUrls.data[0]?.public_url);

await xai.post.v1.files.publicUrl.revoke(file.id);

Public URL lifecycle

• Empty create bodies use xAI defaults. Pass expires_after in seconds to auto-revoke the URL after 1 hour to 30 days.
• A public URL cannot outlive its file. If the file has its own expires_at, an omitted public URL expiry inherits the file expiry; an explicit expires_after must fit inside the file's remaining lifetime.
• Create is idempotent while a file already has an active public URL: repeated calls return the same URL token and can update its expiry.
• get.v1.files(fileId) and get.v1.files({ filter }) preserve public_url and public_url_expires_at metadata so callers can audit which files are currently public.

See xAI's Files Public URLs, Managing Files, and Imagine Files API integration docs for supported content types, size limits, and the storage_options.public_url generation path.

Realtime server transport

@apicity/xai/realtime exports connectXaiRealtime, a WebSocket transport for server-side voice sessions built on ws. It sends your API key as an Authorization: Bearer header, so keep it on the server; browsers should connect with an ephemeral secret from xai.post.v1.realtime.clientSecrets instead.

import { connectXaiRealtime } from "@apicity/xai/realtime";

const connection = await connectXaiRealtime({
  apiKey: process.env.XAI_API_KEY!,
  model: "grok-voice-latest",
  onEvent: (event) => console.log(event.type),
});

connection.send({
  type: "session.update",
  session: { voice: "eve", audio: { output: { speed: 1.1 } } },
});

// Speak a scripted line without model inference. The force_message
// item is the whole turn, so do not follow it with response.create.
connection.send({
  type: "conversation.item.create",
  item: {
    type: "force_message",
    role: "assistant",
    interruptible: false,
    content: [{ type: "output_text", text: "This call is being recorded." }],
  },
});

connection.close();
await connection.closed;

• Pass callId instead of model to attach to an inbound SIP call.
• audio.output.speed is a playback multiplier from 0.7 to 1.5 (default 1.0).
• interruptible: false drops caller audio until the force message finishes playing (default true).
• timeout bounds the handshake (default 15 s) and signal aborts it or an open connection. A rejected handshake rejects with XaiError carrying the HTTP status and parsed body.
• A server event that is not a JSON text frame with a type closes the connection and reports Invalid Realtime event through onError.

See xAI's Speech to Speech docs for session parameters and events.

API Reference

55 endpoints across 18 groups. Each method mirrors an upstream URL path.

apiKey

const res = await xai.get.v1.apiKey();

batches

const res = await xai.get.v1.batches({ /* ... */ });

const res = await xai.get.v1.batches.requests({ /* ... */ });

const res = await xai.get.v1.batches.results({ /* ... */ });

const res = await xai.post.v1.batches({ /* ... */ });

const res = await xai.post.v1.batches.cancel({ /* ... */ });

const res = await xai.post.v1.batches.requests({ /* ... */ });

chat

const res = await xai.get.v1.chat.deferredCompletion({ /* ... */ });

const res = await xai.post.v1.chat.completions({ /* ... */ });

customVoices

const res = await xai.delete.v1.customVoices({ /* ... */ });

const res = await xai.get.v1.customVoices({ /* ... */ });

const res = await xai.get.v1.customVoices.audio({ /* ... */ });

const res = await xai.patch.v1.customVoices({ /* ... */ });

const res = await xai.post.v1.customVoices({ /* ... */ });

documents

const res = await xai.post.v1.documents.search({ /* ... */ });

files

const res = await xai.delete.v1.files({ /* ... */ });

const res = await xai.get.v1.files({ /* ... */ });

const res = await xai.get.v1.files.content({ /* ... */ });

const res = await xai.post.v1.files({ /* ... */ });

const res = await xai.post.v1.files.publicUrl({ /* ... */ });

const res = await xai.post.v1.files.publicUrl.revoke({ /* ... */ });

imageGenerationModels

const res = await xai.get.v1.imageGenerationModels({ /* ... */ });

images

const res = await xai.get.v1.images({ /* ... */ });

const res = await xai.post.v1.images.edits({ /* ... */ });

const res = await xai.post.v1.images.generations({ /* ... */ });

languageModels

const res = await xai.get.v1.languageModels({ /* ... */ });

managementApi

const res = await xai.delete.managementApi.v1.collections({ /* ... */ });

const res = await xai.delete.managementApi.v1.collections.documents({ /* ... */ });

const res = await xai.get.managementApi.auth.teams.apiKeys({ /* ... */ });

const res = await xai.get.managementApi.v1.billing.teams.postpaid.invoice.preview({ /* ... */ });

const res = await xai.get.managementApi.v1.billing.teams.postpaid.spendingLimits({ /* ... */ });

const res = await xai.get.managementApi.v1.billing.teams.prepaid.balance({ /* ... */ });

const res = await xai.get.managementApi.v1.collections({ /* ... */ });

const res = await xai.get.managementApi.v1.collections.documents({ /* ... */ });

const res = await xai.get.managementApi.v1.collections.documents.batchGet({ /* ... */ });

const res = await xai.patch.managementApi.v1.collections.documents({ /* ... */ });

const res = await xai.post.managementApi.v1.billing.teams.usage({ /* ... */ });

const res = await xai.post.managementApi.v1.collections({ /* ... */ });

const res = await xai.post.managementApi.v1.collections.documents({ /* ... */ });

const res = await xai.put.managementApi.v1.collections({ /* ... */ });

models

const res = await xai.get.v1.models({ /* ... */ });

realtime

const res = await xai.post.v1.realtime.clientSecrets({ /* ... */ });

responses

const res = await xai.delete.v1.responses({ /* ... */ });

const res = await xai.get.v1.responses({ /* ... */ });

const res = await xai.post.v1.responses({ /* ... */ });

const res = await xai.post.v1.responses.compact({ /* ... */ });

stt

const res = await xai.post.v1.stt({ /* ... */ });

tokenizeText

const res = await xai.post.v1.tokenizeText({ /* ... */ });

tts

const res = await xai.post.v1.tts({ /* ... */ });

videoGenerationModels

const res = await xai.get.v1.videoGenerationModels({ /* ... */ });

videos

const res = await xai.get.v1.videos({ /* ... */ });

const res = await xai.post.v1.videos.edits({ /* ... */ });

const res = await xai.post.v1.videos.extensions({ /* ... */ });

const res = await xai.post.v1.videos.generations({ /* ... */ });

const res = await xai.post.v1.videos.generations.imageToVideo({ /* ... */ });

Middleware

import { createXai, withRetry } from "@apicity/xai";

const xai = createXai({ apiKey: process.env.XAI_API_KEY! });
const models = withRetry(xai.get.v1.models, { retries: 3 });

Rate Limiting

Client-side rate limiting that queues requests to stay within xAI API limits.

import {
  createXai,
  withRateLimit,
  withRetry,
  createRateLimiter,
  XAI_RATE_LIMITS,
} from "@apicity/xai";

const xai = createXai({ apiKey: process.env.XAI_API_KEY! });

Using xAI tier presets

// Use built-in tier presets (free, tier1, tier2, tier3, tier4)
const limiter = createRateLimiter(XAI_RATE_LIMITS.tier1);
// => { rpm: 60, concurrent: 10 }

const chat = withRateLimit(xai.post.v1.chat.completions, limiter);

Custom limits

const limiter = createRateLimiter({ rpm: 30, concurrent: 5 });
const chat = withRateLimit(xai.post.v1.chat.completions, limiter);

Shared limiter across endpoints

RPM limits apply globally, so share a single limiter across all endpoints:

const limiter = createRateLimiter(XAI_RATE_LIMITS.tier2);

const chat = withRateLimit(xai.post.v1.chat.completions, limiter);
const responses = withRateLimit(xai.post.v1.responses, limiter);

Composing with retry

Place withRateLimit innermost so retries count against the limit:

const limiter = createRateLimiter(XAI_RATE_LIMITS.tier1);

const chat = withRetry(
  withRateLimit(xai.post.v1.chat.completions, limiter),
  { retries: 2 }
);

Batch processing

Fire requests in parallel — the limiter handles pacing automatically:

const limiter = createRateLimiter(XAI_RATE_LIMITS.tier1);
const chat = withRateLimit(xai.post.v1.chat.completions, limiter);
const prompts = ["Name a prime number.", "Name a noble gas."];

const results = await Promise.all(
  prompts.map((p) =>
    chat({
      model: "grok-3",
      messages: [{ role: "user", content: p }],
    })
  )
);

xAI rate limit tiers

Part of the apicity monorepo.

License

MIT — see LICENSE.