@adia-ai/llm

Provider-agnostic LLM client. Three adapters (anthropic / openai / gemini) behind a single chat() + streamChat() facade. Works in browser and Node.

Install

npm install @adia-ai/llm

Usage

import { chat, streamChat } from '@adia-ai/llm';

// Direct API call (apiKey owned by the caller)
const reply = await chat({
  apiKey: 'sk-...',
  model: 'gpt-4o-mini',
  messages: [{ role: 'user', content: 'Hello' }],
});

// Streaming
for await (const chunk of streamChat({
  apiKey: 'sk-...',
  model: 'claude-haiku-4-5-20251001',
  messages: [{ role: 'user', content: 'Hello' }],
})) {
  if (chunk.type === 'text') process.stdout.write(chunk.text);
}

Browser dev-mode warning (since v0.4.3): if createAdapter() resolves an apiKey while running in a browser, the bridge emits a one-shot masked console.warn (e.g. sk-ant-a…Fiw-) noting that the key will be sent in request headers. Fine for local dev — never deploy this shape. The warning is dedup'd via window.__adia_llm_key_warning_shown (one warn per page load). For production, use proxy mode (next section) to keep the key server-side.

Browser proxy mode

proxyUrl routes through a server-side proxy so the API key never reaches the browser. The client supports two proxy shapes and auto-detects which to use based on the URL:

Smart proxy (provider-neutral body)

The default. Send any proxyUrl that doesn't match the passthrough pattern below — typically your own backend route like /api/chat. The client speaks a single provider-neutral protocol; the proxy holds the API key and dispatches internally to the right upstream adapter.

for await (const chunk of streamChat({
  proxyUrl: '/api/chat',
  provider: 'openai',          // optional — auto-detected from model
  model: 'gpt-4o-mini',
  messages: [{ role: 'user', content: 'Hello' }],
})) { /* ... */ }

The body sent to the proxy:

{
  "provider": "openai",
  "model": "gpt-4o-mini",
  "messages": [{ "role": "user", "content": "Hello" }],
  "system": "...optional...",
  "maxTokens": 4096,
  "temperature": 0.7,
  "stream": true
}

The proxy reformats per upstream provider and pipes SSE bytes verbatim. The reference smart-proxy implementation is at packages/llm/core/server.js in the chat-ui repo (route: POST /api/chat, plus /api/generate, /api/generate/reset, /api/convert-html for the A2UI generation pipeline). It is not shipped with the npm package — it's a development convenience for the in-repo apps.

A thinking: true request also accepts an optional thinkingBudget (gh#3517): if the caller names no maxTokens, the anthropic body builder's own default grows to the budget plus 4096 tokens of answer headroom instead of the plain 4096 default, so the default 4000-token thinking budget isn't silently clamped down to leave no room to answer. An explicit maxTokens is left untouched; the adapter still enforces the provider floor (budget_tokens >= 1024, < max_tokens).

Passthrough proxy (real upstream body)

When proxyUrl matches /api/llm/<provider>/<rest> (the Vite-dev shape used by chat-ui apps), the client switches to passthrough mode. The proxy is "dumb" — it just rewrites the URL to the real upstream (https://api.<provider>.com/<rest>) and forwards bytes unchanged. The client sends the real upstream body shape plus the adapter's normal auth headers.

This is auto-detected — you don't pick it explicitly. If you mounted a Vite proxy like:

// vite.config.js
server: {
  proxy: {
    '/api/llm/anthropic': {
      target: 'https://api.anthropic.com',
      rewrite: (p) => p.replace(/^\/api\/llm\/anthropic/, ''),
    },
  },
},

…then passing proxyUrl: '/api/llm/anthropic/v1/messages' will produce a request the upstream understands directly.

Detection lives in adapters/index.js — regex /\/api\/llm\/[a-z]+(\/|$)/.

Production deployment

Neither server.js (smart proxy reference) nor any Vite/nginx URL rewrite (passthrough reference) is shipped by the npm package — both are development-time conveniences for the in-repo apps. Production consumers must deploy their own proxy: a small server that holds your provider API key(s) and either:

1. Speaks the smart-proxy contract — accepts the provider-neutral body documented above and dispatches per-provider. See packages/llm/core/server.js for a reference implementation (Express + chat/streamChat from this package, ~150 LOC).
2. Speaks the passthrough contract — exposes /api/llm/<provider>/<rest> and forwards to https://api.<provider>.com/<rest> with the real API-key header injected server-side. See the Vite config snippet above for the shape; a 50-line nginx or Express proxy works fine.

Either contract works — the client auto-detects which one your proxy implements by URL shape. Pick the one that matches your existing infrastructure.

Platform-native gateways (no proxy code needed)

If your VM runs on a managed platform that provides an LLM Gateway natively (e.g. exe.dev at http://169.254.169.254/gateway/llm/<provider>/<rest> on every VM, included in subscription), the cleanest production deploy skips both contracts above and just reverse-proxies /api/llm/* through Caddy/nginx directly to the platform endpoint. No API key in app config, no hand-rolled proxy code, no per-cycle maintenance — the platform handles auth, billing, model selection.

The AdiaUI repo demonstrates this pattern. infra/caddy/llm-gateway.snippet holds a drop-in Caddyfile fragment for exe.dev VMs. The §188 (v0.5.5) arc deleted ~155 LOC of hand-rolled passthrough from server.js in favor of this approach. See .claude/docs/specs/exe-dev-platform-reference.md

• .claude/docs/guides/production-llm-proxy-deployment.md for the full recipe (the architectural migration arc lives in git history around v0.5.5).

Stage 0b recon checklist: before hand-rolling a proxy, check whether your platform provides one natively. Look for LLM Gateway / AI / Inference in the platform's docs. Other platforms with similar gateways: Cloudflare AI Gateway, AWS Bedrock proxy, Azure OpenAI private endpoints. Match shape; check pricing; prefer platform-native over hand-rolled when available.

Subpath exports