@dbx-tools/model

Resolve a loose model request such as "claude sonnet" or "chat-fast" to a concrete Databricks Model Serving endpoint in the active workspace. The client lists /serving-endpoints, caches and enriches the catalogue, classifies endpoints by capability, fuzzy-matches names, and falls back to a small static floor when the live catalogue is unavailable.

Quick start

import { createModelClient } from "@dbx-tools/model";

const models = await createModelClient();
const selected = await models.resolveModel({ explicit: "sonnet" });
console.log(selected.modelId);

By default, non-exact requests route to workspace serving endpoints such as databricks-gpt-5-6-sol. Opt in to route Responses requests through the matching system.ai.* AI Gateway service instead:

const models = await createModelClient({ preferSystemModels: true });
const route = await models.route({ explicit: "gpt-5-6-sol", protocol: "responses" });
// route.modelId === "system.ai.gpt-5-6-sol"

An exact endpoint id or exact system.ai.* name always wins, regardless of this preference.

Browser-safe request/result schemas and endpoint classification types live in @dbx-tools/shared-model.

The package:

• Lists Databricks Model Serving endpoints through the SDK and normalizes them into a stable summary shape with TypeScript-owned family and reasoning-effort metadata.
• Classifies endpoints into chat-thinking, chat-balanced, chat-fast, and embedding classes using Foundation Model API scores and family heuristics.
• Resolves loose user input such as "sonnet" or "chat fast" to a concrete endpoint id.
• Supports class ceilings so callers can ask for a capability band without accidentally escalating to a larger model.
• Supports requiresTools filtering so callers select only endpoints verified for a complete function-call and result-replay round-trip.
• Supports structured-output filtering and quality, speed, or cost optimization; speed-sensitive selection prefers non-reasoning endpoints before comparing live profile speed scores.
• Caches enriched catalogues per workspace host through AppKit cache utilities, and re-lists once on a resolve miss so a newly deployed model still resolves.
• Builds the invocations URL and mints per-request auth headers for callers that issue their own OpenAI-shaped HTTP requests.
• Provides build-generated retirement, Responses, image-input, apply-patch, web-search, reasoning, tool-calling, structured-output, temperature-support, latency, human-preference rating, ITPM, OTPM, QPH, and token-price metadata through cached in-process lookups. Databricks documentation is enriched with models.dev and modelparams.dev snapshots. Normalized latency metadata uses the MIT-licensed vicaroai/tokens-per-second benchmark after Databricks and those catalogues provide no per-model latency field. Ratings use every CC BY 4.0 LMArena latest parquet configuration published through Hugging Face. Token prices come only from Databricks pricing pages, and committed fallbacks keep refreshes and runtime startup offline-safe.
• Provides a small static fallback floor for local tools and degraded workspace access.

Use with AppKit Model Serving

Use native AppKit's Model Serving plugin when you already know the endpoint alias you want. It gives you authenticated invoke/stream routes, OBO execution, generated endpoint types, request-body filtering, and frontend hooks.

Use this package before or beside that layer when the hard part is choosing the endpoint:

• Resolve loose human input such as "sonnet" or "fast" against the live workspace catalogue.
• Group endpoints into capability classes like chat-thinking, chat-balanced, chat-fast, and embedding.
• Enforce class ceilings so a caller can degrade to smaller models without escalating to a larger one.
• Build model pickers and debug routes from a cached, enriched endpoint list.
• Keep local agents and CLIs working with a static fallback when catalogue access is unavailable.

Route requests

import { createModelClient } from "@dbx-tools/model";

const models = await createModelClient({
  auth: { profile: "DEFAULT" },
  cacheTtlMs: 60_000,
});

const selected = await models.resolveModel({ explicit: "gpt" });
const route = await models.route({
  explicit: selected.modelId,
  protocol: "responses",
});

const response = await fetch(route.url, {
  method: "POST",
  headers: { ...route.headers, "content-type": "application/json" },
  body: JSON.stringify({ model: route.modelId, input: "Hello" }),
});

createModelClient() is the portable auth-backed facade used by Node clients. It lists and normalizes endpoints, caches one catalogue per host, workspace ID, and principal, refreshes once after an unmatched fuzzy request, resolves routes, returns complete authentication headers, exposes profiles, and returns the package's generated metadata. Its portable @dbx-tools/model/python entrypoint excludes the AppKit catalogue wrapper while reusing the same pure ranking and policy implementation.

Select one model

import { createWorkspaceClient } from "@databricks/appkit";
import { resolve } from "@dbx-tools/model";

const client = createWorkspaceClient();
const host = String(await client.config.getHost());

const selected = await resolve.selectModel(client, host, {
  explicit: "claude sonnet",
});

console.log(selected.modelId, selected.source);

selectModel() is the high-level helper for agents and CLIs. It reads the live catalogue, applies fuzzy matching when an explicit string is present, then returns a single modelId plus a source label explaining why that endpoint won. Set liveOnly: true for an unpinned production default that must select the highest-ranked available endpoint and must not fall through to an offline static id when the catalogue has no match. With no explicit id or class, automatic selection prefers the highest-ranked live GPT family member, then falls back to the highest-ranked live chat model.

The source label is useful for logs and debug UIs. It distinguishes explicit matches from class-based selection, environment defaults, and fallback results, so operators can tell whether a request used the intended model policy.

Build a model picker

import { resolve } from "@dbx-tools/model";

const ranked = await resolve.searchModels(client, host, {
  search: "opus",
  modelClass: "chat-thinking",
  requiresTools: true,
  limit: 5,
});

Use searchModels() for UI pickers and debug routes. It returns ranked models with match scores and endpoint summaries, using the same fuzzy threshold and class ceiling logic as selectModel(). Classification, fuzzy scoring, family-version order, and variant preference are implemented directly in this package and require no native binding.

Filter before ranking when a caller needs a specific runtime shape. For example, select the best 1,024-dimensional embedding endpoint:

const [embedding] = await resolve.searchModels(client, host, {
  modelClass: "embedding",
  dimension: 1024,
  limit: 1,
});

Search filters compose: use name for an exact endpoint, search for fuzzy name and label matching, modelClass for class-aware ranking, task for an exact serving task, dimension or minDimension/maxDimension for embedding shape, reasoningEffort for supported reasoning levels, and requiresTools for complete tool-call support.

Work with a held catalogue

When you already have endpoint summaries, use the pure resolver functions from resolve and modelCatalog without another workspace call:

import { modelCatalog, resolve } from "@dbx-tools/model";

const endpoints = await modelCatalog.listServingEndpoints(client, host);
const ranked = resolve.lookupModels(endpoints, { search: "sonnet", limit: 3 });
const picked = resolve.resolveModel(endpoints, {
  explicit: "claude sonnet",
  modelClass: "chat-balanced",
  requiresTools: true,
});

The class acts as a ceiling. chat-balanced may fall back to chat-fast, but does not escalate to chat-thinking. Embedding endpoints are considered only when the requested class is embedding.

List and cache serving endpoints

import { modelCatalog } from "@dbx-tools/model";

const endpoints = await modelCatalog.listServingEndpoints(client, host, {
  ttlMs: 5 * 60_000,
});

const raw = await modelCatalog.listServingEndpointsUncached(client);
await modelCatalog.clearServingEndpointsCache(host);

listServingEndpoints() uses the shared memory-first model project cache, enriches endpoints with classification and embedding dimensions, and isolates entries by workspace host and trusted credential identity. If a refresh fails, it can return the last known-good value retained on disk. listServingEndpointsUncached() is useful for scripts that only need the SDK response and do not want a cache dependency.

Fuzzy resolve endpoint names

const matches = serving.searchServingEndpoints("claude sonnet", endpoints, {
  threshold: 0.35,
});

const endpointName = serving.resolveModelId("sonnet", endpoints);

Fuzzy matching is intentionally a server concern because it depends on the live workspace catalogue and may re-list on misses. Disable it in callers that require exact endpoint ids.

Resolve against a catalogue that may be stale

const resolved = await resolve.rankModelIdLive((force) => loadCatalogue(force), "opus");

rankModelIdLive() matches the catalogue you already hold and, on a miss, reloads once with force before giving up, so a model deployed after your cache warmed still resolves without a restart. You supply the loader, so the caching policy stays yours: listServingEndpoints and its shared project cache, a plain field in a long-lived CLI, or a test double. rankModelId() is the pure form over a single snapshot. Unlike serving.resolveModelId, it breaks equal match scores by class and then version, so "opus" prefers opus-5 over opus-4-7.

Call an endpoint directly

import { invoke } from "@dbx-tools/model";

const response = await fetch(invoke.invocationsUrl(host, endpointId), {
  method: "POST",
  headers: { ...(await invoke.authHeaders(client)), "content-type": "application/json" },
  body: JSON.stringify({ messages }),
});

Use invoke when you need to issue your own request against a serving endpoint with an OpenAI-shaped body, such as from a proxy, a passthrough route, or a streaming client, rather than the SDK's typed servingEndpoints.query. Mint authHeaders() per request: the SDK refreshes the underlying token as it nears expiry, so you never track lifetimes yourself.

Every serving path this repo talks to is a constant here, with a matching URL builder: INVOCATIONS_SUFFIX / invocationsUrl(), RESPONSES_PATH / responsesUrl(), OPEN_RESPONSES_PATH / openResponsesUrl(), and CHAT_COMPLETIONS_PATH / chatCompletionsUrl(). Import one instead of writing a /serving-endpoints/... literal in a consumer: which path a model can accept is a property of the model (see isResponsesOnly() and responsesUpstreamUrl()), so a hard-coded string in one package silently diverges when that routing changes. isResponsesOnly() covers Codex and GPT 5.4+ endpoints, which reject tool-bearing Chat Completions, while keeping GPT-OSS on its supported Chat path. The TypeScript helper applies this policy so every Node consumer shares one version threshold without loading a native library.

Read published model metadata

import { metadata } from "@dbx-tools/model";

const capabilities = metadata.modelCapabilitiesFor("databricks-gpt-5-4");
const latencyScore = metadata.modelLatencyScoreFor("databricks-gpt-6-luna");
const rating = metadata.modelRatingFor("databricks-gpt-6-sol");
const limits = metadata.modelRateLimitsFor("gpt-5.4");
const prices = metadata.modelTokenPricesFor("system.ai.gpt-5-4");
const estimate = metadata.estimateModelTokenPrice(
  "databricks-gpt-5-4",
  { inputTokens: 1_000_000, outputTokens: 500_000 },
  { condition: "short-context", usdPerDbu: 0.07 },
);
const status = metadata.modelStatusFor("system.ai.gemini-2-5-pro");

latencyScore?.score;
latencyScore?.measurements.outputTokensPerSecond;
rating?.score;
rating?.rankings.text_style_control?.lowerBound;

The package ships committed snapshots generated from Databricks documentation and public model catalogues by bun run --filter '@dbx-tools/model' metadata. Runtime calls normalize endpoint, display, foundation-model, and provider identities into memoized sets and maps. They do not fetch documentation or benchmarks. modelMetadataFor() returns status, capabilities, normalized latency metadata, Arena ratings, published limits, and Databricks DBU rates per million input, output, cache-read, and cache-write tokens together for a string or ServingEndpointSummary.

latencyScore.score is the benchmark's median normalized response time in milliseconds, so lower values are faster. latencyScore.measurements retains median time to first token, median output throughput, successful run count, normalized completion length, and measurement time. rating.score is a 0-100 mean percentile across the model's available LMArena configurations, so higher values are better even when source scales differ. rating.rankings preserves each source score, lower and upper confidence bounds, rank, publication date, preference votes and variance, or agent observation and session counts.

When Arena lists multiple reasoning-effort variants, each configuration keeps its highest-rated overall row. Lookups prefer an exact model match and otherwise inherit from the newest measured model in the same family and variant. Consumers can sort the root scores directly or apply their own confidence, freshness, and coverage policy to the nested values. Multiply token rates by the applicable Databricks SKU price per DBU to estimate currency cost.

Native web-search capability starts with the documented model set. Later versions in the same hosted foundation-model family inherit the capability by versionTuple. GPT-OSS models never inherit native web search. Custom and external endpoint names also don't inherit because digits in those names aren't treated as model versions.

The model gateway can override a discovered capability for routing and model catalogues. Use that gateway override to disable web search when a workspace route doesn't support a capability that metadata would otherwise advertise.

Use static fallbacks

import { classes, fallback } from "@dbx-tools/model";
import { model } from "@dbx-tools/shared-model";

const cls = classes.parseModelClass("chat-fast") ?? model.ModelClass.ChatFast;
const modelId = fallback.modelForClass(cls);

The fallback floor gives agents and local scripts a stable answer when a workspace cannot list endpoints. Prefer live catalogue resolution for production policy decisions. Use fallbacks only when live resolution is unavailable.

Modules

• resolve - high-level selectModel, TypeScript-ranked search, and catalogue-held resolver functions.
• serving - Databricks serving-endpoint listing, cache management, fuzzy search, and endpoint-id resolution.
• invoke - serving-path constants, URL construction, and per-request auth headers for calling an endpoint over raw HTTP.
• metadata - build-generated retirement, capability, latency, rating, rate-limit, and token-price snapshots with cached normalized lookups.
• modelClient - portable authentication-aware discovery, selection, routing, profile, and metadata facade.
• classes - model-class parsing, ordering, and class-ceiling helpers.
• policy - family, serving-protocol, tool, and reasoning-effort policy.
• fallback - static fallback model ids per class.

The AppKit-Mastra integration uses this package through @dbx-tools/appkit-mastra.