@feltdb/webllm

Studio proposes and previews. CLI applies to the repository. FeltDB validates and stores the proposal as application state. Human approves. Nothing else writes source.

This package performs local inference and returns hash-bound proposals. It has no repository filesystem authority.

Pass model to select another WebLLM-compatible local model. Hosted and other OpenAI-compatible models are configured with feltdb ai model set; the CLI keeps their credentials out of the browser. All providers share the Core-owned FlowSpec guide, contract grounding, validation, and constrained repair pass.

Run a small language model entirely in the browser and use it as a FeltDB AI provider. Prompts and inference stay on the user's device; no frontier-model account or hosted inference API is required.

Install

npm install @feltdb/webllm

The package requires a browser with WebGPU. On first use, WebLLM downloads the selected model's artifacts and caches them in the browser.

Generate locally

import { WebLLMProvider } from '@feltdb/webllm';

const llm = new WebLLMProvider({
  onProgress: ({ progress, text }) => {
    console.log(`${Math.round(progress * 100)}%`, text);
  },
});

const answer = await llm.generate([
  { role: 'system', content: 'Return concise JSON.' },
  { role: 'user', content: 'Design a FeltDB task tracker.' },
], {
  responseFormat: { type: 'json_object' },
});

Typed semantic decisions

WebLLMDecisionRuntime is a browser-only evaluator for authorized context. It supports binary, choice, score, and isolated DecisionBatch requests using temperature 1 direct-label logprobs; it has no database or persistence authority.

const runtime = new WebLLMDecisionRuntime({ modelRevision: 'pinned-model-sha' });
const result = await runtime.decide({
  kind: 'choice',
  question: 'Which tier?',
  options: ['free', 'pro'],
  context: authorizedContext,
});

Initialization is lazy: the model loads on the first call to generate() or stream(). Call initialize() earlier to show loading progress sooner.

Choose a model

The default is SmolLM2-360M-Instruct-q4f32_1-MLC, a deliberately small model that needs roughly 580 MB of VRAM. Model selection is configurable:

import { RECOMMENDED_MODELS, WebLLMProvider } from '@feltdb/webllm';

const llm = new WebLLMProvider({
  model: RECOMMENDED_MODELS.balanced,
});

• smallest: the default 360M model
• balanced: SmolLM2 1.7B for stronger generation on capable devices
• lowMemoryWithShaderF16: a smaller 360M quantization for devices supporting the WebGPU shader-f16 feature

Pass any model ID from WebLLM's prebuilt catalog, or call getAvailableModels() to inspect the installed catalog.

FeltDB application generation

Contract-aware flow proposals

generateFlow() grounds a proposal in the deterministic, secret-free contract snapshot shared by Core, the CLI, and Studio. It preserves generic generate() and stream() behavior and never writes files or publishes an application.

const llm = new WebLLMProvider({ contract: snapshot });
const proposal = await llm.generateFlow(
  'Add invitations for organization owners.',
  { currentFlow },
);

The proposal carries the base contract and flow hashes. The CLI refuses stale proposals, parses and semantically validates the proposed DSL, highlights authorization changes, and requires human approval before synchronizing feltdb.flow and the generated integration source. The snapshot contains schema and application definitions, not application records, credentials, connections, or environment configuration.

WebGPU is required only for local inference. Projects that decline the optional WebLLM dependency retain the complete non-AI FeltDB experience.

WebLLMProvider implements FeltDB's message-generation provider contract:

import { ApplicationFactory } from '@feltdb/ai';
import { WebLLMProvider } from '@feltdb/webllm';

const factory = new ApplicationFactory({
  provider: new WebLLMProvider(),
});

const application = await factory.generateApplication(
  'A collaborative, offline-first field inspection app',
);

@feltdb/ai is currently an experimental repository workspace. This package is independently usable through its generate() and stream() APIs.

Workers and lifecycle

Inference runs in the bundled Web Worker by default so execution does not block the UI. Set useWorker: false for main-thread execution, or pass an application-owned worker for custom hosting or CSP behavior.

Call interrupt() to stop generation and shutdown() to unload the model and terminate a package-owned worker.

Privacy and deployment

The model executes locally. Hosting and analytics code can still transmit data, and model files must be fetched before they are cached, so review your app's network and content-security policies. This package does not send prompts to FeltDB or an inference provider.

MIT licensed.