@degreesign/voice

Turn speech into text and text into speech, privately, on the user's own device. Use it for dictation, voice notes, transcription, speech to text, text to speech and TTS, with no cloud service, account or network connection after setup.

Try the live demo to run speech to text and text to speech in the browser, with no install.

Table of contents

• Why
• Install
• Quick start
• Speech to text
• Text to speech
• Voice cloning
• Dictation
• Demo
• Models
• Storage and performance
• Architecture
• Licensing

Why

• Offline: weights download once, then every decode runs on the machine.
• Streaming: transcribe returns a live handle and keeps updating while audio arrives.
• One API: the same entry autodetects Node.js and the browser.
• Any audio: 16 kHz mono PCM, a Blob or File, a URL, a filesystem path, or the browser microphone.
• Typed: TypeScript declarations cover every export.
• Two directions: speech to text and text to speech.
• Self contained: the cache, storage and audio decoders are bundled at build time.

DoListy uses the package for task dictation and voice notes.

Install

npm install @degreesign/voice

yarn add @degreesign/voice

pnpm add @degreesign/voice

Node.js 24 or newer is required. The main entry exports recognition and the voice library. The dictation controller has its own entry.

The published files are:

• dist/node/degreesign.node.min.js, the CommonJS entry for Node.
• dist/browser/degreesign.min.js, the full UMD entry as window.dsVoice.
• dist/browser/whisper.min.js, the Whisper only UMD entry as window.dsVoiceWhisper.
• dist/browser/dictation.min.js, the dictation UMD entry.
• dist/browser/dictation.worker.js, the prebuilt dictation worker.

Build from source with yarn build and typecheck with yarn typecheck.

CDN

<script src="https://cdn.jsdelivr.net/npm/@degreesign/voice@latest/dist/browser/degreesign.min.js"></script>

Try the live speech demo before installing. It runs speech to text and text to speech in the browser.

Quick start

Browser microphone

<script src="https://cdn.jsdelivr.net/npm/@degreesign/voice@latest/dist/browser/degreesign.min.js"></script>
<script>
  const { Language, transcribe } = window.dsVoice;
  const session = transcribe({ language: Language.auto });
  const timer = setInterval(() => {
    output.textContent = session.fullText().text;
  }, 500);
  stopButton.onclick = async () => {
    clearInterval(timer);
    const result = await session.stop();
    output.textContent = result.text;
  };
</script>

A browser session with no pcm, no audioFilePath and no manual feed asks for microphone access and records until stop.

PCM input

The input is 16 kHz mono Float32Array PCM.

import { Language, transcribe } from "@degreesign/voice";

const session = transcribe({ pcm, language: Language.auto });
console.log(session.fullText().text);

const result = await session.done;
console.log(result.text, result.duration, result.meta.rtf);

To warm up the model first, call preloadVoice. To feed audio as it arrives, pass microphone: false and append samples with push.

Speech to text

transcribe(options?) returns a TranscribeSession immediately.

Options

Model loading

import {
    Language,
    ModelId,
    Quality,
    isModelCached,
    loadModel,
    preloadVoice,
} from "@degreesign/voice";

await preloadVoice({
    model: ModelId.WhisperBase,
    language: Language.en,
    quality: Quality.Low,
});

const cached = await isModelCached({
    model: ModelId.WhisperBase,
    language: Language.en,
    quality: Quality.Low,
});
const model = await loadModel({
    model: ModelId.WhisperBase,
    language: Language.en,
    quality: Quality.Low,
});

preloadVoice downloads and loads a model ahead of time. loadModel returns the loaded model. isModelCached checks the store without using the network. An explicit Language.en selects the English only checkpoint; every other language, including Language.auto, uses the multilingual checkpoint.

Audio and timing

The first source that is present wins: pcm, then audioFilePath, then a manual feed with microphone: false, then the browser microphone.

In Node, paths are read from the filesystem and decoded as WAV, FLAC or MP3. In the browser, a URL is fetched, a Blob or File is read directly, and decoding uses the Web Audio API. Supported browser formats depend on the browser.

Audio below -50 dBFS is treated as silence and returns empty text without touching the model. Audible input shorter than one second is centred in silence, so a clipped word keeps context on both sides.

A session commits pending audio at a pause, after 12 seconds without one, or when stopped. Committed text no longer changes and its audio is released, so memory stays bounded during long recordings. msElapsed and meta.rtf cover decode work only, not download, load, audio decoding or waiting for audio.

Weights mirror

Set weightsBaseUrl to serve checkpoints from your own host:

await preloadVoice({
    weightsBaseUrl: "https://models.example.com",
});

The mirror keeps the Hugging Face layout: <base>/<repo>/resolve/main/<file>. Whisper reads GGUF weights from the handy-computer repositories and text files from the openai repositories. Support Range requests to enable resumable, parallel downloads.

Text to speech

The voice library has two 48 kHz voices. Both use MOSS-TTS-Nano 100M and clone bundled reference codes, so no reference recording is downloaded. Each voice takes about 133 MB on first use. A 16 kHz preview clip is already bundled.

The text itself selects one of the 20 MOSS languages: Arabic, Chinese, Czech, Danish, English, French, German, Greek, Hebrew, Hungarian, Italian, Japanese, Korean, Persian, Polish, Portuguese, Russian, Spanish, Swedish and Turkish.

import {
    VoiceId,
    getVoiceSample,
    isVoiceCached,
    listVoices,
    loadVoice,
    removeVoice,
    synthesize,
} from "@degreesign/voice";

const voices = listVoices();
const sample = getVoiceSample(VoiceId.Female);
console.log(voices.map((voice) => voice.label), sample.sampleRate);

if (!(await isVoiceCached(VoiceId.Female))) {
    await loadVoice({
        voice: VoiceId.Female,
        onProgress: ({ phase, ratio }) => console.log(phase, ratio),
    });
}

const audio = await synthesize({
    voice: VoiceId.Female,
    text: "Hello from @degreesign/voice.",
});
// audio.pcm is mono Float32 at audio.sampleRate.

await removeVoice(VoiceId.Female);

synthesize accepts optional pitch in semitones, rate and linear gain. Pitch and rate apply after synthesis for every voice. The MOSS voices also accept seed and temperature. Gain defaults to two, pitch to zero and rate to one. The parsed weights stay resident until another voice replaces them.

Voice cloning

cloneVoice records the browser microphone for three seconds and encodes the recording with the MOSS audio tokenizer. The first call downloads the 45 MB encoder, then keeps it resident.

import { cloneVoice, synthesize, VoiceId } from "@degreesign/voice";

const clone = await cloneVoice({ durationMs: 3000 });
const audio = await synthesize({
    voice: VoiceId.Female,
    text: "This is the recorded voice.",
    reference: clone.codes,
});

A VoiceClone holds codes, the frame count and the native 48 kHz rate. Store the codes on the page to reuse the same voice without recording again:

const audio = await synthesize({
    voice: VoiceId.Male,
    text: "The stored reference still speaks.",
    reference: clone.codes,
});

Pass pcm with sampleRate to encode a recording you already have. The encoder resamples to 48 kHz and keeps at most four seconds. A reference shorter than one second is rejected. onProgress follows the encoder download.

Dictation

@degreesign/voice/dictation is a headless controller for live microphone dictation. It owns the worker, capture, pause state, model progress and partial text. The page keeps its own markup and copy.

Copy the prebuilt worker next to the page, or serve it from a same origin path:

cp node_modules/@degreesign/voice/dist/browser/dictation.worker.js \
  public/dictation.worker.js

import {
    createDictation,
    DictationState,
    Language,
} from "@degreesign/voice/dictation";

const dictation = createDictation({
    workerUrl: "/dictation.worker.js",
    language: Language.en,
    onState: (state, { modelReady, ratio }) => {
        status.textContent = state === DictationState.recording && modelReady
            ? "Listening"
            : `${state} ${Math.round(ratio * 100)}%`;
    },
    onText: (text) => {
        input.value = text;
    },
    onError: ({ source, message }) => console.error(source, message),
});

micButton.onclick = () => void dictation.start();
pauseButton.onclick = () => dictation.togglePause();
doneButton.onclick = () => void dictation.finish();
cancelButton.onclick = () => dictation.cancel();

Main options are workerUrl or workerFactory, model, language, quality, threads, timestamps, updateIntervalMs, partialIntervalMs, onState, onText, onFrequency, onError and an advanced capture override. One controller belongs to one input or component. Controllers on a page share one worker, and starting one cancels the active one. A cancel emits onText("", true) so the page can restore its base text.

The states are idle, requesting, recording, paused, loading and working. The controller methods are start, pause, resume, togglePause, finish, cancel, reset, active, state and destroy.

Demo

Open the live demo to try both directions in a browser before installing. It loads @degreesign/voice from jsDelivr and lets you:

• Record from the microphone or pick an audio file and read the transcript.
• Pick a voice, tune pitch, rate, gain, temperature and seed, then listen.
• Record a reference and speak the text with the cloned voice.

The demo for speech to text uses Quality.Low, a 59 MB download. The demo source is docs/index.html. Run node test/example/serve.cjs, then open http://127.0.0.1:17544/docs/index.html to serve it locally.

Models

Whisper base uses handy-computer/whisper-base-gguf with the openai/whisper-base text files. The English checkpoint uses handy-computer/whisper-base.en-gguf with openai/whisper-base.en. Both share one ModelId, one usage record and one meta.model.

Weights are published as F16, Q8_0 and Q4_K_M. Quality.High is the default.

Quantized files are dequantized to F32 on first access. The kernels are the same at every quality, so quality changes download size and first access cost, not the steady state decode path.

Storage and performance

Node.js stores model files through @degreesign/cache under ~/.cache/decoder. Browsers store them in IndexedDB through @degreesign/storage.

Weights are stored as 4 MiB chunks with a manifest, text files as JSON, and usage.json tracks size and last use for the disk budget. A download uses four parallel Range requests and resumes from stored chunks. A failed chunk is retried with backoff before the whole file falls back to one stream.

allowedDiskSpaceMB caps stored model bytes and defaults to 512. A model that is already cached is never pruned.

The graph runs on the CPU with WebAssembly SIMD kernels. threads defaults to ThreadCount.Four. Browsers need cross origin isolation with Cross-Origin-Opener-Policy: same-origin and Cross-Origin-Embedder-Policy: require-corp to use the worker pool; without it the engine uses the single thread path. The F32 tensor cache holds the checkpoint once by default; weightCacheMB lowers the cap.

Architecture

The committed map at docs/architecture.html shows the files and functions as bands, from the public entry through the model family, the tensor runtime, the platform stores and the audio input. It is one offline HTML file. Open it directly and use the panel to trace a call.

Licensing

The runtime code in this repository is MIT licensed. Whisper weights and token data are Apache-2.0. The MOSS TTS weights, tokenizer and reference codes are Apache-2.0. The bundled MP3 decoder is CC0 and the minimp3-wasm bindings are Apache-2.0. See THIRD_PARTIES.md and LICENSE.