aic-sdk-wasm - WebAssembly Bindings for ai-coustics SDK

WebAssembly wrapper for the ai-coustics Speech Enhancement SDK. This package allows you to run ai-coustics models directly in the browser.

For comprehensive documentation, visit docs.ai-coustics.com.

[!NOTE] This SDK requires a license key. Generate your key at developers.ai-coustics.io.

Installation

npm install @ai-coustics/aic-sdk-wasm

The package is built with WebAssembly SIMD (simd128) and needs a runtime that supports it.

Quick Start

import init, { Model, Processor } from "@ai-coustics/aic-sdk-wasm";

// Initialize the WASM module
await init();

// Load a model
const response = await fetch("https://artifacts.ai-coustics.io/models/quail-vf-2-3-s-16khz/v8/quail_vf_2_3_s_16khz_398hl32w_v12.aicmodel");
const modelBytes = new Uint8Array(await response.arrayBuffer());
const model = Model.fromBytes(modelBytes);

// Get optimal configuration
const sampleRate = model.getOptimalSampleRate();
const blockSize = model.getOptimalBlockSize(sampleRate);

// Create and initialize processor
const processor = new Processor(model, "YOUR_LICENSE_KEY");
processor.initialize(sampleRate, blockSize, false);

const audioBuffer = new Float32Array(blockSize);
// Process a mono audio block in place
processor.process(audioBuffer);

Usage

SDK Information

import { getVersion, getCompatibleModelVersion } from "@ai-coustics/aic-sdk-wasm";

// Get SDK version
console.log(`SDK version: ${getVersion()}`);

// Get compatible model version
console.log(`Compatible model version: ${getCompatibleModelVersion()}`);

Loading Models

Download models and find available IDs at artifacts.ai-coustics.io.

Since browsers cannot access the file system directly, you must load the model file as a byte array (Uint8Array).

// Fetch the .aicmodel file
const response = await fetch("https://artifacts.ai-coustics.io/models/quail-vf-2-3-s-16khz/v8/quail_vf_2_3_s_16khz_398hl32w_v12.aicmodel");
const bytes = new Uint8Array(await response.arrayBuffer());

// Create Model instance
const model = Model.fromBytes(bytes);

Model Information

// Get model ID
const modelId = model.getId();

// Get optimal sample rate for the model
const optimalRate = model.getOptimalSampleRate();

// Get the optimal block size for a specific sample rate
const optimalBlockSize = model.getOptimalBlockSize(48000);

Configuring the Processor

The Processor accepts enhancement and bypass models only. Dedicated VAD models go through the Vad class and analysis models through Analyzer; passing the wrong type throws ModelTypeUnsupported.

// Create processor
const processor = new Processor(model, "YOUR_LICENSE_KEY");

// Initialize with audio settings
processor.initialize(
  sampleRate,           // Sample rate in Hz (8000 - 192000)
  blockSize,            // Samples per processing call
  variableBlockSize     // Allow shorter blocks than blockSize (default: false)
);

Processing Audio

The processor is mono. To process multichannel audio, mix it down to mono, or create one processor per channel.

const buffer = new Float32Array(blockSize);
// Process a mono audio block in place
processor.process(buffer);

Processor Context

import { ProcessorParameter } from "@ai-coustics/aic-sdk-wasm";

// Get processor context
const procCtx = processor.getContext();

// Get the delay applied to the audio, in samples
const delay = procCtx.getAudioDelay();

// Reset processor state (clears internal buffers)
procCtx.reset();

// Set enhancement parameters
procCtx.setParameter(ProcessorParameter.EnhancementLevel, 0.8);
procCtx.setParameter(ProcessorParameter.Bypass, 0.0);

// Get parameter values
const level = procCtx.getParameter(ProcessorParameter.EnhancementLevel);
console.log(`Enhancement level: ${level}`);

Voice Activity Detection (VAD)

Voice activity detection is a first-class object. It runs a dedicated VAD model through the Vad class and is driven explicitly, independently of any processor. new Vad(...) throws ModelTypeUnsupported for anything that is not a VAD model.

import { Vad, VadParameter } from "@ai-coustics/aic-sdk-wasm";

// Load a dedicated VAD model and create a Vad from it
const vadResponse = await fetch("https://artifacts.ai-coustics.io/models/vad-ms-2-1-xxs-16khz/v8/vad_ms_2_1_xxs_16khz_mw7jdprk_v36.aicmodel");
const vadModel = Model.fromBytes(new Uint8Array(await vadResponse.arrayBuffer()));

const vad = new Vad(vadModel, "YOUR_LICENSE_KEY");
const vadSampleRate = vadModel.getOptimalSampleRate();
const vadBlockSize = vadModel.getOptimalBlockSize(vadSampleRate);
vad.initialize(vadSampleRate, vadBlockSize, false);

const vadCtx = vad.getContext();

// Configure VAD parameters
vadCtx.setParameter(VadParameter.Sensitivity, 0.8); // probability threshold, 0.0 - 1.0
vadCtx.setParameter(VadParameter.SpeechHoldDuration, 0.03);
vadCtx.setParameter(VadParameter.MinimumSpeechDuration, 0.0);

// Get parameter values
const sensitivity = vadCtx.getParameter(VadParameter.Sensitivity);
console.log(`VAD sensitivity: ${sensitivity}`);

// The VAD is driven explicitly and does not modify the audio
vad.process(block);

if (vadCtx.isSpeechDetected()) {
  console.log("Speech detected!");
}

// Raw model prediction, without speech hold / thresholding post-processing
console.log(vadCtx.getRawVadProbability());

Run the VAD on the original audio

If you use enhancement and VAD together, feed the VAD the original input audio, not the processor's enhanced output. Run the two objects side by side on the same mono block rather than chaining them:

vad.process(block);       // reads the block, does not modify it
processor.process(block); // enhances the block in place

Enhancement is designed to change the signal, so running the VAD on its output means detecting speech in audio that no longer matches what the VAD model expects. It also stacks the processor's delay on top of the VAD's own prediction delay, which makes speech decisions harder to align.

Delays

The processor delays audio, the VAD does not, so the two queries report different things and are independent of each other:

const audioDelay = procCtx.getAudioDelay();          // enhanced audio lags the input by this much
const predictionDelay = vadCtx.getPredictionDelay(); // the VAD prediction lags the same input by this much

getPredictionDelay() is not applied to the audio — vad.process leaves the buffer untouched. Use it to line speech decisions up with the audio timeline.

// Clear VAD state on a stream discontinuity
vadCtx.reset();

// Refresh a JWT-form license without reconstructing the VAD
// vadCtx.updateBearerToken(newJwt);

Energy-based Voice Activity Detection

An enhancement processor can report speech activity without running a separate VAD model. Create a context before processing and read its prediction after each audio block:

const energyVadCtx = processor.getEnergyVadContext();
energyVadCtx.setParameter(VadParameter.Sensitivity, 6.0);

processor.process(block);
const speechDetected = energyVadCtx.isSpeechDetected();

EnergyVadContext also provides getParameter, getPredictionDelay, and reset. It can outlive the processor. Resetting it keeps its settings and does not reset enhancement; resetting the processor also resets its energy VAD. Sensitivity ranges from 1 to 15, with higher values detecting quieter signals. Creating a context keeps inference active even while bypassed or fully dry.

Analysis

For analysis-only models (e.g. Tyto) use the Analyzer class instead of Processor. The analyzer buffers audio in the audio thread via buffer and runs the analysis model on demand via analyze().

import { Analyzer } from "@ai-coustics/aic-sdk-wasm";

// Load an analysis model and create an Analyzer from it
const analysisResponse = await fetch("https://artifacts.ai-coustics.io/models/tyto-1-2-l-16khz/v8/tyto_1_2_l_16khz_ef8fy2t4_v65.aicmodel");
const analysisModel = Model.fromBytes(new Uint8Array(await analysisResponse.arrayBuffer()));

const sampleRate = analysisModel.getOptimalSampleRate();
const blockSize = analysisModel.getOptimalBlockSize(sampleRate);

const analyzer = new Analyzer(analysisModel, "YOUR_LICENSE_KEY");
analyzer.initialize(sampleRate, blockSize, false);

// From the audio path:
analyzer.buffer(audioChunk);

// From any context:
const result = analyzer.analyze();
console.log(result.riskScore, result.speakerReverb, result.speakerLoudness,
            result.interferingSpeech, result.noise, result.codecDegradation,
            result.packetLoss);
result.free();

// Optional: clear buffered audio on stream discontinuity.
analyzer.reset();

// Optional: refresh a JWT-form license without reconstructing the analyzer.
// analyzer.updateBearerToken(newJwt);

analyzer.free();
analysisModel.free();

Each class accepts exactly one family of models, and throws ModelTypeUnsupported for the rest:

Memory Management

WebAssembly memory should be manually freed when objects are no longer needed. This does not need to be done if the browser supports weak references.

// When finished with the VAD
vadCtx.free();
vad.free();

// When finished with the processor
processor.free();

// When finished with the model (and you won't create new objects from it)
model.free();

Ending the telemetry session early

A telemetry session is stopped automatically when the owning object is freed. Because JavaScript object deallocation is up to the garbage collector and can be delayed arbitrarily, Processor, Vad and Analyzer also expose an explicit terminateSession() for lifecycle events. After it has been handled, the object is no longer allowed to process audio.

processor.terminateSession();
vad.terminateSession();
analyzer.terminateSession();

Examples

See the examples/ directory for complete working examples, including:

• index.html: Full interactive demo with file inputs for an enhancement model, a VAD model and an analysis model, parameter controls, and real-time VAD visualization.
• benchmark.html: Performance benchmark tool to measure processing latency.
• basic.html: Demonstrates all core SDK APIs.

Documentation

• Full Documentation: docs.ai-coustics.com
• Available Models: artifacts.ai-coustics.io