@brainchop/mindgrab

Browser MeshNet brain segmentation on WebGPU, with WebGL2 and threaded-CPU fallbacks: MindGrab skull stripping, 18-class parcellation at two widths (16chan18cls, mindmap), and a 104-class Desikan-Killiany parcellation (mindsnap), over a fixed 256³ grid. A NIfTI ArrayBuffer goes in and a NIfTI ArrayBuffer comes out.

The whole pipeline — conform, normalise, the 13 or 25 MeshNet layers, classify, largest-component, reslice — runs inside a wasm module compiled from brainchop-c. This package owns only what that module deliberately does not: gzip, WebGPU device and WebGL2 context acquisition, the cross-origin-isolation probe, and the decision to refuse.

import { segment } from '@brainchop/mindgrab'

const input = await file.arrayBuffer()          // .nii or .nii.gz
const { image, elapsedMs } = await segment(input, { model: 'mindgrab' })
// image is gzipped if the input was

MindMap now filters selected tissue classes and refills suppressed speckles from surviving neighboring labels by default. Use { model: 'mindmap', legacyCleanup: true } for the original whole-brain-only cleanup. This requires wasm assets rebuilt from the same revision as the wrapper.

Categorical native-space output uses the browser's eight-sample majority vote, including background votes outside the source and first-sample tie breaking. Masks still use nearest neighbor; continuous tissue maps use linear resampling. Invalid, empty, more than 95% filled, or excessively fragmented categorical results fail with inference-failed before cleanup can disguise the failure.

Continuous tissue maps

import { segmentTissues } from '@brainchop/mindgrab'

const { tissues, backend } = await segmentTissues(input, {
  model: 'mindmap', worker: true,
})
// tissues.gm, tissues.wm, tissues.csf and tissues.brain (GM + WM) are NIfTI
// ArrayBuffers. Native geometry and input gzip encoding are preserved by default.

tissues.brain's 0.5 isosurface is a sub-voxel pial surface, ready to mesh.

segmentTissues supports mindmap and 16chan18cls on WebGPU, WebGL2 and CPU. The GPU paths compute grouped priors, blur and support gating on the device; the shared C CAT-lite fitter runs on CPU. Fitting uses the MindMap-calibrated constants for both models. Model16 coverage checks implementation agreement, not tissue-model calibration. saveConform and gzipOutput work as for segment; mask, border and legacy categorical cleanup options are refused. The WebGL2 module reserves a 768 MiB wasm heap for the host-side tissue fit.

Backends

auto (the default) tries hardware webgpu, hardware webgl2, then cpu (cross-origin isolated pages only), reports which one ran in result.backend, and throws no-webgpu if none can. It never picks a renderer identified as software (SwiftShader, llvmpipe, lavapipe, WARP, or a WebGPU fallback adapter): they pass every capability probe but ran MindGrab in 990 s against the CPU module's ~10 s. checkSupport() and checkWebgl2Support() report software: true for them. A masked WebGL2 renderer string cannot be identified (software: undefined), so it runs only when cpu cannot. Whatever runs, an empty MindGrab mask rejects with inference-failed, as empty labels do. A named backend still runs a software one, but outlives the default timeoutMs.

MindGrab on an M4 Pro, as time inside the wasm module and as what a segment() call actually costs. The difference is this package's own work — fetching and instantiating the module, gzip and gunzip, and copying the volume into and out of the module's filesystem — not the segmentation, which is wholly inside the first number:

The CPU row's 2.6 s of wrapper time is about 3× the GPU rows' 0.7–0.8 s, which is not an arithmetic slip: it is a separate and larger module, and instantiating it spins up a worker per core.

WebGL2 is what Linux Firefox and Chrome take today, since neither enables WebGPU by default; those stacks are known to run correctly but have never been timed, so the figures above are macOS. Naming a backend explicitly makes an unsupported choice an error rather than a silent downgrade. The CPU module is the same fp32 C the command-line executables run and is bit-exact with them; the two GPU paths are fp16 and carry the voxel budgets under Accuracy.

The CPU fallback needs two headers

It is a pthreads build, so it imports shared memory, and a page that is not cross-origin isolated cannot instantiate it — the module hangs rather than failing, which is why this is checked before it is fetched. Serve the top-level document with:

Cross-Origin-Opener-Policy: same-origin
Cross-Origin-Embedder-Policy: require-corp

Putting them on the .wasm response instead does nothing: isolation is a property of the document. Note the cost, which is the usual reason a deployment turns isolation back off again: under require-corp every cross-origin subresource the page loads — images, fonts, analytics, embeds — must itself send Cross-Origin-Resource-Policy or proper CORS, or it will be blocked. Then:

import { checkCpuSupport } from '@brainchop/mindgrab'

const cpu = checkCpuSupport()
if (!cpu.supported) console.warn(cpu.reasons.join('\n'))

Playwright's patched Firefox Nightly runs this module about 20× slower than Chromium, but installed Firefox 153.0.1 ran the same bit-exact model16 case in 7.6 s. The Nightly result is retained as a harness/compiler anomaly, not a warning about Firefox users generally.

Requirements

For the WebGPU path: WebGPU with the shader-f16 feature, maxStorageBufferBindingSize and maxBufferSize of at least 512 MiB (one f16 activation buffer), and a secure context — https or localhost. about:blank and file:// are not secure contexts, and there navigator.gpu is undefined, which looks exactly like a browser with no WebGPU at all.

Ask before you offer the user a button:

import { checkSupport } from '@brainchop/mindgrab'

const report = await checkSupport()
if (!report.supported) console.warn(report.reasons.join('\n'))

Models

Each is a separate wasm module with its weights compiled in, fetched only when you ask for it. Options are per-model and the mismatched ones are refused before anything is downloaded, because an executable never advertises what it would decline:

await segment(buf, { model: 'mindgrab', saveConform: true })
// BrainchopError: mindgrab does not support `saveConform`: its output is the
// input image with non-brain voxels floored, so it is inherently in the input space

Each MODELS entry (and listModels()) carries colormap: { R, G, B, labels }, one entry per output value, index 0 background (for mindgrab, its mask). It is the model's own colormap.json, in the shape viewers such as NiiVue take directly: nv.setColormapLabel(1, MODELS.mindsnap.colormap).

Options

segment(input: ArrayBuffer | ArrayBufferView, options: SegmentOptions)

Output is in the input image's own space by default, for every model. This inverts brainchop-cli, where the conformed grid is the default.

--crop and --export-classes, which the native executables offer, are deliberately absent. Both force the CPU engine inside the GPU modules, which is compiled without threads there and takes 141–266 s per volume; accepting them would turn a two-second call into a four-minute one with no warning. The cpu backend is a different module and does not change this.

Bundlers

No copying and no assetPath: the worker and each model's glue are imported with literal new URL('./worker.js', import.meta.url) and import('./<glue>.js') specifiers, and each glue locates its .wasm the same way, so Vite, Rollup and webpack emit all of them (hashed) and a caller still downloads only the model and backend it runs. Vite needs two settings: worker.format: 'es' for vite build (the default iife cannot hold the worker's top-level await), and optimizeDeps.exclude for vite dev (the dev pre-bundler moves the package into .vite/deps/, where the worker URL 404s):

// vite.config.js
export default {
  optimizeDeps: { exclude: ['@brainchop/mindgrab'] },
  worker: { format: 'es' },
}

A bundler emits every module it can see, so @brainchop/mindgrab deploys all twelve (~10 MB). A host that runs one model imports that model's entry instead, which has the same API and deploys only its three modules (MindGrab: ~2.6 MB under vite build):

import { segment } from '@brainchop/mindgrab/mindgrab'   // or /16chan18cls, /mindmap, /mindsnap

listModels() there returns that one model (MODELS still describes all four), and any other model is refused with unsupported-option (unless assetPath is set, which loads by URL).

esbuild and webpack also follow the imports but do not stub the Node-only node:* imports that the CPU modules carry (they also run under Node): mark them external (esbuild --external:'node:*', webpack externals: /^node:/). esbuild does not emit new URL() assets, so an esbuild host also sets assetPath.

assetPath remains for hosts that serve dist/ themselves: copy the worker and the model's .js and .wasm somewhere served and point assetPath at it. It must be same-origin; it feeds a dynamic import().

Errors

Every package error throws a BrainchopError with a code you can branch on: no-webgpu, no-adapter, device-too-small, no-f16, unsupported-option, bad-input, initialization-failed, inference-failed. initialization-failed means the module never started, so retrying can be meaningful; inference-failed means it started and did not finish. There is no fallback behind any of them, by decision — a silently downgraded segmentation is worse than a refusal. auto choosing between backends is not an exception: it happens before anything is fetched and only when you expressed no preference.

no-webgpu is also what a cpu refusal reports — "this page is not cross-origin isolated" arrives under that code rather than a fourth one, so read the message, or call checkCpuSupport() first.

Accuracy

The cpu backend is held to zero differing voxels against frozen CPU references. The model16 native-space majority fixture has a separate wasm reference: one pre-existing native/wasm conformed label difference is sampled by the new vote. It was reproduced from unmodified merged main and the new fixture independently verified against the browser voting function. See reference provenance. The two GPU paths are fp16, held to the same per-voxel budgets as the project's Metal backend rather than to bit-exactness. Historical nearest-neighbor results on the project's fixtures, as voxels of 16.7M (2 mm case, 1.6M):

The current majority-vote checks retain these budgets. Tissue maps retain the existing 2/255 RMSE gate over the nonzero union and exact NIfTI geometry.

Building

npm install
npm run build     # build all three backends' modules, stage into dist/, types, bundle
npm test          # drives the built package in a real browser

The test runs headed — headless Chromium on macOS has no GPU — and needs a checkout with playwright installed (BC_PLAYWRIGHT_ROOT).