slicc-scrub

One scrubber for every place a SLICC secret could slip into a model's context: the vault's forward responses and model requests, the kernel's exec children, and slicc-agent's tool output. It replaces a secret's value, in every encoding below, before the text moves on. The design is §3.15 of secrets-tokens-design.md (revision 4) in slicc-agent-plan; this README is the spec of what's built.

• Zero runtime dependencies, no DOM and no Node APIs, no eval and no generated code. It runs as-is in a browser frame, a worker, Node and the kernel.
• Deterministic: the same values and input always give the same output.
• One set of vectors, vectors.json, which every consumer runs against its copy.

import { compile } from '@ai-ecoverse/slicc-scrub';

const scrubber = compile([{ name: 'GH_TOKEN', value: 'ghp_…', placeholder: 'fnox_proxy_…' }], { version: 3 });
scrubber.text('token: ghp_…');

API

hits is [{ name, form, count }]: how often each secret was found in each form (raw, fragment, json, percent, html, base64, hex, wrapped). It never contains a value.

The forms

Each value v is matched, as UTF-8 bytes, in these forms. A form is added only when it differs from the ones before it.

Matching

• One Aho–Corasick automaton over every pattern, on bytes. Fragments are found from their 16-byte windows and extended byte by byte along the value. A second automaton over the base64 and hex patterns runs on a view of the input with CR and LF removed, mapped back to the original bytes.
• Leftmost, then longest: at each point the match that starts first wins, and among those the longest. Ties go to raw, then fragment, json, percent, html, base64, hex, then the earlier secret. Matches never overlap, and replaced text is never scanned again.
• Replacements: a raw match (a fragment covering the whole value is one) becomes the secret's placeholder; anything else becomes [redacted:<name>].
• sameLength: true replaces every match, in every form, with exactly as many bytes as it matched: each byte becomes * (0x2A), except CR and LF, which stay where they are. The filler is that one fixed byte, nothing taken from the value, the placeholder or the name, so the output has the input's length and line breaks, and length-prefixed framing around a match (an HTTP Content-Length, a chunk size, a WebSocket frame) still holds. Matching, the forms, hits and streaming are the same as without it; only the replacement differs.
• Matching is case-sensitive; hex is matched in lower and upper case.

Streaming

stream() returns { push(Uint8Array) → Uint8Array, idle() → Uint8Array, end() → Uint8Array, hits }.

• push returns every byte that can no longer be part of a match, scrubbed, and holds back the rest: from the earliest point where the input so far could still become a match (a proper prefix of a pattern, a fragment still growing along its value, or a wrapped pattern across CR and LF). A match split across any chunking is therefore found exactly as in one piece; for every input, any chunking followed by end() gives the same bytes as bytes().
• end scrubs and returns everything held.
• idle is for the caller to call after idleMs without input, so an interactive prompt can't stall. It releases the held bytes, scrubbed, only when fewer than 16 are held, and otherwise returns nothing. A held tail of 16 bytes or more could be the start of a longer match.
• hits totals the stream so far.

Vectors

vectors.json (@ai-ecoverse/slicc-scrub/vectors.json) is { format: 1, fragment, minValue, idleMs, cases }. Each case is { name, secrets, input, output, hits }, or with inputHex and outputHex for bytes that aren't UTF-8. scripts/vectors.mjs writes it from how each input is built (Node's own base64, hex and URL encoders), never from the scrubber.

A consumer's test runs, for every case:

1. bytes(input) (and text where the case is text) gives output and hits, in any order;
2. for every split point, push(first), push(rest), end() gives output;
3. one byte at a time gives output.

Develop

npm install
npm run build    # dist/index.js and dist/types
npm run lint
npm test         # vectors.json is fresh, the vectors at every split point, the API, a fuzz test, and CI
npm run vectors  # rewrite vectors.json

Unit tests stay in the gitignored test/unit/ (npm run test:unit), and the pre-commit hook requires them to cover every changed line of src/. The Biome, TypeScript, lefthook and Renovate configuration comes from slicc-shared-web.

Release

.github/workflows/ci.yml is the only workflow: one caller of slicc-shared-web's pipeline, at the same version as the @ai-ecoverse/slicc-shared-web devDependency. A pull request gets lint, npm test (test/integration/*.test.mjs, on one shard, as ci / integration) and the Claude and Codex reviews (ci / review). A push to main runs semantic-release, which tags a version from the conventional commits and publishes @ai-ecoverse/slicc-scrub to npm with provenance, through trusted publishing from ci.yml. There is no other way to release. test/integration/workflows.test.mjs keeps it so.