@browserless.io/anti-bot-classifier

A pure, dependency-free classifier that decides whether a page load was blocked or challenged by an anti-bot system (Cloudflare, DataDome, Imperva, PerimeterX, DDoS-Guard, or a generic block page).

The package is usable both offline (batch-scoring recorded page snapshots) and inside a runtime service (classifying live sessions), so that "block rate" means the same thing in both places. There is no I/O, no browser or DOM globals, and no non-determinism (no Date, no Math.random) anywhere in the classification path — the same Observation always produces the same Classification.

The Observation → Classification contract

classify(observation, ruleset) takes an Observation — a privacy-scrubbed snapshot of a page — and a Ruleset, and returns a Classification.

Observation channels

All fields are optional; supply whatever you were able to capture.

Callers are responsible for the privacy scrub before constructing an Observation: strip query strings, drop cookie values (names only), and truncate markup/frame markup to whatever the caller is comfortable retaining.

Classification fields

inconclusive covers cases where the classifier can't tell — a browser-level network error (net::ERR_CONNECTION_RESET-style strings), no content at all was captured, or the page was sampled but came back empty. Treat inconclusive rows as "exclude from scoring", not as clear.

Usage

classify

import { classify, DEFAULT_RULESET } from '@browserless.io/anti-bot-classifier';

const result = classify(
  {
    host: 'example.com',
    status: 403,
    headers: { 'cf-mitigated': 'challenge' },
  },
  DEFAULT_RULESET,
);

// result.status === 'blocked'
// result.vendor === 'cloudflare'
// result.kind === 'challenge'

detectBlock (DOM-only convenience shim)

For call sites that only have title / text / markup (no headers, cookies, status, or final URL) — this is a drop-in replacement for the legacy detectBlock, classifying against DEFAULT_RULESET under the hood:

import { detectBlock } from '@browserless.io/anti-bot-classifier';

const result = detectBlock(
  document.title,
  document.body.innerText,
  document.documentElement.outerHTML,
);

// result: { status, blocked, inconclusive, reason }

Overriding or extending the ruleset

classify takes the ruleset as an explicit argument — there's no hidden global state. To customize detection, either build a Ruleset from scratch, or spread DEFAULT_RULESET and add or replace signals:

import {
  classify,
  DEFAULT_RULESET,
  type Observation,
  type Ruleset,
} from '@browserless.io/anti-bot-classifier';

const myRuleset: Ruleset = {
  ...DEFAULT_RULESET,
  signals: [
    ...DEFAULT_RULESET.signals,
    {
      vendor: 'acme-waf',
      channel: 'header',
      key: 'x-acme-block',
      marker: 'B',
      kind: 'block',
      weight: 0.9,
      note: 'x-acme-block header present',
    },
  ],
};

const observation: Observation = {
  status: 200,
  headers: { 'x-acme-block': '1' },
};

classify(observation, myRuleset);

A Ruleset is { signals, structural?, networkErrorSignals, threshold, version }. signals are substring/exact-value matches against a single channel; structural entries are JS predicates ((o: Observation) => boolean) for patterns that need to look across multiple fields at once (e.g. "empty top frame plus an Imperva iframe"). Both kinds carry a marker: 'B' (block-only — scores) or 'P' (present-always — context only, reported in matched but never moves the score).

The confidence model

Only 'B'-marked signals contribute to scoring; 'P' signals are fingerprint context that show up in matched but never move confidence. Matched 'B' signals are grouped by vendor and combined per-vendor via noisy-OR — score = 1 − Π(1 − weightᵢ) — so multiple weak signals from the same vendor can still add up to a confident verdict without any single signal needing to be definitive on its own. The vendor with the highest noisy-OR score wins; the page is classified blocked when that winning score is ≥ ruleset.threshold, and clear (or inconclusive, see above) otherwise. The weights on every signal in DEFAULT_RULESET, and its threshold (currently 0.5), are placeholders reproducing the legacy detector's boolean behavior — they get fit against real block/clear outcomes from a labeled corpus in a future release, not in this package's current form.

Privacy

Classification is metadata-only: vendor, kind, status/blocked/ inconclusive, confidence, tier, and reason (plus matched, which itself only carries vendor/channel/marker/weight/note). It never echoes back page content, cookie values, or query strings — those live only in the Observation you pass in, which the classifier reads but never returns any part of.

Development

npm test            # vitest
npm run build       # tsc -> dist/