@adlc/consensus-fix

N-version programming for failing tests. Fan N independent LLM completions at the same broken test, evaluate each candidate, group survivors by agreement, and recommend the smallest fix from the largest consensus group.

ADLC phase served: C7 / P4 — hard bugs that single-shot fixes can't crack, exploiting sampling diversity (E1).

Usage

consensus-fix --test-cmd "..." --files a.mjs,b.mjs [options]

Required flags

Optional flags

Exit codes

How it works

1. Pre-flight: run --test-cmd once. If it exits 0, the test already passes — opError.
2. Snapshot: capture the full content of every --files path.
3. Fan: send N independent, stateless LLM completions in parallel with the prompt: failing command + last 4000 chars of test output + a windowed excerpt of each source file (the region around whatever lines the test output references it at, or a length-capped tail when no reference is found — never the whole file for anything past a small size). Request JSON {changes: [{file, hunks: [{startLine, endLine, replacement}]}]} — a line-range edit, not full-file content.
4. Evaluate sequentially: for each candidate:
   • Parse and validate JSON. Discard if invalid, if any file is not in --files, or if changes is an empty array — a candidate proposing no changes is a no-op, not a fix, and is never allowed to become a survivor (it would otherwise apply cleanly and be judged solely by whether --test-cmd happens to pass a second time against the untouched source).
   • Apply hunks, run --test-cmd (the repro gate). A hunk whose startLine/endLine doesn't cleanly apply against the real file (out of bounds, overlapping another hunk) discards just that candidate — never the run. If the repro passes and --rails was supplied, run --rails (the regression gate) against the same applied changes. A candidate survives only if BOTH exit 0. A candidate that passes the repro but reddens the rails is rejected (lands in failed, not survivors).
   • Record pass/fail and changed-line count (computed directly from the hunks' own line ranges — no separate diff step needed).
   • Restore snapshot (finally) regardless of outcome.
5. Agreement grouping: normalize each surviving changeset's hunks (collapse whitespace per line, key on line range + normalized replacement) and group identical sets.
6. Winner selection: among candidates that passed BOTH gates, pick the member of the largest agreement group with the fewest changed lines (ties broken by candidate index). The smallest-diff tiebreaker never sees a candidate that failed the rails, so it cannot reward a gaming fix.
7. Output: dry-run report by default; --apply writes the winner.

All-divergent detection

When --n >= 3 and every survivor is in its own singleton group, the tool exits 2 with the message all-divergent: spec ambiguity — escalate. This is a signal that the test specification is ambiguous about something load-bearing — divergence is evidence, not noise.

Examples

# Dry-run: see what the winning fix would be without writing it
consensus-fix --test-cmd "node --test test/math.test.mjs" \
              --files src/math.mjs \
              --n 5

# Repro gate + full rails regression gate (recommended).
# A candidate that fixes the one test but breaks any other test is rejected.
consensus-fix --test-cmd "node --test test/math.test.mjs" \
              --rails    "node --test test/*.test.mjs" \
              --files src/math.mjs \
              --n 5

# Apply the winner
consensus-fix --test-cmd "npm test -- --grep 'add function'" \
              --rails    "npm test" \
              --files src/add.mjs,src/utils.mjs \
              --apply

# Preview prompts without any API key
consensus-fix --test-cmd "cargo test test_parse" \
              --files src/parser.mjs \
              --n 3 \
              --prompt-only

# Machine-readable output for orchestrators
consensus-fix --test-cmd "pytest tests/test_core.py::test_sum" \
              --files lib/core.mjs \
              --json

# Allow running against an uncommitted working tree
consensus-fix --test-cmd "node --test test/*.test.mjs" \
              --files lib/broken.mjs \
              --allow-dirty --n 3

# Draw one candidate fix per distinct provider family instead of 3 samples
# of one auto-detected model (requires an API key for each named provider).
consensus-fix --test-cmd "node --test test/math.test.mjs" \
              --rails    "node --test test/*.test.mjs" \
              --files src/math.mjs \
              --providers anthropic,openai,gemini

# Force a single non-default provider for the whole run
consensus-fix --test-cmd "node --test test/math.test.mjs" \
              --files src/math.mjs \
              --provider gemini --n 3

JSON output shape (--json)

{
  "summary": {
    "total": 3,
    "passed": 2,
    "failed": 1,
    "discarded": 0,
    "groups": 1,
    "allDivergent": false,
    "railsChecked": true
  },
  "groups": [
    { "groupIndex": 0, "size": 2, "candidateIndices": [0, 2] }
  ],
  "winner": {
    "index": 0,
    "changedLines": 1,
    "largestGroupSize": 2,
    "changes": [{ "file": "src/math.mjs", "hunks": [{ "startLine": 12, "endLine": 12, "replacement": "..." }] }],
    "applied": false
  },
  "discardedDetails": []
}

Safety guarantees

• Dirty-tree guard: refuses to run against a dirty git tree unless --allow-dirty. Prevents accidental loss of unstaged work.
• Snapshot + restore in finally: every candidate evaluation restores all --files to their original content, even if the test crashes or the LLM returns garbage.
• Termination signal restore: SIGINT, SIGTERM or SIGHUP at any point before the report (including while a candidate's test or rails command is running) stops the run: the in-flight command's process group gets SIGTERM, then SIGKILL after 2 s, the original files are restored from the snapshot, no winner is applied, and the process exits 1. SIGKILL sent to consensus-fix itself cannot be handled, so it can leave a candidate's hunks on disk; recover with git checkout -- <files>.
• Atomic writes: file restoration, candidate hunk application, and winning candidate --apply writes are atomic (write temp file in same directory, then renameSync) to guard against partial writes or truncation.
• File-list enforcement: any candidate that references a file outside --files is discarded with a diagnostic message, never written.

LLM provider detection

Inherits from @adlc/core. Checks (in order): ANTHROPIC_API_KEY, OPENAI_API_KEY, GEMINI_API_KEY (then agy if ADLC_AGY is set). Force a provider with ADLC_PROVIDER (env, whole shell) or --provider <name> (per-invocation, overrides ADLC_PROVIDER). Override the model per tier via ADLC_MODEL_CHEAP, ADLC_MODEL_MID, ADLC_MODEL_FRONTIER.

Single-provider auto-detect remains the default (cost/latency per ADR-0007) — --provider/--providers are additive, not a default-behavior change. Use --providers <a,b,c> when you want N candidate fixes drawn from N distinct model families instead of N samples of one model; each requested provider must have its own API key configured, or the run fails closed before any LLM call with a list of the missing ones.

Relationship to sibling tools

• flail-detector (C6): detects when a single agent is spinning. consensus-fix is the escalation path — run it when flail-detector fires on a hard bug.
• premortem (C2): identifies failure modes before building. consensus-fix fixes failure modes discovered during test runs.
• rails-guard (C5): enforces read-only rails. consensus-fix respects the --files list as its own rails boundary.
• hollow-test (C4): finds tests that don't actually test anything. A test that consensus-fix keeps failing on might be hollow — run hollow-test to check.

Core gaps

None. fan, complete, extractJson, detectProvider, resolveModel, PROVIDER_NAMES, parseArgs, pass, gateFail, opError, printJson, promptOnly, and isDirty are all provided by @adlc/core.