@app-studio/qa-doctor

Proves that a repository's qa.bindings.json describes the repository it is actually in.

Why this exists

A mature QA setup was copied from one repository into another without its bindings being rewritten. In the new repository the guard hook's patterns matched no file, so it denied nothing while looking exactly like a guard with nothing to complain about. The documented ports were wrong, so every instruction built on them sent people to a door that was not there. The seeded accounts did not exist, so agents signing in with them reported an authentication bug. And the documentation that was accurate was referenced by no entrypoint, so nobody read it.

None of that failed loudly. All of it was greppable. This package is that observation turned into a check.

Two ways to run it, one implementation

On demand:

pnpm exec qa doctor            # human report
pnpm exec qa doctor --json     # for agents and CI
pnpm exec qa doctor --strict   # warnings become errors

Inside the repository's own suite, which is what makes it un-skippable:

// tests/qa-doctor.test.ts
import { doctorSuite } from '@app-studio/qa-doctor/vitest';

doctorSuite();

One it() per rule. qa doctor catches a wrong binding when somebody runs it; this catches it on every CI run, in the suite that already gates merges — so skipping the check means skipping the suite, which is a decision somebody has to make out loud. A self/* rule verifies this file exists and falls inside the globs the test command selects, because a test nothing runs is worse than no test: the repository looks checked.

What it checks

Rule ids are stable: the CLI groups by them, tests assert on them, and hosts grep for them.

The report

Each finding carries three things, not one:

✖ ports/bound-by-agrees — each declared port is the number the file that binds it actually uses
   ports.front says 5173, but playwright.config.ts never mentions that number.
   in: playwright.config.ts
   what this means: Every instruction built on this port sends people to a door
     that is not there, and the resulting connection refused reads as a broken
     application rather than a wrong binding.
   fix: Open playwright.config.ts, read the port it binds, and put that number
     in ports.front.

A doctor that prints rule names and exit codes gets ignored within a week, and an ignored doctor is indistinguishable from no doctor at all.

Writing your own rules

import { runRules, createDoctorContext } from '@app-studio/qa-doctor';
import type { Rule } from '@app-studio/qa-core';

const noOrphanedFeatures: Rule = {
  id: 'features/registered',
  describes: 'every feature directory is in the composed registry',
  appliesTo: (context) => context.exists('src/features'),
  check(context) {
    const declared = context.read('src/generated/registry.ts') ?? '';
    return context
      .glob('src/features/*/manifest.json')
      .filter((path) => !declared.includes(path.split('/')[2]!))
      .map((path) => ({
        level: 'error' as const,
        rule: 'features/registered',
        message: `${path} is not in the composed registry.`,
        means: 'The feature exists on disk and is not part of the application, which is invisible until someone looks for it.',
        fix: 'Run the composer, or delete the directory.',
      }));
  },
};

Rules are pure functions of a DoctorContext, so they test against fixture repositories rather than against your real one — which matters more than usual here, because a doctor nobody has watched go red is exactly the inert machinery this package exists to prevent.

Requirements

Node.js 20 or newer. vitest is an optional peer, needed only for the /vitest entry point.