@aipathway/conformance
Run a published AI Pathway build standard against your own build, on your own machine, before you talk to anybody.
npx @aipathway/conformance init booked-after-hours-build-standard
cd booked-after-hours-build-standard && npm install && node run.mjs
That writes a folder that runs: the harness, a build file to implement, and one scenario per check carrying that check's own words. Every check starts red. Going green is the work, and a green run means something because a red one was possible.
npx @aipathway/conformance list # the standards
npx @aipathway/conformance show <slug> # every check, what it injects, what it asserts
npx @aipathway/conformance json <slug> # the same, for your own harness
npx @aipathway/conformance init <slug> [dir]
npx @aipathway/conformance connect [dir] [--wait] # a key for the hosted step
Getting the key
Where a standard names one of our providers, the second run is against us, and
that needs a key. connect gets one without leaving the terminal: it prints a
link and a code, the account owner opens the link, signs in, checks the code and
approves, and the key is written to .env beside the scaffold (git-ignored;
run.mjs reads it). It never asks for the scope that makes a phone ring. Then
it prints what the account still needs, with the link the owner opens to connect
their job system or ledger, and --wait keeps checking until they have.
What init writes
| file | whose | what |
|---|---|---|
build.mjs | yours | the build. The only file that does the job |
scenarios.mjs | yours | one holds per check, generated with the check's own words |
run.mjs | ours | fetches the checks, runs, prints. No edits needed |
AGENTS.md | ours | the same instructions, written for a coding agent |
It does not look for, patch or launch your existing code. It writes a new folder beside it, because guessing how somebody's project boots is how a tool becomes a framework.
What this is
Each standard is a set of numbered checks. Most of them can be proved on a stub on your laptop. A few cannot: they need a live number, a real ledger or a real register, and those say so rather than failing you for not having one.
The verdicts mean different things and each one has a different next step.
| verdict | meaning | what you do |
|---|---|---|
pass | asserted green here | nothing |
fail | asserted red here | this is a fixture. It names one check |
shape_only | shape holds on the stub, truth needs the live provider | re-runs when you switch provider |
with_us | cannot be proved here | the step the standard says not to hand-roll |
evidence_required | not a software check | go and do it, then attach what proves it |
A fail is the useful one. It names a single check, carries what went in and
what your build did, and is enough to act on without anybody reconstructing your
business.
The one rule that makes it work
Everything your build does to the outside world goes through the ports. The
runner owns the world your build writes into, so a build that calls a vendor SDK
directly leaves that world empty and the run reports no write observed through the port rather than quietly passing. That is not pedantry: it is the only way a
result means anything.
import { StubWorld, runStandard, fetchChecks } from "@aipathway/conformance";
const std = await fetchChecks("booked-after-hours-build-standard");
// Your build takes the ports and does the job. Nothing else.
const build = (p) => {
p.disclose("This is an AI assistant and the call is being recorded.");
const said = p.hear() ?? "";
const addr = p.resolveAddress(said);
if (!addr?.inArea) return;
const open = p.findOpenJob("+61400000001");
if (open) return void p.updateJob(open.id, "intent-1");
const job = p.writeJob({ phone: "+61400000001", address: addr.resolved, type: "emergency" }, "intent-1");
if (job) p.notify("on-call", job.id, "emergency");
};
You supply a scenario per check: the world it starts in, how to drive your build, and what "held" means. The check tells you what to inject and what to assert; the scenario is the few lines that connect that to your code.
The checks are not in this package
They are fetched from the site, so a standard corrected on Tuesday does not need a release here, and you cannot end up conformant against a stale copy. The package is the harness. The site is the standard.
The standards
27 of them, each one a job that leaks money in an Australian small business or
network. Every one publishes its checks as data at the same /checks.json path.
| Standard | The job |
|---|---|
| booked-after-hours | The phone answered out of hours, the job written once |
| debtor-chasing | Overdue invoices chased against a live ledger |
| rent-arrears | Arrears worked without breaching a tenancy rule |
| database-reactivation | A dormant contact list worked properly |
| invoice-check | Subcontractor invoices matched before they are paid |
| rejected-pack | Paperwork that fails after the work is done |
| compliance-calendar | Obligations that fall due on somebody else's schedule |
| cited-answer | Answers that carry their source and stop when it moves |
| multi-site-conformance | Branches that have quietly diverged |
| fire-service-pack | Service evidence bound to the right asset |
| quote-out | A job that exists turned into a draft quote, priced only from the book |
| advice-file | Every interview becomes one file the next person can pick up |
| knowledge-graph | A fact lives once, and retiring a source takes its answers with it |
| knowledge-base | Everything the business relies on, in one place, with a name on it |
| invoice-out | The invoice exists the day the job finishes, one per job, priced only from the book |
| dispatch-board | The right person at the right house at the right time, told once |
| inbox-triage | Nothing in the inbox lost, nothing answered that a person should answer |
| follow-up | The quote decision or document owed, chased until it arrives and not after |
| two-records | Two systems say the same thing, and a person sees both numbers first |
| timesheet-check | Every hour traced to a person, a day and a job before payroll |
| material-order | Materials ordered from the job, from the catalogue, once |
| recurring-report | The report on schedule, every figure traced to its source |
| cash-seam | One record from the call to the chase, each carrying the id of the one before |
| simpro-claude-connector | Claude on your own Simpro, on your own key, touching nothing a customer sees |
| price-one-pager | The vendor sees where the price came from, and the record proves it later |
| open-home-feedback | Every open-home attendee followed up with your yes, and the vendor told what buyers said |
| just-sold-pack | A sale nearby told to the neighbours, every message approved by a person |
Also available
- The standards themselves: aipathway.com.au/explore-ai,
free under CC BY 4.0, with a Markdown twin at
.mdon any URL. - As Agent Skills: github.com/digitalgym/aipathway-standards,
installable with
/plugin marketplace add digitalgym/aipathway-standards. - This package's source: packages/conformance in the same repo. It is generated from the standards rather than hand-written, so corrections go to the standard and the package follows.
- For agents: AGENTS.md, llms.txt, llms-full.txt.
Who publishes this
AI Pathway, Brisbane. We publish the standards free and sell the one step in each that should not be hand-rolled, which is usually the part carrying an Australian legal obligation a machine cannot hold: a live number with AI disclosure, Do Not Call Register checks, calling hours by state, and a record that survives a dispute.
If a check will not come good, send the failing run from the foot of that standard's page. It names one check, so it gets a fixed price rather than a discovery call.
Licence
CC BY 4.0. The standards are free to implement, free to fork, and free to build a competing product against. If a check is wrong, say so: the fastest way is to run it and send what came out.