@expo/verify
Portable engine for the /verify agent pipeline: an AI agent investigates a
GitHub issue or pull request in an isolated sandbox, a second agent attacks
the draft, and the revised findings post back to the thread. This package
carries all the logic; the repos that use it hold only a thin workflow file
and a .expo-agents/verify/ profile.
Design of record: LLP 0001 in this repository.
Usage
No install needed:
npx @expo/verify <command>
Commands
| Command | What it does |
|---|---|
verify run [--dry-run] | Run the investigation here — on a CI runner, an EAS worker, or a laptop: context → investigate → critic → revise → publish. A comment body in env selects the comment path (gate, ack, announce, report). --dry-run investigates for real but posts nothing. |
verify mint / verify revoke | The scoped-token steps a workflow runs around run: mint exchanges the parent EXPO_SANDBOX_MCP_TOKEN for a scoped token and writes the agent's mcp.json to VERIFY_MCP_CONFIG (0600); revoke releases it afterwards. The run step then holds only the scoped token — the parent is mapped into the mint/revoke steps on GHA and blanked on the run step on EAS. Without VERIFY_MCP_CONFIG, run mints in-process (the laptop path). |
verify dispatch <target> | Start a run remotely on the profile's runner (GitHub Actions or EAS Workflows) for an issue/PR number or URL, quietly. verify <target> is shorthand. |
verify triage <owner/repo#N> [--dry-run] | Classify an issue or PR (classify → critic → revise, no tools) and publish routing labels plus a triage comment as the profile's bot. --dry-run writes the proposed result locally without touching GitHub. Only expo/expo is supported for now; see Triage. |
verify init --repo <owner/name> | Scaffold .expo-agents/verify/ and the thin runner workflow. Repos set up before 0.9 keep .verify/: every command looks for the new name first and falls back to the old one. |
verify verify-config | CI guard: profile validation, credential-env lock, version-pin assertion. |
verify doctor | Read-only environment/profile/credential triage. |
verify status / verify ls | Dashboard of in-flight and recent runs / in-flight only. On GitHub Actions this lists the dispatch and comment workflows, skipping gate-skipped runs, and reports each finished run by its jobs' outcome — the thin workflows use job-level continue-on-error, so the run itself always says success. EAS runs are titled from the job outputs the engine emits (target, mode, outcome), since EAS has no per-run name. |
Comment commands (/verify, @expo-bot verify, @expo-bot check,
@expo-bot triage, and work requests) receive an 👀 acknowledgment without a
"started" comment by default. Pick the agent model with --model <name>, a
shorthand flag (--fable, --opus, --sonnet, --haiku), or the phrase
with <name> on verify, check, and triage commands, for example
@expo-bot verify with fable. Work requests accept only the flag forms, since
"with" is ordinary prose in a task. Add --announce on the first line to post one,
for example @expo-bot verify --announce. --no-announce takes precedence if
both flags are present. Findings and failure reports still post to the thread.
Use verify status to find runs without a start comment.
Library use
The package also exports the sensitive-path classifier, so another engine can use the same rules as the publisher:
import { sensitivePaths } from "@expo/verify";
sensitivePaths([".github/workflows/ci.yml", "src/index.ts"]);
// [{ path: ".github/workflows/ci.yml", category: "ci", reason: "..." }]
Each hit has one category:
ci: files CI runs, such as.github/,.eas/,fastlane/,.changeset/config.json, and the rootbin/,tools/, andscripts/.agent-instructions: files agents read, such asAGENTS.md,CLAUDE.md,GEMINI.md,.claude/,.agents/,.cursor/, and.mcp.json.supply-chain: lockfiles (includingnpm-shrinkwrap.json),patches/,binding.gyp, registry config includingbunfig.toml, and dependency files such asGemfileandPodfile.vcs-hooks: git hooks, hook-manager config such as lint-staged and lefthook files, and git config such as.gitmodulesand.gitattributes.secrets: keys, certificates,.envfiles, andsecrets/.package-manifest: apackage.jsonwhosescripts,packageManager,devEngines, git hook config (lint-staged,simple-git-hooks,husky),trustedDependencies, or dependency overrides change. Pass{ manifests }with the before and after text; without it, everypackage.jsonis a hit.
Matching ignores case, because a macOS checkout does. The publisher refuses
every hit, and also refuses any staged symlink or submodule change, which no
path rule can see. The package also exports deniedPaths
(the publisher denylist), unsafePath, and hasControlCharacters.
Duplicate pull requests
In pr fix mode the publisher checks the fix against every open pull
request before opening one. File overlap only shortlists: an open PR that
touches every non-changelog file the fix touches is a candidate, capped at
five. For each candidate a tool-less, schema-bound model call reads the
shared-file patches on both sides and answers whether the candidate already
makes the same change (templates/duplicate.md is the policy). A confident
duplicate keeps the pushed branch and opens no PR; anything else opens the PR
and lists the candidates, with the judge's one-line reason, for the reviewer.
Without a judgment (no model, a failed call, a malformed reply) the fallback
is a symmetric file-set match. Everything the decision saw is written to
.verify-out/duplicate-judge.json.
Profile field, all keys optional: "duplicateJudge": { "enabled": true, "model": "claude-haiku-4-5-20251001", "threshold": 0.7 }. The judge runs on
the run's model credential with the same isolation as triage.
Triage
From this source checkout, with gh authenticated and a Claude Code version
supporting --safe-mode and --json-schema:
bun src/cli.ts triage expo/expo#49760 --dry-run --out-dir /tmp/verify-triage
Triage supports Expo issues and PRs. It collects context and runs
classify → critic → revise with Opus, without tools or sandbox access.
--dry-run renders proposed results without writing to GitHub.
Maintainers can comment @expo-bot triage on an issue or PR to classify it
and publish routing labels plus a preliminary triage comment. Commands receive
an 👀 acknowledgment; the start reply is opt-in with --announce. The first
verify run also attempts triage before investigation. Later verify runs
skip targets with a completed triage comment authored by the configured bot;
an explicit triage command always reruns it. Triage failure never blocks verify.
For local publishing, omit --dry-run and use the target repository's trusted
profile (--config-dir PATH if needed), authenticated as its configured bot.
Publishing only adds existing routing labels and updates the bot's triage comment;
it never closes targets or removes labels. Platforms become iOS, Android,
or Platform: web. A PR gets no SDK label; its author decides whether to
add one. Bug issues may receive
severity: critical, severity: high, severity: medium, severity: low, or
severity: trivial. Any existing severity label is preserved, including on
explicit reruns. Reports spell out the code, name, and meaning (for example,
S3 - Low). Comments use short impact and scope bullets rather than a
narrative summary. The verdict still records workaround status, which informs
severity, but the comment leaves it out. An uncertain completed verdict receives no labels but still counts
as triaged. In expo/expo, every newly opened issue and PR, external
contributors' included, is triaged automatically with
verify triage <target> --if-untriaged, which skips a target the bot has
already triaged.
The policy is templates/triage.md; configurable label maps remain deferred.
Each target gets a unique output directory containing context, phase responses,
draft.json, review.json, result.json, and any proposed-comment.md.
The result includes proposed/skipped labels, input provenance, elapsed time,
and model-reported USD cost (an estimate, not a subscription invoice).
Failures retain error.json and any completed phase artifacts.
Options: --model (default claude-opus-5-5), --timeout-seconds (default 120 per
phase, maximum 300), and --max-budget-usd (default 0.50 per phase, maximum
5). Each run has three phases and no automatic retries. Inputs sent to the
model are capped at 60,000 characters, with truncation explicitly recorded.
The classifier has no tools or MCP servers; it receives an allowlisted
environment and runs with customizations disabled in an empty temporary
working directory. Only the fixed collector uses GitHub credentials.
Adopting a repo
- In the repo's checkout:
verify init --repo owner/name [--runner gha|eas] [--fix pr|gist|inline|comment|none]Fix modes:propens a pull request from the bot fork;gist/inlinepost a separate patch comment;commentfolds the guarded patch into the findings comment itself (a<details>block with a concise explanation and a fenced diff — for repos that should never receive a second comment or a PR);noneverifies only. - Fill in
.expo-agents/verify/build.md(the repo's sandbox build recipe) and.expo-agents/verify/policy.md(repo-specific agent policy). - Add the secrets the generated workflow names, then run
verify verify-config. - Dispatch:
verify <issue-number>(orverify <n> --runner eas).
Running on EAS Workflows
verify <n> --runner eas from a repo's checkout uploads that checkout's
verify files — .expo-agents/verify/, .eas/workflows/verify.yml, app.json,
eas.json — and runs the investigation on EAS. The upload is a few KB
(.easignore trims the rest; the run clones the target repo itself in a
sandbox), uncommitted files are included, and there is no GitHub link or
repo-ownership requirement. verify init --runner eas --eas-project <id>
writes all of those files and resolves the project's owner/slug into
app.json. On the worker the engine clones the target's default branch
itself (shallow — the same trusted base a GitHub Actions checkout gives),
so fix mode and the publisher work on EAS exactly as on GHA.
This is also how a repo you do not own gets verified: the same files, kept
as local uncommitted changes, customized freely in .expo-agents/verify/; commit them
only if and when you want to land verify upstream.
The profile is enforcement, not documentation: its capability fields drive the agent tool allowlist, the scoped-token mint, and which publish steps run.
Releases
Push a tag: git tag v<semver> && git push origin v<semver>.
release.yml stamps the version from the tag and publishes via npm trusted
publishing (OIDC). The repository is private; the published package contains
only build/ and templates/.