frankctl

Terminal-first CLI for the Frank Low-Code Pipeline platform — scriptable access to sources, streams, transforms, pipelines, backing datasets, and AI assistance.

Install

npm install -g @aiaiai-pt/frankctl
frankctl --help

Requires Node.js >= 18. The binary is frankctl.

Quickstart

frankctl auth login            # authenticate against your Frank instance
frankctl --api-url https://frank.example.com sources list
frankctl pipelines apply -f pipeline.yaml

# Or render a curated vertical through the same apply path
frankctl templates render cira -o cira.yaml
frankctl pipelines apply -f cira.yaml --dry-run-server --live-ontology

Full command reference: the Frank docs and frankctl <command> --help.

Develop from source

npm install
npm run build
npm link
frankctl --help

Scope

• frankctl status
• frankctl auth login|logout|status|token
• frankctl config set|get|list
• frankctl templates list|show|render
• frankctl pipelines list|get|validate|apply|export
• frankctl sources list|get|create|update|delete|discover|sync|logs|history
• frankctl sources credentials set|rotate (Vault mode only)
• frankctl backing-datasets list|get|create|update|delete|sync|entities (alias: frankctl bd; sync --force replays unchanged data/spec)
• frankctl sources streams list|set|refresh-schema
• frankctl transforms list|get|runs|logs|trigger
• frankctl runs get|wait|cancel
• frankctl patterns list|get|validate-params|register
• frankctl schedules list|get|set|delete|pause|resume|trigger (source-scoped; pause|resume --note "why" sets the Source's status and its schedule together)
• frankctl sinks list|get|create|update|delete|sync|pause|resume|health|test-connection|update-credential|update-connection
• frankctl datasets list|preview|snapshots
• frankctl ai compose-pipeline|generate-transform|review-sql|fix-ci-failure|suggest|publish-transform

Config files for sources create|update, backing-datasets create|update, sinks create|update, and sources streams set accept YAML or JSON (use - to read stdin).

Sinks: hosted and subscribed (ADL-047)

A sink ingests device data into Kafka and Bronze in one of two modes.

• hosted (default) — devices send to an endpoint Frank runs, with credentials Frank issues. --protocol mqtt (publish to our broker; created from a config file, because its topic rules need one: frankctl sinks create -f sink.yaml) or --protocol https (POST to an ingest URL; created from flags, since it has no topic rules).
• subscribed — Frank connects out to a provider and subscribes: --protocol mqtt, amqp_0_9, amqp_1, grpc or coap. Created from flags or a file.

# Subscribed sink from flags. The password never appears as an argument.
export FRANKCTL_SINK_PASSWORD='…'          # or use --password-stdin
frankctl sinks create --mode subscribed --name "Provider Feed" \
  --broker-url wss://broker.example.com/mqtt \
  --topic 'fleet/+/position' --topic 'fleet/+/status' \
  --auth password --username frank --qos 1

# Try the settings before saving them (<=10s probe)
frankctl sinks test-connection --broker-url wss://broker.example.com/mqtt \
  --topic 'fleet/+/position' --auth password --username frank

# ...and afterwards, against what is stored
frankctl sinks test-connection <id>
frankctl sinks health <id>

--kafka-topic is optional for a subscribed sink: the server defaults it to <tenant>.subscribed.<slug>.

Secrets never go on the command line. There is no --password flag, because an argument lands in shell history, ps output and CI logs. Pass the provider password through --password-stdin or FRANKCTL_SINK_PASSWORD, and an mTLS key through --client-key-file or FRANKCTL_SINK_CLIENT_KEY. In a config file, name an environment variable rather than writing the value:

name: Provider Feed
mode: subscribed
connection:
  broker_url: wss://broker.example.com/mqtt
  topics: ['fleet/+/position']
  auth_mode: password
  tls:
    skip_verify: false     # verification is on by default
credential:
  username: frank
  password_env: PROVIDER_MQTT_PASSWORD   # the NAME, not the password

A literal password: in that file is rejected with an explanation rather than silently ignored.

--tls-skip-verify turns off certificate verification and prints a warning: the connection stays encrypted but the broker's identity is not checked. Prefer --ca-file for a broker with a private CA.

Rotating the credential or changing the broker:

frankctl sinks update-credential <id> --username frank --password-stdin < pw.txt
frankctl sinks update-connection <id> --broker-url wss://new.example.com/mqtt \
  --topic 'fleet/+/position' --auth password --username frank
frankctl sinks sync <id>     # both take effect on the next sync

Exit codes. sinks health <id> and sinks test-connection exit 0 only when the sink is actually working — state: connected, and for test-connection ok: true. Any other state exits 4 (API), including unknown, which a paused or never-synced sink reports: a CI health gate should fail on a silent feed rather than pass. Usage mistakes exit 2.

frankctl sources sync <id> waits for the exact workflow_id returned by its sync request, never the Source's latest status. Its JSON result is now the run details (workflow_id, source_id, sync_run_id, status, available workflow identifiers, row/stream counts and error details), replacing the old Source summary/counters. Only a persisted completed run succeeds; failed, cancelled, partial and skipped runs exit 5. Timeout exits 4 with timed_out: true and the last exact-run observation. A missing or mismatched identity fails closed. A temporarily missing run (HTTP 404) is polled again at the same workflow URL. --no-wait still returns only workflow_id; draft Sources still become ready before the sync starts.

frankctl backing-datasets sync <id> polls exact run evidence when the API advertises sync contract v2. For an ordinary sync against an older API, it falls back to BackingDataset status polling and explicitly reports that replay evidence is unavailable. --force is stricter: the CLI verifies the capabilities endpoint before the mutating request and sends zero sync POSTs when force replay is unsupported or unadvertised.

Declarative apply (pipelines apply -f)

frankctl pipelines apply -f vertical.yaml can provision a complete Source + Pipeline + BackingDataset triple from a single multi-doc YAML file when the tracked Source contains no credential value. Docs use the kubectl-style envelope (apiVersion: frank.platform/v1, kind:, metadata.name:, spec:). The CLI sorts by kind (Source → Pipeline → BackingDataset), resolves cross-doc name references to UUIDs, and POSTs each doc with ?if-not-exists=true. frankctl pipelines export <id> -o yaml round-trips an existing pipeline back to a multi-doc YAML file for review.

Credential handling follows the server's explicit SOURCE_CREDENTIAL_MODE. Current mutable/base Compose uses legacy_inline: Source create/update accepts write-only credential_values and persists them in Source.source_config; read/export stay redacted, and credential references plus the sources credentials commands are unavailable. The immutable release uses vault: values are written separately with sources credentials set|rotate --values-file -, and Git-owned Sources may carry the tenant-owned opaque credential_ref. Export never returns values; it strips inline credential fields and preserves only a Vault-mode reference when one exists. See ../docs/guides/cli.md for the full reference.

Relationship-bearing BackingDatasets declare their prerequisites in metadata.dependsOn and may use either spec.schedule_type (on_materialization or manual). The graph orders that invocation. The server separately derives automatic sync prerequisites from persisted relationship mappings and waits for registered targets in the same Frank and ontology tenant to be synced at their current snapshot and mapping. An upstream change after selection can still race a sync; unresolved relationships use bounded retry. --wait triggers BackingDatasets in topological order and requires external targets to be already synced; missing dependencies, cycles, and self-referential relationships fail before execution.

Source.spec.streams is desired state on create and reapply: declared streams are created/patched by name without resetting cursor, chunk, counter, or run state. Omitted streams are preserved and reported as unmanaged; explicitly set is_enabled: false to disable one. Client dry-run shows the declaration, server dry-run shows per-stream actions, and export emits config-only stream declarations without runtime state. With --wait, an explicit declaration syncs only its enabled stream names, so preserved unmanaged streams do not run. For a migration, explicitly declare known obsolete enabled streams with is_enabled: false as well, so scheduled/direct Source syncs converge safely.

Use --dry-run-server --live-ontology for an opt-in, GET-only check of BackingDataset mappings against the server's configured ontology. Use --wait --json to execute with exact Source, Transform, and BackingDataset run correlation and emit one evidence object. --wait does not replace a separate ontology entity readback; an ontology run reported as skipped means the snapshot was already converged and no rows were replayed. REST bulk runs include durable job IDs and submitted/completed counts from the exact ontology SyncRun; credentials and row payloads are never included.

Environment

• FRANKCTL_API_URL — Frank API base (default http://localhost:8000)
• FRANKCTL_KEYCLOAK_URL — Keycloak base (default http://localhost:8180)
• FRANKCTL_KEYCLOAK_REALM — realm (default frank)
• FRANKCTL_CLIENT_ID — override the OAuth client id. Browser PKCE defaults to the dedicated public frank-cli client; headless password login retains frank-low-code; service-account login requires its own client id.
• FRANKCTL_CLIENT_SECRET — client secret (service account only)
• FRANKCTL_PROFILE — active config profile
• FRANKCTL_TOKEN — bypass token store entirely
• FRANKCTL_TENANT_ID=<organization-slug> — explicit tenant context for tenant-scoped API requests; use --tenant <organization-slug> for a one-command override or persist profiles.<name>.tenantId
• FRANKCTL_SINK_PASSWORD — provider password for a subscribed sink (sinks create|update-credential|update-connection|test-connection); alternative to --password-stdin
• FRANKCTL_SINK_CLIENT_KEY — PEM client key for a subscribed sink using mTLS; alternative to --client-key-file
• PATTERN_WEBHOOK_SECRET — HMAC-SHA256 signing key for frankctl patterns register (CI only; command exits 6 when unset)

Browser login stores the OAuth client id beside the token and reuses that same client for refresh. Profile, environment, and --client-id overrides therefore remain valid without refresh silently switching back to a built-in default.