@agent-forge/plugin-desktop-macos-observe

Read-only macOS desktop observation capability plugin for Agent Forge, structured as the macOS counterpart of plugins/desktop-windows-observe. It registers five read-only tools — desktop_probe, desktop_session_open, desktop_observe, desktop_reobserve, desktop_session_stop — backed by the macOS Accessibility tree through a JXA driver (osascript -l JavaScript). It never registers an input or replay tool.

The plugin ships its own osascript/JXA adapter. On non-macOS platforms importing the package does not throw; every tool reports an explicit unsupported result instead of probing or falling back to a different driver. A missing Accessibility (assistive access) grant is reported fail-closed as the probe reason accessibility-permission-required; there is no degraded observation mode.

Install

# from a local checkout
agent-forge plugin install ./plugins/desktop-macos-observe

# from npm
agent-forge plugin install @ai-agent-forge/plugin-desktop-macos-observe

The package ships two manifests: agent-forge.json (ecosystem manifest for the deterministic plugin resolver, platform.os: ["macos"]) and plugin.json (host-load manifest consumed by loadCapabilityPluginFromManifest, entry ./dist/entry.js).

Library surface

import { DesktopMacosAdapter, MacosAccessibilityDriver } from "@agent-forge/plugin-desktop-macos-observe";

• DesktopMacosAdapter — read-only Accessibility adapter facade.
• MacosAccessibilityDriver(options?) — JXA driver; supports command/timeoutMs/spawnSync/platform injection for brokers and tests.
• createDesktopObserveCapabilityFactoryV1(runtime) — registers the five observation tools on a CapabilityAPI.
• Default export of dist/entry.js — the precomposed factory used by external plugin loading.

The observation projection (window/control identity, budgets, truncation) is a deliberate inlined copy shared with @agent-forge/plugin-desktop-macos-input so both packages stay self-contained.

Real-host status

The driver and adapter are contract-tested with an injected fake spawnSync (canned JSON, no real desktop). Real darwin behavior — the JXA enumeration script, Accessibility permission classification and observed window/control fidelity — is verified only by the explicitly gated test (AGENT_FORGE_MACOS_AX_REAL_TEST=1 on macOS) and has not been validated on a real macOS host as of this change.

Development

npm install --ignore-scripts        # repo root, creates the workspace symlink
npm run build                       # plugins/desktop-macos-observe (tsgo)
npm test                            # vitest; the entry fidelity test requires `npm run build` first