@agent-native/pinpoint

Visual feedback and annotation tool for web applications. Users select UI elements, add feedback, and send structured context to any AI agent.

Pinpoint is a Toolkit capability module installed on demand. It lives in its own npm package so apps only take the visual feedback surfaces and agent integration they need.

Works standalone, with Builder.io, or with any agent runtime that speaks the agent-native protocol (Builder.io, Claude Code, Codex, Gemini CLI, Cursor, etc.).

AI Agent Setup Guide

Instructions for an AI agent (Builder.io, Claude Code, Codex, Cursor, etc.) to add Pinpoint to an agent-native repository.

Step 1: Install

pnpm add @agent-native/pinpoint

Step 2: Initialize agent actions and skill

npx @agent-native/pinpoint@latest init

This copies agent actions to actions/ and the Pinpoint skill to .agents/skills/pinpoint/. No manual file copying needed.

Step 3: Mount the client overlay

Find the root React component (typically app/App.tsx or src/App.tsx). Add the <Pinpoint /> component:

import { Pinpoint } from "@agent-native/pinpoint/react";

// Add as a sibling to your app root — renders nothing, mounts overlay in Shadow DOM
<Pinpoint author="User" endpoint="/api/pins" autoSubmit colorScheme="auto" />;

If the app is not React-based, use the imperative API:

import { mountPinpoint } from "@agent-native/pinpoint";
mountPinpoint({ author: "User", endpoint: "/api/pins", autoSubmit: true });

Step 4: Add server middleware

Find the Express server setup (typically server/index.ts). Add the pin routes:

import { pagePinRoutes } from "@agent-native/pinpoint/server";

// Add before your page routes
app.use("/api/pins", pagePinRoutes());

This creates REST endpoints at /api/pins for pin CRUD. Pins are stored as JSON files in data/pins/.

Step 5: Verify

1. Start the dev server
2. Open the app in a browser
3. Press Cmd+Shift+. to toggle the Pinpoint toolbar
4. Click any element, add a comment, click "Add Pin"
5. Click the send button — the annotation should appear in the agent chat

Working with Pins

Pins are stored as individual JSON files in data/pins/{uuid}.json:

{
  "id": "uuid",
  "pageUrl": "/dashboard",
  "comment": "This button color is wrong",
  "element": {
    "tagName": "button",
    "selector": ".sidebar button.primary",
    "classNames": ["primary", "btn"]
  },
  "framework": {
    "framework": "react",
    "componentPath": "<Sidebar> <ActionButton>",
    "sourceFile": "src/components/Sidebar.tsx:42"
  },
  "status": { "state": "open" }
}

Key fields for agents:

• sourceFile — the exact file and line to edit
• componentPath — the React/Vue component hierarchy
• selector — CSS selector to find the element in the DOM
• comment — what the user wants changed

Agent commands (available after npx @agent-native/pinpoint@latest init):

• pnpm action get-pins --status open — list unresolved pins
• pnpm action resolve-pin --id <uuid> — mark as resolved after fixing
• pnpm action create-pin --pageUrl / --selector ".btn" --comment "Fix this"
• pnpm action update-pin --id <uuid> --comment "Updated feedback"
• pnpm action delete-pin --id <uuid> — remove a pin

Workflow:

1. User creates annotations in the browser
2. Read with pnpm action get-pins --status open
3. Use sourceFile to locate and edit the relevant code
4. After fixing, mark resolved: pnpm action resolve-pin --id <uuid>

Troubleshooting

Install

pnpm add @agent-native/pinpoint
# or
npm install @agent-native/pinpoint

Quick Start

React

import { Pinpoint } from "@agent-native/pinpoint/react";

function App() {
  return (
    <>
      <Pinpoint author="Designer" endpoint="/api/pins" autoSubmit />
      <YourApp />
    </>
  );
}

Vanilla JS / Script Tag

<script src="https://unpkg.com/@agent-native/pinpoint/dist/index.browser.js"></script>
<script>
  Pinpoint.mountPinpoint({ author: "Designer" });
</script>

Imperative API (Non-React)

import { mountPinpoint } from "@agent-native/pinpoint";

const { dispose } = mountPinpoint({
  author: "Designer",
  endpoint: "/api/pins",
});
// Call dispose() to unmount

Setup in an Agent-Native App

1. Install

pnpm add @agent-native/pinpoint

2. Initialize

npx @agent-native/pinpoint@latest init

Copies agent actions to actions/ and the Pinpoint skill to .agents/skills/pinpoint/.

3. Client — Mount the overlay

// app/App.tsx
import { Pinpoint } from "@agent-native/pinpoint/react";

function App() {
  return (
    <>
      <Pinpoint
        author="Designer"
        endpoint="/api/pins"
        autoSubmit
        colorScheme="auto"
      />
      <YourApp />
    </>
  );
}

4. Server — Add the REST middleware

// server/index.ts
import { createServer } from "@agent-native/core/server";
import { pagePinRoutes } from "@agent-native/pinpoint/server";

const app = createServer({
  /* ... */
});
app.use("/api/pins", pagePinRoutes());

How It Works

1. Toggle the toolbar: Cmd+Shift+. (or Ctrl+Shift+.)
2. Click any element to annotate it
3. Type feedback in the popup
4. Send to the agent: Cmd+Shift+Enter

The agent receives structured context: CSS selector, component hierarchy, source file location, and the user's comment.

Configuration

All options can be passed as props to <Pinpoint /> or as the config object to mountPinpoint():

CLI

npx @agent-native/pinpoint@latest init   # Copy actions and skill to your project
npx @agent-native/pinpoint@latest        # Show help

Keyboard Shortcuts

Package Exports

Server Middleware

Express middleware for pin CRUD:

import { pagePinRoutes } from "@agent-native/pinpoint/server";

app.use("/api/pins", pagePinRoutes({ dataDir: "data/pins" }));

Agent Actions

Available after running npx @agent-native/pinpoint@latest init:

Primitives API

Standalone functions for programmatic element inspection (no UI):

import {
  getElementContext,
  freeze,
  unfreeze,
  openFile,
  detectFramework,
} from "@agent-native/pinpoint/primitives";

const context = getElementContext(document.querySelector(".sidebar"));

freeze(); // Pause all animations
unfreeze(); // Resume

openFile("src/components/Sidebar.tsx", 42); // Open in editor

Plugin System

import type { Plugin } from "@agent-native/pinpoint/types";

const myPlugin: Plugin = {
  name: "analytics",
  hooks: {
    onPinCreate(pin) {
      analytics.track("pin_created", { page: pin.pageUrl });
    },
    onPinResolve(pin) {
      analytics.track("pin_resolved", { id: pin.id });
    },
    transformOutput(output) {
      return output + "\n\n_Sent via Pinpoint_";
    },
  },
  actions: [
    {
      label: "Export to Jira",
      handler(element, context) {
        createJiraTicket(context);
      },
    },
  ],
};

A2A & MCP

Expose pins to external agents via A2A or MCP:

import {
  registerPinpointA2A,
  createPinpointMCPTools,
} from "@agent-native/pinpoint/server";

registerPinpointA2A(app); // /.well-known/agent-card.json

const { tools, handleTool } = createPinpointMCPTools(); // MCP tool handlers

Framework Support

Storage Adapters

Builder.io Integration

Inside Builder.io's Fusion, annotations are sent via sendToAgentChat() from @agent-native/core:

<Pinpoint author="Builder User" autoSubmit outputFormat="standard" />

No additional configuration needed when running inside a Builder.io project.

Hosts with their own chat implementation can pass sendToAgent to reuse Pinpoint's pin, draw, queue, and prompt UI while delivering { message, context, submit } themselves.

Architecture

• SolidJS overlay in Shadow DOM — zero interference with host app styles or React reconciliation
• Canvas-based hover highlighting with LERP interpolation — smooth 60fps
• One file per pin — eliminates concurrent write conflicts
• Pluggable storage — MemoryStore, RestClient, FileStore
• MIT libraries: bippy, @medv/finder, element-source

License

MIT