@framers/agentos-ext-topicality

Embedding-based on/off-topic detection guardrail for @framers/agentos, with optional LLM fallback for ambiguous cases.

What it does

Compares incoming user messages against a configured topic policy. It checks input only: agent output passes through. An input close to a blocked topic is blocked; an input close to none of the allowed topics is flagged as off-topic.

Each input goes to the first tier that answers:

1. Embeddings. The guardrail imports @huggingface/transformers (an optional dependency), loads Xenova/all-MiniLM-L6-v2 once per process, and compares cosine similarity with each topic against maxBlockedSimilarity (default 0.5) and minSimilarity (default 0.3). When the package is missing or the model fails to load, each guardrail writes one warning with the error and the tier it uses instead (see Logging), and goes to the next tier.
2. LLM judge. When you pass an llmInvoker callback, the guardrail asks the model for a JSON verdict. A failed call goes to the next tier.
3. Keywords. A case-insensitive substring match between the input and each topic string, always available.

Use it to keep a customer-support agent on the product's domain.

Install

npm install @framers/agentos-ext-topicality

This package is an AgentOS extension pack. It expects @framers/agentos as a peer dependency.

Quickstart

import { AgentOS } from '@framers/agentos';
import { createTopicalityGuardrail } from '@framers/agentos-ext-topicality';

const agentos = new AgentOS();
await agentos.initialize({
  extensionManifest: {
    packs: [
      {
        factory: () =>
          createTopicalityGuardrail({
            allowedTopics: ['product support', 'billing', 'account access'],
            blockedTopics: ['politics'],
            minSimilarity: 0.3,
          }),
        enabled: true,
      },
    ],
  },
});

With no topics, or one list

createTopicalityGuardrail(); // or a manifest entry with no options
// first input: topicality: no allowedTopics or blockedTopics configured, so this guardrail passes every input

A pack given neither allowedTopics nor blockedTopics (a manifest entry with no options, or the extensions registry's options, which carry only a priority and secrets) passes every input. On its first input it writes the line above once, through the logger the host passes in the pack's context, else the activation context's, else the console.

One list alone enforces: blockedTopics alone blocks a matching input and lets the rest through; allowedTopics alone flags an input that matches none of them.

Logging

When the embedding tier cannot load, each guardrail writes one line:

topicality: the embedding tier is unavailable (Xenova/all-MiniLM-L6-v2 could not load: <error>); deciding with the keyword tier

Like the line about missing topics, it goes to the logger the host passes in the pack's context, else the activation context's, else the console.

Public API

• TopicalityGuardrail — guardrail class implementing the AgentOS guardrail contract
• CheckTopicTool — tool agents can invoke to ask the guardrail for an explicit decision
• createTopicalityGuardrail(options) — factory returning an ExtensionPack
• createExtensionPack(context) — auto-discoverable factory used by AgentOS extension auto-pickup
• cosineSimilarity, clearEmbeddingCache — utilities for direct use

See src/types.ts for the full options shape (TopicalityOptions).

Examples

• test/ — unit tests demonstrating threshold tuning and fallback behavior

Lazy loading and optional install

This package is an optional dependency of @framers/agentos-extensions-registry. The registry ships catalog metadata; createCuratedManifest() calls import.meta.resolve() per entry and silently skips anything not installed. npm install @framers/agentos-ext-topicality is the gate.

Nothing loads at activation. The guardrail imports @huggingface/transformers and loads the embedding model on the first input it checks. Embeddings are cached per text in memory; clearEmbeddingCache() empties the cache, and the pack calls it on deactivation.

The LLM tier runs only when you pass an llmInvoker.

The guardrail registers with config.canSanitize = false and config.evaluateStreamingChunks = false: it never rewrites text and runs in Phase 2 of the two-phase dispatcher (parallel classifiers). A blocked-topic match returns BLOCK; an off-topic input returns FLAG.

For the full DI model and end-to-end walkthrough, see How extensions stay optional and lazy and the auto-loading guide.

License

Apache 2.0 — see the repo root LICENSE.