@framers/agentos-ext-grounding-guard

NLI-based factual grounding verification for @framers/agentos. Extracts claims from agent outputs and verifies each against the supplied source documents using natural-language-inference scoring.

What it does

Extracts each factual claim from an agent response, runs it against retrieval context (or supplied passages) using an NLI model, and reports per-claim entailment scores. Claims below the configured entailment threshold are flagged as ungrounded.

Use it to:

• Reduce hallucination in RAG agents
• Enforce a minimum-grounded-claims policy before responses ship
• Build hallucination-rate dashboards from per-claim scores

Install

npm install @framers/agentos-ext-grounding-guard

Peer dependency: @framers/agentos. Dependency: chrono-node 2.10.3, which whenSaid reads dates with.

Quickstart

import { AgentOS } from '@framers/agentos';
import { createGroundingGuardrail } from '@framers/agentos-ext-grounding-guard';

const agentos = new AgentOS();
await agentos.initialize({
  extensionManifest: {
    packs: [
      {
        factory: () =>
          createGroundingGuardrail({
            entailmentThreshold: 0.7,
            minGroundedClaims: 0.8,
          }),
        enabled: true,
      },
    ],
  },
});

Public API

• ClaimExtractor — pulls discrete factual claims from a response
• GroundingChecker — runs each claim through NLI scoring
• GroundingGuardrail — guardrail wiring both for the AgentOS contract
• createGroundingGuardrail(options?) — factory returning an ExtensionPack
• createExtensionPack(context) — auto-discoverable factory used by AgentOS extension auto-pickup

See src/types.ts for GroundingGuardOptions. ValueAgreementChecker and the value reader have their own section, Value agreement, a claim with no number in it is under A stated fact without a number, and the rule for a phrase a public source is searched with, phraseProblem, under A search phrase; verifyQuotes is under Verbatim quotes, whenSaid under The day a line names, checkSentence and checkLabel under Sentence check, and createNliFunction with the model's options under NLI.

Value agreement

Embeddings and NLI grade topic and entailment, not whether 1918 equals 1919. ValueAgreementChecker decides that by code:

import { ValueAgreementChecker } from '@framers/agentos-ext-grounding-guard';

const checker = new ValueAgreementChecker();
const source = { text: 'The Treaty of Versailles was a peace treaty signed on 28 June 1919.', group: 'Treaty_of_Versailles' };
const decision = checker.decide({
  heardWords: 'The Treaty of Versailles was signed in 1918.',
  heardSentence: 'The Treaty of Versailles was signed in 1918.',
  proposed: 'differs',
  named: [source],
  found: [source],
});
// decision.relation === 'differs'; decision.marks[0] marks "1919" in the source's sentence
// decision.pairs: [{ kind: 'year', unit: null, heard: { start: 39, end: 43 }, source: { start: 62, end: 66 } }], "1918" beside "1919"

Each value the claim states (a year, a date, a month, or a number with its unit) pairs with the one value of the same kind in the source's sentence, a number with the one of its unit class, the nearest shared word deciding when the sentence holds several; a word of a value's unit decides nothing, so in "The bridge in Leeds is 50 metres long and 20 metres wide." the "metres" of 50 says nothing of the 20, and a heard "20 metres wide" pairs with neither. The sentence matches when every pair is equal and both sides count the same negations, and differs when a pair is unequal. A "differs" also needs every content word of the claim in the sentence and a name the speaker said, so a source sentence is never shown as differing on a name or a wording alone; the letters of a unit, the C of "100°C", the A of "A$450" and the USD of "USD 500", name nothing. A number in a name ("Apollo 17") or a Roman numeral of one or two letters after a capitalised word ("World War I") must also stand in the source's sentence as written (nameMarks reads them) before a "matches", a "differs" or a "disagree" is shown, so a claim about Apollo 17 is never shown as matching or differing from a sentence about Apollo 11.

decide returns null when nothing may be shown: a range, an approximation ("around 1918"), an ordinal, a decade or a number that is part of a name ("Apollo 12") is never compared, names on or off, and a "differs" is withheld when another passage found agrees with the claim's values. Two sentences of one source that pair a value of the claim with different values return disagree. passages picks the sentences worth grading: at most 40 words and 600 characters, at least two stems shared with the claim, and a comparable value of a kind the claim holds; top is the first three.

A decision on values carries pairs, the value pairs its rule read: each heard value's kind ('year', 'date', 'month' or 'number'), the pair's unit class ('m', 'km', 'pct', 'usd', 'deg' and the rest, alike on both sides: 'deg:C' and 'deg:F' for a temperature with its scale ("100°C", "100℃") and 'deg' for a degree without one, 'cur:A$' for "A$450" and 'usd' for "$450", "US$450" and "USD 450", 'cur:GBP' for "GBP 450"; null for a number without one and for a year, a date or a month), its span in heardSentence (heard), and its partner's span in the first of sentences (source), all UTF-16 offsets, so a caller writes "1918 heard, 1919 in the source" or "8849 metres heard, 8,848.86 metres in the source" from the decision and never from a second reading of the sentences. Units stay with their values: every span the value rules give, heard, source and marks alike, covers the value with its unit as the reader found it: from the currency sign before it, right before it or with white space between, with a dollar's country letters ("$500", "€ 450", "A$450"), or from the currency code before it ("USD 500"); to the %, ‰, °, ℃, ℉ or currency sign after it, right after it or after white space ("61%", "64 %", "90℃", "450 €"), with a temperature's scale after the degree sign ("100°C", "90 °C"); to the currency code after it ("450 EUR"); to "per cent" after it; or to the unit word after it ("8849 metres"), with the scale after "degrees" ("212 degrees Fahrenheit"). So a disagree over two lengths marks "6,992 km" and "6,800 km", and a heard "100 degrees" never pairs with "212 degrees Fahrenheit". The pair the relation is shown on comes first: for a differs, the pair whose source span is marks[0]; for a disagree, the pair whose partners differ between the two sentences, its source span marks[0]; the rest follow in the heard words' order, so a value that agrees, heard before the value that differs, never leads. A matches marks nothing and still carries its pairs: "Mount Everest is 8849 metres tall." beside "Mount Everest is 8,848.86 metres tall." is one number pair of unit 'm', heard at 17 to 28 ("8849 metres") and source at 17 to 32 ("8,848.86 metres"). The heard words are found in the heard sentence as exactSpan finds a quote, white space collapsed on both sides, and each span is given in the sentence's own characters, the white space between a number and its unit word among them. pairs is empty on the names path, which pairs nothing, and when the heard words do not stand in the heard sentence.

Each value values reads carries its unit class (unit) and its offsets with that unit as written (withUnit), its own start and end when it has no unit; a decision's spans are these. A currency sign after a number is its unit only when no number follows the sign, which is then that number's, white space between or not ("In 2019 $5 bought a ticket." reads the year 2019 and "$5", and "In 2019 € 5 bought a ticket." the year 2019 and "€ 5"). A sign after a number with white space between leaves a year in its place ("In 2002 € coins replaced the franc." reads the year 2002), and a currency sign before a year's digits makes them money ("Prices rose from € 2000 to € 3000." reads two amounts). A currency code before a number, or after it, white space between, is its currency when it is one of USD, EUR, GBP, JPY, CNY, CHF, CAD, AUD, NZD, HKD, SGD, INR, SEK, NOK, DKK, KRW, BRL, MXN, ZAR and RUB ("USD 500", "450 EUR"): USD and EUR are the classes of $ and €, 'usd' and 'eur', and any other code is a class of its own ('cur:GBP'), so "GBP 450" pairs with "GBP 500" and never with "£500". A code after a number follows the rules of a sign after it ("In 2019 USD 5 bought a ticket." reads the year 2019 and "USD 5", and "In 1999 EUR prices rose." the year 1999), and a code before a year's digits makes them money ("Prices rose from USD 2000 to USD 3000." reads two amounts). Capitals outside the list before a number head a name, as "Apollo" does, so the number of "ISO 9001" is never compared. The words before an amount are read before its currency, so "at least USD 500", "more than US$500" and "up to A$500" are approximations, and a currency before either number of a range leaves it a range ("between USD 400 and 500", "between $400 and 500", "$400–$500"): neither is ever compared. A unit outside the reader's table reads as none, a code outside the list after a number ("500 PLN") and a unit word the table lacks ("500 hectares") among them, so a heard number without a unit pairs with such a value as with any other number without one.

The value reader (values, valuesEqual), the gates (checkable, and checkableStatement with nameWords), the sentence cuts (sentenceSpans), the verbatim span (exactSpan) and the sentence check (checkSentence, checkLabel) are also the entry @framers/agentos-ext-grounding-guard/values, which imports nothing and runs in a browser. The checker stems with the Porter stemmer of @framers/agentos/nlp; pass { stem } to the constructor to use another.

A stated fact without a number

"Joe Biden is the president." holds no number or date for the value check to compare. checkableStatement, claimProblem with names and a checker with names take such a claim, and the grader's quote gives code the words to compare:

import { ValueAgreementChecker } from '@framers/agentos-ext-grounding-guard';
import { checkableStatement, claimProblem } from '@framers/agentos-ext-grounding-guard/values';

checkableStatement('Joe Biden is the president.'); // true: "Biden" names something
checkableStatement('Coffee is great.'); // false

const fields = { focus: 'Joe Biden is the president.', heard: 'Joe Biden is the president', search: 'Joe Biden' };
claimProblem({ ...fields, statement: 'Donald Trump is the president.' }, { names: true }); // 'statement_names'

const checker = new ValueAgreementChecker({ names: true });
const source = {
  text: 'Joe Biden is an American politician who served as the 46th president of the United States from 2021 to 2025.',
  group: 'Joe_Biden',
};
const decision = checker.decide({
  heardWords: 'Joe Biden is the president.',
  heardSentence: 'Joe Biden is the president.',
  proposed: 'differs',
  named: [source],
  found: [source],
  quote: 'served as the 46th president of the United States from 2021 to 2025',
});
// decision.relation === 'differs'; decision.marks[0] marks the quote in the source's sentence

checkableStatement passes a sentence that may be checked and holds a number in digits, a month's name, or a word after its first that names something (nameWords: it starts with a capital and is not "I", a month or a stop word). A sentence capitalises its first word whatever it is, so "Einstein developed relativity." does not pass. checkable stays the gate for a number or a date.

claimProblem(fields, { names: true }) takes heard words that hold no value at all in place of a value when they hold a name the checker reads and a word of their own. claimNames reads the names: the words after the first that name something, with three letters or more, so "US", "UK", "EU" and "Xi" are not read and "the president of the US" is read as "the president". ownWords reads the claim's own words: its content words that are not capitalised ("president" in "Joe Biden is the president"), or, when it has none, a capitalised word standing alone right after "the" ("the President", as speech-to-text writes it), which is then a name as well. Heard words whose only values are never compared ("signed around 1918", "the 46th president"), and a claim the checker could never decide ("London is the capital of the UK", "Kamala Harris is the Vice President"), are refused with heard_no_value before any source is searched. A statement that drops or swaps a name heard is refused with statement_names, as a statement that changes a number is.

new ValueAgreementChecker({ names: true }) finds and decides the claim. passages keeps a sentence that shares two stems with the claim and holds every name heard after the first word. decide reads the grader's quote, copied from the named sentence:

A disagree is never decided without a value, and a claim with a value, compared or not, is decided on its values whether names are on or off: "The Treaty of Versailles was signed around 1918." stays undecided. Many real differences and matches stay undecided: a one-word name ("Canberra" for "Sydney"), a place given after a preposition alone, a verb whose forms stem apart ("wrote" and "written"), a title of two capitalised words ("the Vice President"), and the long form "Joe Biden is the president of the United States.", whose quote then names nothing the claim lacks, while "Joe Biden is the president." and "Joe Biden is the president of the US." are shown as different.

Two outcomes rest on the grader's proposal, since code reads neither whether a full name in the quote bears on the claim nor the quote's tense. A differs stands on any full name the quote holds that the claim lacks, one named in passing among them: "Paris is the capital of France." against the quote "the capital of France and its largest city, home to the Eiffel Tower" is shown as different when the grader proposes differs, and a quote that names "the United States" in passing does the same for a claim about an office. "Joe Biden is the president." against the quote "Joe Biden was the president" matches when the grader proposes matches (row N24). The quote's tense alone never shows a claim said in the present as different (row N24b): a fact that held only in the past is shown as different only when the quote also holds a number or date the reader compares, or a full name the claim lacks, as row N1's quote does through "United States". The value-agreement vectors pin these outcomes, so a change to any of them is made on purpose. Since 0.7.2 every decision says which branch decided it, basis: 'values' | 'quote': the rows of the names path, these two outcomes among them, come back quote, and a caller that shows a relation to anyone but the speaker can hold a quote decision back. A quote decision pairs no values: its pairs is empty.

A search phrase

phraseProblem holds a phrase a public source is searched with to the caller's limits:

import { phraseProblem } from '@framers/agentos-ext-grounding-guard/values';

const phrase = 'when did the city of lisbon first host the web summit conference';
phraseProblem(phrase, { maxWords: 12, maxChars: 120 }); // null: it may be sent
phraseProblem(phrase, { maxWords: 6, maxChars: 80 }); // 'search_too_long'
phraseProblem('Treaty/Versailles', { maxWords: 6, maxChars: 80 }); // 'search_characters'

A phrase may be sent when it holds 1 to maxWords words, counted as words counts them (every character that is part of no word separates two words, so "O'Brien" is two), at most maxChars characters, and letters, digits, spaces and .,'()-:& alone. Otherwise the answer is the first rule it breaks, in that order: search_empty, search_too_long or search_characters. claimProblem holds a claim's topic phrase to it at 6 words and 80 characters.

Verbatim quotes

verifyQuotes holds each quote to a verbatim span of the source it names:

import { fold, verifyQuotes } from '@framers/agentos-ext-grounding-guard';

const sources = [
  'The Treaty of Versailles was a peace treaty signed on 28 June 1919.',
  { content: 'Latency is 1.25 ms, and the treaty’s terms were “harsh”.' },
];

verifyQuotes([{ text: 'signed on 28 June 1919', sourceIndex: 0 }], sources);
// [{ found: true, sourceIndex: 0, start: 44, end: 66 }]

verifyQuotes([{ text: 'signed on 28 Ju', sourceIndex: 0 }], sources)[0].found;
// false: the quote cuts a word

const quote = { text: 'the treaty\'s terms were "harsh"', sourceIndex: 1 };
verifyQuotes([quote], sources)[0].found; // false: the source's quotation marks are typographic
verifyQuotes([quote], sources, { normalize: fold })[0].found; // true

The comparison is character for character and case-sensitive, in the source's text with its white space collapsed, and a quote that cuts a word or a number is not found. start and end are offsets into the collapsed source, both -1 when the quote is not found. A quote is looked for only in the source its sourceIndex names, and an index the list does not hold is not found. A source is a string or an object with a content string, such as a retrieved chunk. normalize reads both sides before they are compared and must keep one character for one, so the offsets hold; fold maps typographic quotation marks to their plain forms.

A line within a span

lineWithinSpan(line, span) checks whether a line may stand beside a quoted span using its own words:

import { lineWithinSpan } from '@framers/agentos-ext-grounding-guard/values';

const span = 'Q3 revenue was 1.2M, up 18 percent.';
lineWithinSpan('Revenue was 1.2M, up 18 percent.', span); // true
lineWithinSpan('Revenue was not up 18 percent.', span); // false
lineWithinSpan('Revenue was up 18 points.', span); // false
lineWithinSpan('Revenue was $1.2M.', span); // false

The line needs at least one word, and each word must occur in the span, compared in lower case. Negations must match by form and count. Each number must occur with the same sign and, when the line has a following word in the next 60 code points, the same following word. A number at the end of a line may omit that word. Numbers use NUMBER and numberOf, with ASCII digits; word membership still applies to their written digit groups.

Every %‰°+−±$€£¥×÷<>=≤≥≠ symbol and every other currency sign in the line must occur in the span. Words, numbers and symbols may be reordered or repeated, except that negation counts must stay equal. This is a lexical check, so it does not establish who did what; show the line beside its span.

The day a line names

whenSaid(text, { reference, timeZone }) reads the first day a text names, from the instant it was said and the IANA zone it was said in:

import { whenSaid } from '@framers/agentos-ext-grounding-guard/when';

// Tuesday 3 November 2026, 10:00 in Los Angeles.
const at = { reference: new Date('2026-11-03T18:00:00Z'), timeZone: 'America/Los_Angeles' };

whenSaid('Send the deck by Thursday', at); // { day: '2026-11-05', time: null, words: 'Thursday' }
whenSaid('next Monday', at); // { day: '2026-11-09', time: null, words: 'next Monday' }
whenSaid('in a week', at); // { day: '2026-11-10', time: null, words: 'in a week' }
whenSaid('tomorrow at 3 pm', at); // { day: '2026-11-04', time: '15:00', words: 'tomorrow at 3 pm' }
whenSaid('Friday at 3 pm EST', at); // { day: '2026-11-06', time: '12:00', words: 'Friday at 3 pm EST' }
whenSaid('Friday at 9 am get the deck to Sam', at); // { day: '2026-11-06', time: '09:00', words: 'Friday at 9 am' }
whenSaid('soon', at); // null
whenSaid('tomorrow', { ...at, timeZone: 'Asia/Tokyo' }); // { day: '2026-11-05', ... }: Wednesday 03:00 there

The text is read by chrono-node 2.10.3's English parser with forwardDate: true, from the zone's wall clock at reference. day is YYYY-MM-DD in that zone. time is HH:MM on a 24-hour clock when the text names a time and null otherwise. words is the span that named the day, without a comma or bracket around it, cut to WHEN_WORDS_MAX (60) characters.

• A result names a day only when chrono read a day or a weekday in it. A time alone ("at 3 pm"), a month alone and words with no date answer null.
• With several dates in the text, the first one that names a day is read.
• A weekday alone takes chrono's implied noon. Said on that weekday, it is that day until noon in the zone and the same weekday a week on after it.
• A time said with its own zone ("3 pm EST", "3 pm GMT+9") is given in the caller's zone, and its day can change with it. Once its instant has passed, a weekday with such a time moves a week on and a date with no year a year on, compared on true instants: "Thursday at 3 pm EST" said on a Thursday at 16:00 in Los Angeles is the next Thursday at 12:00.
• A zone is an abbreviation written in capitals or an offset. chrono takes the two to four letters after a time as a zone whatever their case, so "get", "eat" and "art" would read as GET (+4), EAT (+3) and ART (-3); whenSaid blanks such a word out and reads the text again, so "Friday at 9 am get the deck" is Friday at 09:00 and its words "Friday at 9 am". The word taken for the zone is the one whose blanking leaves the date as it was, the same hour and minute and a range still a range, so "Friday 9am to 5pm EST" keeps its zone (06:00 in Los Angeles). A zone written in lower case ("3 pm est") is read the same way, as no zone.
• A reference that is not a valid date, or a zone the runtime does not know, throws a RangeError.

chrono-node takes a reference zone as a minute offset or an abbreviation, and its forward step compares the zone's wall clock, read as the runtime's own clock, with an instant built from that offset. whenSaid reads the zone's wall clock itself and gives chrono that wall clock as a plain Date, so a server in UTC and a browser in the zone read the same words as the same day. When the runtime's own clock skips that wall clock (its daylight-saving gap, in a runtime whose zone is not the caller's), the gap's end stands for it: a time said inside the skipped hour is then not yet past. A runtime whose zone skipped the whole calendar day of the reference (Apia's clock skipped 30 December 2011) reads it from the next day, so "tomorrow" there counts from that next day. The entry @framers/agentos-ext-grounding-guard/when imports chrono-node and nothing else, and runs in a browser. whenSaid is also exported from the main entry.

Sentence check

checkSentence holds a line, such as a summary's sentence or an answer's, to the words it quotes; checkLabel holds a title or a tag to the same terms:

import { checkLabel, checkSentence } from '@framers/agentos-ext-grounding-guard/values';

const turn = 'Yes, we grow the budget by ten percent next quarter.';
const quotes = [{ quote: 'we grow the budget by ten percent', source: turn }];

checkSentence('We grow the budget by ten percent.', quotes); // null: the line stands on its quote
checkSentence('Cut the budget by 20 percent.', quotes); // 'beyond_quotes': "cut" is not the quote's
checkSentence('We do not grow the budget by ten percent.', quotes); // 'negation_differs'
checkSentence('We will not grow the budget by ten percent.', quotes); // 'qualifier_not_quoted': "will"

checkLabel('Budget', quotes); // null
checkLabel('Acme budget review', quotes); // 'name_not_quoted'

Each quote is at least minQuoteChars characters long (20 when not given) and stands in its source: its words in order and next to each other, in any letter case and spacing, with the source's own symbols (<, =, % and the like) between them, while quotation marks, emphasis marks, commas, stops, brackets and dashes may differ. A quote also keeps every negation of the sentence it stands in: "the gateway supports IPv6" cut out of "The team never confirmed that the gateway supports IPv6." is quote_drops_negation. Where a quote stands more than once, its first stand that keeps them is read. A blank line ends a source's sentence; a single line break does not, since wrapped text breaks mid-sentence.

Each sentence of the line then needs content words, and at least support of them (0.7 when not given) are words of the quotes, compared without their endings ("-ing" past five characters, "-ed" and "-es" past four, "-s" past three but not "-ss") or, in a script written without spaces (isUnspacedScript), by neighbouring pairs of characters (letterPairs). The term checks come last, in this order:

checkSentence answers the first problem found, a SentenceProblem, or null when the line stands. checkLabel runs the term checks alone: it asks no share of content words and no length of a quote, and each quote still stands in its source and keeps its sentence's negations. A label's negations equal its quotes', so "approved" never stands on "not approved"; hold a label to the quotes it rests on, since a negation of any quote passed counts against it. With no quotes, a label passes only when it holds no name, symbol, number, qualifier or negation.

The checks are lexical: a line that swaps who did what can pass them, so show each line with its quotes.

NLI

createNliFunction() scores one pair, premise first, on a local model, in the shape CitationVerifier's nliFn takes:

import { CitationVerifier } from '@framers/agentos';
import { createNliFunction } from '@framers/agentos-ext-grounding-guard';

const nli = createNliFunction();

const answer = await nli('A man is eating pizza', 'A man eats something');
// answer.label: 'entailment', 'contradiction' or 'neutral'
// answer.score: that label's score; answer.scores: all three, which sum to 1

const verifier = new CitationVerifier({ embedFn, nliFn: nli }); // embedFn: your batch embedding function

await nli.dispose();

The premise is the source and the hypothesis is the claim, the order the model card's examples use. The pair goes to the model as the tokenizer's text and text_pair, and the labels come from the model's own id2label. A premise that takes the pair past the model's window (512 tokens for the default model, the pair's special tokens counted) is cut by tokens at its end, so the whole hypothesis stays in the pair: transformers.js cuts a pair that is too long at its tail, which would drop the claim. transformers.js (@huggingface/transformers, an optional dependency) is imported and the model loaded when the first pair is asked: the first load downloads the model's files, and later loads read them from transformers.js's cache. A load that fails is not kept, so the next pair loads again. dispose releases the model, and resolves with nothing to release after a failed load. GroundingChecker and the guardrail score every claim through the same function, with the source chunk as the premise.

Examples

• test/ — claim-extraction and entailment scoring fixtures

Lazy loading and optional install

This package is an optional dependency of @framers/agentos-extensions-registry. The registry ships catalog metadata; createCuratedManifest() calls import.meta.resolve() per entry and silently skips anything not installed. npm install @framers/agentos-ext-grounding-guard is the gate.

The NLI model does not load at activation. The checker keeps one NLI function under the agentos:grounding:nli-pipeline key in SharedServiceRegistry, and transformers.js and the model file load only when the first claim is checked against a source. The claim extractor runs first against extracted claims; only claims that pass extraction reach NLI.

GuardrailOutputPayload carries ragSources?: RagRetrievedChunk[] when the response was generated from RAG context. The grounding guardrail uses these as the entailment-source set; without them, the claim extractor still runs but per-claim entailment falls back to whatever passages the host provides.

Streaming checks are on by default (enableStreamingChecks): the guardrail registers with config.evaluateStreamingChunks = true, checks each completed factual sentence against the sources as the response streams, and checks every claim of the whole response at the end. Set enableStreamingChecks: false to check the final response alone. It runs in Phase 2 of the two-phase dispatcher. Below-threshold claims return FLAG by default; configure contradictionAction: 'block' to escalate to BLOCK.

For the full DI model and end-to-end walkthrough, see How extensions stay optional and lazy and the auto-loading guide.

License

Apache 2.0 — see the repo root LICENSE.