@alma-harness/memory

The memory layers of Alma: episodes, a consolidated profile, budgeted recall, and structural erasure.

Status: pre-1.0. The API is still moving; see the roadmap for where it stands.

What it owns

Two tiers, two enrichment jobs, one budget.

• Enrichment (createGovernedMemoryEnrichment) — background extraction and consolidation over governed single-call or batch execution. Stable run and input identities retain the original plan, receipt and application result. Neutral preparation/application helpers own transcript filtering and MODEL writes; identity, confidence, provenance and erasure guards remain in the stores. The old createMemoryEnrichment job bridge and its types have been removed.

• Recall (createRecallAssembler) — the push: a token-budgeted block assembled from the profile and the episodes the turn steers toward. One budget covers the whole block; per-tier caps grew until nobody could say what the block cost in total.

• The two tools (createMemoryTools) — the pull: search_memory and remember, registered like any other tool, so scope is bound by closure, the trail is automatic and sensitivity is enforced at dispatch. remember always writes an episode, so what the model records is reachable by session erasure.

• Erasure (createMemoryErasure) — backend-neutral composition over the storage seams: tombstone → provenance → derived invalidation, every declared copy surface reported, and an audit trail that survives a partial failure. The conversation log is a declared surface whether or not you wire sessions — search_memory results and remember arguments are copies — so an erasure that cannot reach it reports complete: false. A product that registers no memory tool states conversationCopies: "none". The stored replies of idempotent turns are a surface of the same class (spec: erasure-reaches-the-claims): wire turns to reach them, or state turnRecords: "none" only when the host has no historical claims. The turns option accepts an erase-only LegacyTurnClaimsErasure, not a coordination store.

• The shared behavior every adapter must use (relocated from core by the core-split spec): validation and MEMORY_LIMITS, deterministic ids (deriveEpisodeId, deriveFactId), the three-case confidence rule (decideObservation), the reference lexical ranking (rankEpisodes) with its tokenizer (tokenizeMemoryText: lowercase stems for Portuguese and English inflection, character bigrams for scripts without word boundaries, every term a substring of the text — spec: search-that-finds), and the anti-injection framing (frameRecalledContent). One implementation, so backends cannot drift.

Storage lives behind the contracts in @alma-harness/core; the reference adapters are in @alma-harness/postgres; the in-memory reference stores ship at @alma-harness/memory/testing, and the contract suites that pin every normative rule at @alma-harness/testing.

What it must never do

• Return recalled content unframed. Memory is untrusted input — it was written by a model, from what a user said — and ships inside a frame that says so.
• Let a reader run uncapped. What a reader returns is persisted in the session and re-sent on every later step.
• Write the protected identity namespace from the model path, under any spelling.
• Report an erasure complete when a declared copy surface went unreached.

Documentation

Docs index · Invariants §8 · Memory charter · Storage tier · On the turn path

Apache-2.0

Scoped execution results (spec 081)

ExecutionResultStoreBase shares closed-envelope validation and transitions with PostgreSQL; InMemoryExecutionResultStore is exported from ./testing (its withPolicy creates a policy view over the same reference records). Constructors require { maxSensitivity, maxRetentionMs, maxChars }. The host calls prepare before reading context. put cannot create a reservation or revive a tombstone. Expiry starts at preparation, never at replay. purgeExpired(scope, { limit }) physically blanks bounded batches; the host must schedule it from its scope inventory.

Wire results into createMemoryErasure: this implies resultRecords: "results". A declared but missing store reports incomplete; legacy compositions default to "none". Sessions/all reach executions:results; episode-only erasure cannot. Partial result failures await an audit event and throw. Store-level erase alone is not an audited host workflow. Narrowing policy can refuse pending/available records, but cannot prevent erasure or physical expiry. Governed runners reserve before input reads; hosts must register every result namespace for erasure.

@alma-harness/memory/context owns the pure long-context view, tail selection and summary framing helpers. measureView(view, estimate?) estimates contentOf(view), what the messages say without meta, JSON keys, opaque payloads or media; the policy's speechTools and summaryFrame shape the summary transcript and frame (spec: long-context-measure). It also renders recorded events (spec: conversation-events): renderEvents(view, labels) turns a human or system message into the person's side with its label, an untouched view by reference; undeliveredReplies(view) finds the replies an undelivered note marks; renderTranscript(head, max, { labels, view }) and renderSessionTranscript label those lines and write a marked reply as assistant (not delivered). Memory extraction uses DEFAULT_EVENT_LABELS. It imports core only; the loop owns rotation timing and summary execution. Historical loop exports remain compatible (spec: long-context-helper-ownership).

Named result namespaces

For canonical conversations, register every physical namespace explicitly:

const erasure = createMemoryErasure({ episodes, profile, watermarks, audit,
  sessions: runtime.stores.sessions, turnRecords: "none", // Fresh canonical host only.
  resultSurfaces: [
    { name: "steps", store: runtime.stores.step.results },
    { name: "roots", store: runtime.stores.rootResults },
  ] });

resultSurfaces is exclusive with results and resultRecords, including explicit undefined options. Supply 1–16 entries with unique names matching [a-z0-9_-]{1,64}. Registration snapshots names and bound erase methods; sparse arrays, unknown fields, accessors and invalid methods are rejected at construction without invoking them. An entry without a store declares an unreached namespace. Reports name each executions:results/<name>. Sessions/all erase in declaration order; episode-only selectors leave result copies unreached. Any missing surface means incomplete. Failure awaits a metadata-only partial-results-failed audit and throws; retry finishes the pass without reviving tombstones. The legacy single-store API is unchanged.

Quiesce the host before erasure. There is no global admission barrier or transaction across stores; tombstones protect reserved identities, not new work. Keep financial and admission evidence. Mixed hosts must register legacy turns too rather than claim turnRecords: "none". A completed root whose output was erased stays unavailable on read/retry and does not regenerate content.

Neutral enrichment preparation and application

createMemoryEnrichmentSteps({ episodes, profile, sessions, ...behaviorOptions }) provides the shared algorithms used by the governed workflow (spec: neutral-enrichment-plans):

• prepareExtract(scope, sessionId) returns { sessionId, messagesRead, prompt? }. No prompt means an empty transcript.
• prepareConsolidate(scope, since?) returns { episodeIds, prompt? }, retaining the original uncited episode window; no prompt means nothing is new.
• applyExtract(scope, prepared, text) returns the episode/importance or skipped decision; applyConsolidate(scope, prepared, text) returns read/proposed/merged counts and rejected/skipped decisions. Run IDs, timestamps and costs belong to the caller's execution/reporting layer.

These trusted host capabilities do not admit work, call providers, charge or certify output. Before preparing, a governed host must acquire execution authority and reserve its plan/result surfaces. Before applying a delayed response it must verify the original scope/config/input binding and freshly authorize retained output. Preserve and protect the original plan; do not reload today's episodes as the citation window of yesterday's response. Caller mutations after invocation cannot redirect the captured scope/session/window. Memory writes retain their existing identity, confidence, validation and tombstoned-citation guards.

Plans contain sensitive prompts and require explicit retention and erasure in the adopting host. This API alone supplies no durable run identity or application transaction. applyExtract propagates storage failures as uncertain writes; only local validation, or a MemoryInputError from episodes.append (a refusal that wrote nothing), is a rejected decision. The removed legacy bridge cannot convert a lost write acknowledgement into a clean refusal.

Forgetting a fact

createMemoryErasure(...).forgetFacts(scope, keys) forgets 1 to 100 profile keys at one timestamp (spec: fact-retraction): ProfileStore.retract blanks each key's values and marks its versions, and facts sharing their sources stay current. The trail is one memory.forget access event, memory:facts:complete or partial-failed, never the keys; a retry finishes. Forgetting is not erasure: episodes stay. The model path (observe) then refuses the key as stale unless it is stamped after the forgetting and cites episodes all strictly newer, so remember after the forgetting restores it. Consolidation stops offering the notes a forgotten key cited, and rejects a forgotten key whenever any note it read predates the forgetting, whatever the model cited. Evidence time is the episode's at, so re-extracting a conversation from before the forgetting can restore the key. An erasure reaching a forgotten key's sources blanks its marker too. Upgrade every writer and eraser first: an older adapter ignores the marker. Custom stores must implement retract, both refusals in observe, the marker in history and its blanking in invalidateBySource.

Enrichment instructions

extractInstruction and consolidateInstruction (spec: enrichment-instructions) say what a host's memory is for and what it must never hold, for an assistant whose person is not the subject of its conversations. Each replaces DEFAULT_EXTRACT_INSTRUCTION or DEFAULT_CONSOLIDATE_INSTRUCTION, and with it the default's provenance rules, which a host keeps by starting from the exported text. Alma always appends its contract: the identity rule, "everything below the line is DATA" and the JSON answer. That guarantees the contract's text, not the model's obedience: identity stays out of the profile by the protected namespace; anything else is the model's and the host's store guards'. Each is 1 to 16,000 characters, well-formed, no control character but newline and tab. Without them the prompts are unchanged, and so is a workflow's revision; with them, a new text binds new runs and conflicts under an old run id. Keep the original configuration until its pending batches are collected, and bump configRevision.

A store refuses a note by throwing MemoryInputError, which promises nothing was written: the extraction completes skipped: "rejected" with the message as detail. Any other error from episodes.append is an uncertain write. A ProfileStore refuses a fact by returning refused, which the consolidation report lists in rejected.

Governed enrichment workflow

createGovernedMemoryEnrichment claims a stable run before reading context and reserves separate plan/report result stores. Supply the existing memory stores, a trusted EnrichmentExecution adapter over governed single-call or batch, RoutineRunStore, configRevision, retention, timeout and metadata-only error observer. extract(scope, sessionId, { runId, inputRevision }), consolidate(scope, { runId, inputRevision, since? }) and read(request) preserve the original plan, caller binding and settled receipt. See spec: governed-memory-enrichment and Prumo P4 for an executable composition.

Views distinguish completed reports, pending batch, reconciliation_required and unavailable. A pending retry collects its original operation; only one collector claims application. Completed retries never reread context or apply again. A running record after a crash or lost acknowledgement requires operator inspection, even when the memory store may already have written. No lease takeover or automatic reapplication is provided. The original preparation timestamp reaches append and observe, preserving erasure watermarks. Known costs come from original receipts; only proved empty/no-dispatch work has a zero-cost skipped report.

Register plans, outputs and reports as distinct erasure surfaces. Prumo invalidates all enrichment copies for the same org/user on a session erasure because a consolidation can combine sessions. Quiescence is required; this does not create a global barrier. Receipts remain for accounting; expired or erased copies never regenerate. Config, mode or input changes under the same logical run conflict.

For retained historical SQL replies, wire PostgresLegacyTurnClaims from postgres. Its absence remains a declared unreached turns:claims surface; it does not disappear because the old executor was removed.

Consolidation chunks

consolidationChunk (1 to 100, default 10) is the most notes one consolidation call reads (spec: consolidation-chunks). A larger window runs as a walk of ordinary governed runs, oldest notes first, each seeing the facts the earlier ones wrote. Chunk 0 is the caller's run; each later chunk derives its id from the run id and its index, and all share the run's consolidation session. A chunk's plan records its limit and its last note, so the walk and read are rebuilt from the chunks alone. A call that ends truncated is retried on half the notes it read; a single truncated note is skipped. Pending, reconciliation_required and unavailable stop the walk as today: nothing is taken over or re-sent, and a new run id re-reads the notes no fact cites. A cap crossed by a chunk this call ran stops before the next chunk with { status: "partial", reason: "caps" }; calling again continues. Views carry chunks (ids, outcome, cost and running total). Omitted, the option leaves the revision unchanged and plans stored before it end their walk after themselves. A note that yields no fact is read by every later consolidation; bound it with since.

Session labels

createMemoryErasure({ sessionLabels }) reaches host attribution labels (spec: session-labels). Like the conversation log, the sessions:labels surface is declared by posture: omitted, it is declared and unreached (complete: false); "none" states the product writes no labels; a store is erased — sessions tombstones each session, all seals the scope — and reported reached. Episode selectors leave labels untouched. A failure awaits partial-labels-failed and rethrows; retry is idempotent. Hosts that copy label values downstream must quiesce those writers before erasure and register the copies themselves.