@ariestools/aries-datalake-client

REST clients for the Aries datalake control and data planes, with Node-only local development helpers kept behind the Node entry point.

Entry points

• @ariestools/aries-datalake-client is the Node entry point. It includes the REST clients plus local filesystem-backed clients, credentials, defaults, and ARIES_HOME helpers.
• @ariestools/aries-datalake-client/browser is the browser entry point. It exports REST clients, the composite client, their interfaces, the asynchronous token-supplier contract, and browser-safe datalake wire contracts.

Use an asynchronous token supplier that is invoked for each request. The application adapter may return a still-valid short-lived token from page memory or request a fresh wallet JWT through the XL1 browser wallet gateway:

import {
  RestDatalakeClient,
  RestPayloadsClient,
} from '@ariestools/aries-datalake-client/browser'

// Application-supplied adapter around the installed XL1 wallet gateway.
// The wallet derives and signs the browser origin; the page does not supply it.
declare function requestWalletJwt(audience: string): Promise<string>

const authToken = async ({ audience }: { audience: string }): Promise<string> => {
  return await requestWalletJwt(audience)
}

const control = new RestDatalakeClient({
  baseUrl: 'https://control.example',
  authToken,
})

const payloads = new RestPayloadsClient({
  baseUrl: 'https://data.example',
  datalakeId: 'dl_example',
  authToken,
})

The supplier receives the exact control-plane audience or datalake id. Treat its return value as a sensitive, short-lived bearer token: cache it only in page memory when useful, and never persist it in browser storage or logs. The wallet gateway controls key access; the Aries HTTP services independently verify audience, signature, origin, scope, signer identity, and live ACL state.

Bounded exact-root archive

ExactRootArchive is available from both entries. Supply an explicit native SDK root-identity callback, configured hash domain and lake, a trusted append-only issuer, and all six positive finite limits: root count, request bytes, decoded response bytes, token bytes, whole-operation milliseconds and concurrent work. The client snapshots JSON bodies, verifies native roots, sends the native insert request, then independently resolves every required root through anonymous readback with credentials omitted and redirects rejected. Duplicate receipts never replace readback. No write cache, queue, automatic retry or recovery journal is introduced.

For the opt-in service-wallet profile, the issuer signs a JWT with the lake id as its exact audience and payloadScopes: ['payload:append'], without browser origin/scope or record/blob claims. The data plane must explicitly enable service wallet JWTs and the signer must hold a live runner grant. A live public-viewer grant independently permits readback. Tokens without payloadScopes retain legacy service authority and do not qualify this restricted profile.

ExactRootArchiveError reports a secret-free code and writeState: not-started, uncertain after dispatch, or acknowledged after an HTTP success. Every public failure is a fresh error with a validated code and captured write state; callback messages, causes and private receipts are discarded. A caller timeout does not undo storage. Pending callbacks/effects retain the concurrency slot until they settle, even after the caller receives cancellation. Late response bodies are cancelled, and native stream cancellation cleanup is awaited before that slot is released; hanging cleanup keeps capacity occupied. Byte limits count UTF-8 JSON requests and decoded response streams, including errors; raw network/decompression/header/chunk allocation limits belong to the trusted host transport. The trusted issuer separately bounds its own transport, returns no logged/persisted credentials and honors the supplied abort signal. The identity callback preserves client/application fields and uses the owning SDK projection for storage metadata; claimed hashes are never proof.

Multiple required datalakes

CompositePayloadsClient writes each target's eligible subset and verifies it by reading the content back. Configure a complete S3 collection alongside a selected Auto Drive collection using already authenticated RestPayloadsClient instances:

import { CompositePayloadsClient, type RestPayloadsClient } from '@ariestools/aries-datalake-client/browser'
import { asAnyPayload, PayloadBuilder } from '@xyo-network/sdk'

declare const autoDriveClient: RestPayloadsClient
declare const s3Client: RestPayloadsClient

const composite = new CompositePayloadsClient({
  identify: async (payload) => {
    const canonical = asAnyPayload(payload, true)
    return {
      hash: await PayloadBuilder.hash(canonical),
      dataHash: await PayloadBuilder.dataHash(canonical),
    }
  },
  targets: [
    {
      name: 'auto-drive',
      client: autoDriveClient,
      policy: {
        mode: 'selected',
        allowedSchemas: ['com.example.smallrecord'],
        maxPayloadBytes: 4096, // Example deployment limit, not a provider default.
        revision: 'v1',
      },
    },
    { name: 's3', client: s3Client, policy: { mode: 'all' } },
  ],
})

const result = await composite.insertWithReceipts([
  { schema: 'com.example.smallrecord', value: 1 },
  { schema: 'com.example.other', value: 2 },
])
const read = await composite.getWithReceipts(result.acknowledged.map(payload => payload._hash))

All targets are required for their eligible subset. A selected policy requires an explicit allowlist and positive byte limit; an empty list selects nothing. disallowedSchemas takes precedence. schemaMaxPayloadBytes may tighten the global byte limit, and isValid may supply a trusted structural validator. Sizes count UTF-8 JSON including client metadata and excluding storage metadata. The receiving service must enforce its policy independently; this client does not authorize permanent uploads or implement service quotas.

insert(payloads) returns the verified acknowledgment array, including duplicates; insertWithReceipts(payloads) also returns per-target eligibility and outcomes. A rejected expected-eligible item or failed required target throws CompositePayloadsWriteError. Its result retains verified partial success and sanitized receipts, not raw provider errors. A complete target's success does not satisfy a different target's obligation. Callers own durable retry state and policy-revision coordination; this client does not resume a failed operation.

get(hashes) and getMany(hashes) return verified payload arrays. getWithReceipts(hashes) adds source attribution. Reads try targets in declared order for unresolved hashes only. Provider errors or corrupt responses throw CompositePayloadsReadError; they are not silently converted into misses. Reads never copy content between targets. Supply canonical XYO PayloadBuilder hashes through identify; the composite package itself adds no XYO runtime dependency to browser consumers.

Only flat payload bodies are accepted. Validate and flatten known hydrated protocol envelopes before insertion. There is no global sequence, clear, delete, or usage API and no automatic registration into an XL1 connection. The payload-array insert/get methods are intended for a protocol adapter; keep the SDK's exact transaction-content acknowledgment check in that adapter. Successful read-back is readable storage, not proof of provider network archival.

Explicit public payload reads

Use RestPublicPayloadsReader when the provider has granted public viewer access to the datalake. This is separate from the authenticated clients above; an authentication failure never triggers an anonymous retry.

import { RestPublicPayloadsReader } from '@ariestools/aries-datalake-client/browser'

const reader = new RestPublicPayloadsReader({
  baseUrl: 'https://data.example/plane/v1/datalakes/dl_public',
  datalakeId: 'dl_public',
})
const controller = new AbortController()
const payload = await reader.get('configured-content-hash', {
  maxResponseBytes: 65_536,
  signal: controller.signal,
})

The reader exposes only get(hash, options?) and getMany(hashes, options?). It has no token supplier, header hooks, listing or mutation methods. Requests omit ambient credentials and reject redirects. Configuration accepts absolute HTTP(S) URLs, including a shared-host path prefix, and rejects userinfo and fragments. Local HTTP is permitted explicitly; address/DNS policy remains the host's responsibility. An optional fetchImpl is a trusted transport override that must retain those policies. Without it, fetch is resolved at request time.

maxResponseBytes is an optional positive safe integer. It counts decoded Fetch response-stream bytes before JSON parsing, on success and error responses, including the whole native object/array and its metadata. Omission means no size limit. It does not bound raw compressed traffic, headers, decompression work or connection buffers. The signal is forwarded to fetch and remains active during body consumption; aborting does not wait indefinitely for stream cleanup.

Successful reads validate the stored-payload shape and preserve all fields. Neither requested paths nor _hash/_dataHash metadata verify content identity; callers that need content assurance must hash and validate the returned data. Transport, size, cancellation, malformed-response and access failures remain failed reads, not evidence that an advertised object never existed.

Public viewer access applies to the whole datalake, including provider-side enumeration and usage APIs. Prefer a dedicated public viewer lake with authenticated writers; public runner access also permits mutations. New lakes are private until granted access, and browser access additionally requires the provider's CORS configuration. This client does not change ACLs or retention.

The public reader requires the released @ariestools/sdk 8.2.0 or a compatible 8.2 patch, plus its required @opentelemetry/api peer. It does not require zod for these reads; other SDK features and datalake peers can have separate needs.