BEAM SDK for TypeScript

This release requires Core's transfer-client-control/v7 contract. Multipart uploads use consecutive parts; Core may request retained staging for recovery through the existing SDK signing connection. Workers receive ordinary upload URLs. Keep the client connected until the transfer reaches a terminal state so Core can renew grants, promote recovered data, and clean up staging. Drain active transfers before upgrading Core and SDK consumers.

Install:

npm install @beam-network/sdk

Example:

import { BeamClient, R2ProviderConfig, S3ProviderConfig } from "@beam-network/sdk";

const beam = new BeamClient({
  apiKey: "b1m_your_key"
});

const transfer = await beam.createTransfer({
  sources: [
    R2ProviderConfig.create({
      bucket: "source-bucket",
      key: "exports/report.parquet",
      account_id: "cloudflare-account-id",
      access_key_id: "r2-access-key",
      secret_access_key: "r2-secret-key"
    })
  ],
  destinations: [
    S3ProviderConfig.create({
      bucket: "destination-bucket",
      key: "imports/report.parquet",
      region: "us-east-1",
      access_key_id: "aws-access-key",
      secret_access_key: "aws-secret-key"
    })
  ],
  name: "r2-to-s3-report"
});

const status = await beam.waitForTransfer(transfer.transfer_id);
console.log(status.status);
await beam.close();

The main createTransfer API is provider-aware and strictly typed for S3, R2, S3-compatible, Hippius, Hugging Face and Box configs.

Storage Credentials

Source and destination credentials must not be restricted to specific IP addresses or networks. When storage refuses access, waitForTransfer rejects with BeamStorageAccessError (code is source_access_denied or destination_access_denied).

Credit And Key Errors

A refused lifecycle request rejects with a LifecycleRequestError (status, code, detail). Credit, billing, key and permission refusals use these subclasses:

• InsufficientCreditsError (402, insufficient_credits): creditsRequired and creditsAvailable (numbers, or null when unknown) and reason (pool_exhausted, key_cap_exhausted, key_budget_blocked, or null).
• BillingAccountNotLinkedError (402, billing_account_not_linked): the API key has no billing account.
• ApiKeyInvalidError (401 or 403, api_key_invalid): the API key is unknown, expired, revoked or disabled.
• PermissionRequiredError (403, permission_required): permission names the permission the key needs.

The optional onWarning callback receives advisories attached to lifecycle replies, such as credits_low when the credit balance is at or below the low threshold (detail.remaining, detail.low_threshold). Each distinct warning message is delivered once per client.

Billing Parent

billingParent names the billing operation a transfer belongs to, such as the workflow run that starts it; Beam records the transfer's charge under that operation. createTransfer, prepareProviderTransfer, resumeProviderTransfer, createRawTransfer and prepareTransfer accept it. A billing parent is 1 to 256 letters, digits and . _ : @ / + = -; the client rejects any other value before sending a request.

Signed URL Lifetime

createTransfer, prepareProviderTransfer and resumeProviderTransfer sign transfer URLs for expiresIn seconds. Without expiresIn, the lifetime is one hour plus one second per 100 MB delivered (source bytes × destinations), at most seven days, as signedUrlLifetimeSeconds(deliveredBytes) returns. The source URLs sent with transfer.prepare are signed before the sizes are known, for expiresIn or one hour.

S3-Compatible Providers

Use S3CompatibleProviderConfig for S3-compatible providers with custom endpoints, such as Wasabi, MinIO, Backblaze B2 S3 API, DigitalOcean Spaces, Scaleway Object Storage, and self-hosted S3-compatible systems.

import {
  BeamClient,
  S3CompatibleProviderConfig
} from "@beam-network/sdk";

const beam = new BeamClient({ apiKey: process.env.BEAM_API_KEY! });

await beam.prepareProviderTransfer({
  sources: [
    S3CompatibleProviderConfig.create({
      provider: "wasabi",
      bucket: "my-bucket",
      key: "input/file.bin",
      region: "us-east-1",
      endpoint_url: "https://s3.us-east-1.wasabisys.com",
      access_key_id: process.env.WASABI_ACCESS_KEY_ID!,
      secret_access_key: process.env.WASABI_SECRET_ACCESS_KEY!
    })
  ],
  destinations: [
    S3CompatibleProviderConfig.create({
      provider: "minio",
      bucket: "archive",
      key: "file.bin",
      endpoint_url: "https://minio.example.com",
      force_path_style: true,
      access_key_id: process.env.MINIO_ACCESS_KEY_ID!,
      secret_access_key: process.env.MINIO_SECRET_ACCESS_KEY!
    })
  ],
  name: "S3-compatible transfer"
});

Use S3ProviderConfig for ordinary AWS S3 if preferred, and keep using R2ProviderConfig for existing Cloudflare R2 code. Use S3CompatibleProviderConfig when a provider needs a custom S3-compatible endpoint.

Hugging Face Hub

Use HuggingFaceProviderConfig to pull a file out of a Hub repo or push one into one. The token stays in this process: a source resolves to the presigned CDN URL the Hub redirects to, and a destination uses the presigned part URLs the Hub's LFS batch endpoint issues.

import { BeamClient, HuggingFaceProviderConfig, S3ProviderConfig } from "@beam-network/sdk";

const beam = new BeamClient({ apiKey: process.env.BEAM_API_KEY! });

const transfer = await beam.prepareProviderTransfer({
  sources: [
    HuggingFaceProviderConfig.create({
      repo_id: "org/dataset",
      path: "data/train.parquet",
      repo_type: "dataset",
      token: process.env.HF_TOKEN!
    })
  ],
  destinations: [
    S3ProviderConfig.create({
      bucket: "archive",
      key: "datasets/train.parquet",
      access_key_id: process.env.AWS_ACCESS_KEY_ID!,
      secret_access_key: process.env.AWS_SECRET_ACCESS_KEY!
    })
  ],
  name: "Hugging Face pull"
});

// waitForTransfer commits any Hugging Face destination once the parts land.
await beam.waitForTransfer(transfer.transfer_id);

Constraints, all of them the Hub's rather than Beam's — see HUGGINGFACE_PROVIDER.md for the full protocol:

• Sources are large-file (LFS or Xet) content. Only those get the credential-free CDN redirect; a small regular file is served inline and is rejected with that reason. Hub buckets work as sources too, but cannot be written to — they expose no LFS endpoint.
• A destination needs the object's sha256 up front, because the LFS batch endpoint will not issue upload URLs without it. When the source is not itself a Hub file, set allow_source_rehash: true to let the SDK read the source once to compute it.
• One source per Hugging Face destination, because the Hub dictates the part size and a plan carries a single chunk size.
• If you do not call waitForTransfer, call finalizeHuggingFaceUploads(transferId) yourself once the transfer completes — without it the parts are uploaded but no commit is made and the file does not appear in the repo.

Box

Use BoxProviderConfig to transfer files out of Box, and BoxDestinationConfig to write into a Box folder. BOX_PROVIDER.md describes both protocols.

Box as a source

The access token stays in this process: the SDK looks the file up, asks Box for its content with redirects off, and hands BeamCore only the pre-authorised download URL Box redirects to. Workers get that URL and a Range header, never the token.

import { BeamClient, BoxProviderConfig, R2ProviderConfig, boxFolderSources } from "@beam-network/sdk";

const beam = new BeamClient({ apiKey: process.env.BEAM_API_KEY! });

// Every file under a folder ("0" is the root folder). For one file, pass
// BoxProviderConfig.create({ file_id: "1234567890", token }) instead.
const sources = await boxFolderSources({ folder_id: "9876543210", token: process.env.BOX_ACCESS_TOKEN! });

const transfer = await beam.prepareProviderTransfer({
  sources,
  destinations: [
    R2ProviderConfig.create({
      bucket: "archive",
      key: "box-export/",
      account_id: process.env.R2_ACCOUNT_ID!,
      access_key_id: process.env.R2_ACCESS_KEY_ID!,
      secret_access_key: process.env.R2_SECRET_ACCESS_KEY!
    })
  ],
  name: "Box pull"
});

• Each file is read at one version. Preparing a source records Box's current file version as version_id in the source metadata, with file_id, sha1, name and Box's item etag as box_etag. Every download URL the transfer resolves, re-signed routes and integrity audit reads included, asks Box for that version.
• One Box call per file every few minutes, not one per chunk. Every chunk reads the same download link with its own Range. The SDK resolves a link once, shares it across chunks and concurrent signers, and resolves a fresh one only when less than 11 minutes (BOX_DOWNLOAD_URL_MIN_REUSE_SECONDS) remain. Links are cached in memory by file version and a SHA-256 of the token, never the token itself, up to 1,024 of them.
• Download links last about 15 minutes. Routes claim 14 minutes, so BeamCore asks the SDK to re-sign them during a longer transfer, and a re-sign resolves a fresh link for the same version.
• The SDK never refreshes the token. It resolves every download link, so it must stay valid for as long as the transfer needs routes. A rejected token raises BoxApiError with status 401 and the message "Box access token expired or invalid".
• Folders. boxFolderSources recurses into subfolders, follows Box's marker paging, and sets each source's path to the file's path inside the folder, so a destination key ending in / receives the folder's layout. Zero-byte files are skipped. A single file is named after its Box name unless you set path.
• Retries. A 429, and a 202 for a file Box is still processing, is retried after Retry-After, up to 5 attempts; aborting prepareProviderSource's signal stops the wait. Any other failure raises BoxApiError naming the status and Box's error code.
• File size. Box's free plan caps files at 250 MB; larger files need a paid plan.

Box as a destination

A BoxDestinationConfig writes each source as one file in a Box folder, through Box upload sessions. Workers upload parts straight into the session with file-scoped upload tokens the SDK mints; BeamCore verifies and commits the session with a separate control token. The user's token stays in this process: token_provider returns it for every Box call that needs it, and the SDK never sends it to BeamCore.

import { BeamClient, BoxDestinationConfig, BoxProviderConfig } from "@beam-network/sdk";

const beam = new BeamClient({ apiKey: process.env.BEAM_API_KEY! });

const transfer = await beam.createTransfer({
  sources: [BoxProviderConfig.create({ file_id: "1234567890", token: process.env.BOX_ACCESS_TOKEN! })],
  destinations: [
    BoxDestinationConfig.create({
      folder_id: "9876543210",
      token_provider: async () => boxOAuth.currentAccessToken(),
      // Or revoke_token: async (token) => ..., to revoke minted tokens yourself.
      client_id: process.env.BOX_CLIENT_ID!,
      client_secret: process.env.BOX_CLIENT_SECRET!
    })
  ],
  name: "Box to Box"
});

// Keep the client open until the transfer is terminal: BeamCore asks it to renew, revoke,
// finalize and abandon the Box session.
await beam.waitForTransfer(transfer.transfer_id);
await beam.close();

• Placement. A file of the same name in the folder receives a new version; otherwise the SDK creates an empty placeholder, which it deletes again if the transfer fails. file_name names the file for a single source; without it each file is named after its source.
• Sizes. A source of 20 MiB or more is uploaded through a new-version upload session in parts of the size Box chooses (8, 16 or 32 MiB); a smaller one is one chunk written with a single form upload; an empty one is written by the SDK directly. A transfer cannot mix the first two kinds, or session sources Box gives different part sizes: split them into separate transfers (BoxDestinationError with code box_mixed_upload_kinds or box_part_size_mismatch).
• Trusted digest. Box requires the whole-file SHA-1 to commit a session. The SDK reads it from a Box source's version or from an S3 or R2 object's full-object SHA-1 checksum. Without one, BeamCore's source digest pass reads the source and supplies it. The pass reads only a directly readable HTTPS source: S3 and R2 presigned URLs qualify; Hugging Face and Box downloads, which redirect, and plain-HTTP URLs do not. A session source without a trusted SHA-1 that BeamCore cannot digest is refused before anything is written (box_trusted_digest_required). The pass never reads with the prepared source URL: BeamCore asks the client to sign one GET of the whole source for it (sign_source), valid for expiresIn and pinned to the ETag and version recorded at prepare, when the pass starts and again when that URL is about to expire.
• Alone. A Box destination is its transfer's only destination, and another client cannot resume it.
• Tokens. worker_credentials (1 to 16, default 1) tokens per session go to the workers, one control token to BeamCore. Each is revoked when BeamCore finalizes or abandons the file, and any still live is revoked when the transfer is terminal or the client closes. Closing the client while a Box transfer runs abandons it.
• Limitation. A Box token cannot be restricted to a byte range: any worker token can write any range of the session. Box keeps the first write of a range and lists session parts with sha1: null, so BeamCore verifies each listed part by the part ID Box returned to its accepted worker, pins those part IDs and the whole-file Digest at commit, and aborts the session right after the commit to seal it. After the commit the SDK checks, with the user's token, that the file's current version and every version added since the plan hold the committed content.

For low-level raw transfer configs, use createRawTransfer; lifecycle transport still goes through NATS.

Route Streaming And Payload Size

Provider transfers use transfer-client-control/v7. Every reply carries Runtime and transport epochs; prepare and every route-stream message carry a UUID route generation. S3, R2, and S3-compatible destinations retain direct multipart UploadPart/ListParts/HEAD handling. Hippius and Hugging Face destinations take a plain PUT per chunk instead, so they carry no multipart group manifest. A Box destination sends one box_session manifest group per source in its first route stream, and its routes name that group and carry no credential; a route replay sends the routes alone, and BeamCore recovers each group with describe_group.

The client signs up to 64 routes concurrently by default and emits 1,024-route logical batches. Encoded MessagePack requests target 4 MiB physical batches under maxPayloadBytes (8 MiB by default; explicit positive overrides remain supported). The splitter reserves control-envelope headroom for the live auth token and stable request identity so the final encoded request remains below the configured guard. Lifecycle control refreshes cached SDK auth before the final 30 seconds of token lifetime and retries auth_token_expired replies with the same request identity and payload but a fresh token, preserving Core request-conflict protection during large route streams. A single route above the 4 MiB target but within the configured guard publishes alone; a single route larger than the configured limit fails before publication with both sizes in the error. Multipart create and abort requests use a separate multipartControlConcurrency limit of 2 by default, with explicit positive overrides supported, and S3-compatible control requests make up to five attempts for transient provider failures. This keeps URL signing throughput independent from provider control-plane pressure.

Callers that need durable recovery accounting can provide onMultipartGroupReady. The SDK invokes it after creating a multipart upload and before streaming that group’s signed routes. Its payload contains transfer, source, destination, object-key, upload, size, part-count, and expiry identities only; it never includes credentials, request headers, or signed URLs. A rejected callback aborts the newly created upload and fails preparation closed.

resumeProviderTransfer requires the complete saved callback payloads in multipartGroups. It validates every transfer/source/destination/object/size/part coordinate, then uses the same route-stream implementation as creation with those upload IDs. Missing, duplicate, or changed identities fail before new provider uploads or route publication. If a process died before recording an upload identity, treat it as uncertain cleanup instead of inventing a replacement. The onPrepared callback also runs during resume, before routes are streamed.

Create and resume accept an ownership signal. Aborting it stops initial routes, multipart creation, and recovery signing/replay; it does not cancel the transfer or abort a replacement owner's uploads. Recovery leases, signers, and background resume/replay are keyed by owner identity: a fenced-off owner never replays into, releases, or stops the signers of a replacement owner registered for the same transfer, and a replacement always gets its own recovery. User cancellation still uses cancelTransfer and verified provider cleanup. prepareProviderSource accepts the same signal for metadata requests.

The stream ID derives from the transfer, selected flow, and immutable plan identity, and each ordered batch ID includes its route-coordinate checksum. Manual multi-destination attachment requires delivery_index on every route. Lifecycle mutations retry transient failures up to three times with the same request identity. A non-recoverable provider setup or route failure raises BeamProviderTransferError, whose errors, transferCancelled, and multipartCleanupComplete fields retain the original provider, cancellation, and cleanup outcomes. waitForTransfer subscribes to the API-key-owned, at-most-once terminal signal before its first status read and reconciles every signal through authoritative status; subscription failure degrades to the same jittered 15-to-30-second status fallback. Call close() when the client is no longer needed; it also closes outstanding terminal waiters.

During the pre-completion integrity_check phase, Runtime may include an integrity_audit_challenge in status. The TypeScript client signs exact read-only source and final-destination GET ranges and submits transfer.integrity_audit_grants. S3-compatible source and destination grants use the prepared source ETag and Runtime-verified final-object ETag as If-Match conditions. Submission failures retry on the next status poll and appear as a URL-safe integrity_audit_submission_error on that status response; the transfer completes with a warning if the check cannot finish by its deadline.

Hippius uses canonical non-multipart signed_url routes (initial and recovered routes both PUT to the planned <final>/<plan_nonce>/chunk-NNNNNN key), so its manifest is empty and group-level final HEAD verification is not available from that provider flow.

NATS requires this guard because the broker rejects messages above its configured max_payload. This limit applies only to lifecycle/control metadata; transfer file bytes do not flow through NATS.

Restart Recovery

The client keeps one in-memory recovery lease per active transfer and one runtime.hello monitor per active shard. The lease is installed before route-stream begin, and a per-transfer lock coalesces initial streaming with replay. Multipart recovery retains the existing upload IDs and compact group state, then re-signs only the expiring route and commit controls. Each resumeProviderTransfer invocation re-prepares an existing transfer by explicit transfer id with a fresh request and route generation, while transport retries within that invocation reuse the same request. It then reattaches route-recovery signing from the current provider configs without creating duplicate multipart uploads. A Runtime epoch change invalidates cached auth, coalesces one transfer.resume, and regenerates routes under a fresh generation; a transport-only epoch change replays only when Runtime reports routes missing or expired. Provider credentials and signing inputs are retained only in memory and released on terminal status or close(). Low-level attachSignedUrls requires routeGenerationId, the plan fingerprint/checksum, and a recoveryFactory; signed URLs are never journaled to disk. Foreground cancellation, deadline expiry, and Runtime state-loss responses keep the background lease and retained multipart upload alive.

Hybrid endpoint signing

Credential adapters can use signDestinationUrl independently of signDestinationRoute, using the same provider configuration and multipart signer. Destination-only signing requires no synthetic source descriptor and reads no source data. Optional contentMd5 binds the worker-computed checksum to the provider upload without a separate source checksum scan.

signSourceReadRange and signSourceObject (one GET of the whole source, with no Range) accept ifMatch and versionId for frozen S3-compatible sources. Replay the returned headers unchanged. The ETag condition is signed and the version is included in the signed request. Unsupported non-S3 credential modes reject these conditions instead of dropping them; a Box source takes only versionId, the file version recorded at prepare. These helpers support the existing S3-compatible profiles used for R2, Hippius S3 and Hugging Face Storage Buckets; Hub repository tokens are a different provider mode.

Only short-lived range/upload routes go to workers. Multipart creation, verification, completion and abort remain in the existing control path. A worker upload response alone is not final-object completion.

listMultipartParts, completeMultipartUpload, and inspectDestinationObject reuse the same provider client and return metadata only. ListParts walks every page; completion submits the selected provider-verified parts. Adapters must durably retain the upload and verified manifest before completion so restart recovery cannot infer delivery from size alone.

Manual releases from dev require a committed -dev.N version and publish only to npm tag dev; the stable tag is unchanged.

Multipart control helpers (createMultipartUpload, listMultipartParts, completeMultipartUpload, inspectDestinationObject, and abortMultipartUpload) accept an optional AbortSignal and cancel through the existing provider HTTP transport. Pass it as signal in object arguments or as the last positional argument. Cancellation stops waiting and network activity; it cannot undo a provider operation already accepted. After an interrupted completion, inspect the durable upload/object identity before retrying or reporting cleanup complete.

Preparation diagnostics

Route streaming flushes at 1,024 routes, with no timer, and sends the final partial batch at stream completion. Signing overlaps acknowledged publication through bounded buffering; existing encoded-payload limits still apply. A source grant is reused across destinations only within the same signing generation and actual expiry.

The optional onDiagnostics callback receives bounded preparation measurements and source-grant reuse counts. Callbacks are best effort, may be dropped under load, and do not affect transfer outcomes. Durations may overlap; do not sum them into elapsed transfer time. Transfer status also exposes a typed performance summary when supported by Core.

Optional storage_location (StorageLocation in Go) describes the physical storage location. It is separate from the region used to sign provider requests. Leave it unset when unknown; a signing region such as R2's auto is not a location. Without it, an R2 or Google Cloud Storage source or destination carries the location its bucket reports for itself as bucket_location in its plan metadata.

When supported by Core, diagnostics use sdk-performance/v2: bounded histograms, preparation milestones, concurrency high-water marks, and separate provider, callback, manifest, signing, and transport waits. Histogram bins are noncumulative, with upper bounds in milliseconds of 0.1, 0.5, 1, 2, 5, 10, 25, 50, 100, 250, 500, 1000, 2500, 5000, 10000, 30000, 120000, then overflow. unmeasured explicitly identifies unavailable measurements. Process CPU includes other concurrent work in the SDK process; it is not transfer-exclusive CPU. Detailed reporting can be disabled with BEAM_SDK_PERFORMANCE_DETAILS=false. Older peers retain v1 reports. No credentials or storage grants are included in these diagnostics.

sdk.producer_wait measures waiting for a signed route separately from publication backpressure. Configured-limit gauges preserve starting limits; effective-limit gauges report the highest limit reached. Optional source_renewals counts source grants recreated in later preparation generations. A bounded 16 KiB history tracks 131,072 chunk indices; the counter is omitted after takeover or beyond that bound. It does not include separate recovery-control signing operations.