@adobe/data

Adobe Data Oriented Programming Library

Documentation

API Reference

ECS Performance Test

Breaking API Changes

Until we reach 1.0.0, minor version changes may be API breaking.

Data Oriented Programming

This library is built using a data oriented and functional programming paradigm. We prefer composition over inheritance, avoid classes when possible and emphasize separation of concerns.

Data

This library uses data oriented design paradigm and prefers pure functional interfaces whenever practical.

For our purposes, Data is immutable, (de)serializable objects and primitives — JSON extended with ReadonlySet, ReadonlyMap, and Blob. A ReadonlyArray is ordered (position is meaningful); a ReadonlySet / ReadonlyMap is unordered. Serialize Set/Map-bearing Data with Data.stringify / Data.parse (plain JSON.stringify renders a Set/Map as {}); equals compares them faithfully.

Why immutable Data?

We prefer Data because it is easy to:

• serialize
• deserialize
• inspect
• compare
• hash
• validate

We prefer immutable Data because it is easy to:

• reason about
• avoid side effects
• avoid defensive copying
• use for pure function arguments and return values
• use with concurrency
• memoize results
  • use as cache key (stringified)
  • use as cache value

What is Data Oriented Design?

Sanders Mertens covers this thoroughly in his ECS FAQ:

https://github.com/SanderMertens/ecs-faq?tab=readme-ov-file#what-is-data-oriented-design

Data Schemas

When runtime type information is required we use JSON Schema. In order to avoid redundancy, we can use FromSchema to derive a typescript type at compile time.

Example:

const Float32Schema = { type: "number", precision: 1 } as const satisfies Schema;
const Float32 = FromSchema<typeof Float32Schema>; // number

It is important that we use as const for the schema definition. Without that, the typeof Float32Schema would be { type: string, precision: number } instead of { type: "number", precision: 1 } and we would not be able to derive a correct type using FromSchema.

Normalized Data

Data is considered "Normalized" when all object keys contained anywhere within it are lexigraphically sorted. We include a normalize function which performs this conversion. This is particularly useful when you want to use data as a cache key.

const notNormalized = { b: 2, a: 1 };
const normalized = normalize(notNormalized); // { a: 1, b: 2 }

normalize operates on plain JSON structure only — it does not canonicalize Set or Map values (which have no JSON form). Don't feed Set/Map-bearing Data through normalize for cache-keying; key on a Data.stringify of the value, or model the collection with plain arrays/objects where a canonical cache key is required.

Observables

An Observable<T> is a subscription function that you can pass a callback function to. Your callback function can accept a single argument of type T. The subscription function returns a dispose function that accepts no parameters and can be called at any point in the future to cancel your subscription.

Your callback function may be called back synchronously (before the initial call returns) zero or one times and asynchronously later any number of times.

For more information see the Observable API documentation

Observable Types

/**
 * Called to add a new observer to an Observable.
 * Returns an Unobserve function that can be called to stop observing the Observable.
 */
export type Observable<T> = (observer: Callback<T>) => Unobserve;
/**
 * Callback function called zero or more times with a sequence of values.
 * *may* be called back synchronously, immediately when selector function is called.
 * *may* also be called back asynchronously later any number of times.
 * *may* be called multiple sequential times with the same value.
 */
export type Callback<T> = (value: T) => void;
/**
 * Function called to stop observing an Observable.
 */
export type Unobserve = () => void;

An Observable can be thought of sort of like a Promise but with a few important differences.

• A Promise only yields a single value, an Observable yields a sequence of values.
• A Promise can reject with an error, an Observable can not. (It could yield type MyResult | MyError though.)
• A Promise can only resolve asynchronously, an Observable may yield an initial result synchronously.
  • An Observable is allowed to call the Callback callback function immediately upon observation if it has a value.
• A Promise begins executing immediately, an Observable may lazily wait for a first observer before taking any action.
• An Observable can be unobserved.

Observable Creation

This simple definition of an Observable makes it very easy to create them from various sources.

• fromConstant<T>(value: T) => Observable<T>
• fromPromise<T>(promise: Promise<T>): Observable<T>
• fromPromiseWithError<T, E extends Error>(promise: Promise<T>): Observable<T | E>
• fromObservableProperties converts an object with named Observable values into a single Observable of an object with corresponding types.

For instance:

fromObservables({ a: fromConstant(1), b: fromConstant("s")})
// yields type Observable<{ a: number, b: string }>

fromObservableProperties will only yield a result once every contained observable has provided a result. If some properties are optional then use the withOptional higher order function described below.

Observable Modification

They are also very easy to modify in order to build higher order Observables.

For the purposes of discussion let us use an array to indicate the sequential values yielded by an Observable with the first element being any synchronous callback and void meaning no synchronous callback.

So our fromConstant(12) would yield [12] and fromPromise(Promise.resolve(12)) would yield [void, 12]

These examples will use the form of: Sequence Before => Higher Order Function => Sequence After.

withDeduplicate removes any sequentially equivalent values:

[1, 1, 2, 2, 2, 3] => withDeduplicate => [1, 2, 3]

withDefault replaces any undefined values, including initial synchronous value with a default value:

[void, 1, 2, undefined, 4] => withDefault(-1) => [-1, 1, 2, -1, 4]

withMap applies a mapping function to each value at notification time:

[void, 1, 2, 3] => withMap(x => x * 2) => [void, 2, 4, 6]

withOptional converts an Observable<T> to Observable<T | undefined> and will always respond synchronously.

[void, 1, 2, 3, 4] => withOptional => [undefined, 1, 2, 3, 4]

Cache

The cache module contains functions and interfaces related to caching, hashing, memoization and storage and retrieval of Blobs.

The memoize function and BlobStore are the most important public interfaces in the cache submodule. They are described below. See the API reference for hashing and other lower level functions.

The memoize function can be used to cache expensive, deterministic, asynchronous functions that take data arguments and return data results. In addition to caching the results, it also blocks multiple simultaneous calls with the same arguments and makes them wait for the first result.

BlobStore

It is important to handle Blobs correctly within an application.

The BlobStore interface and corresponding blobStore exported instance provide a convenient way to convert Blobs into small JSON handles to them called BlobRefs. Those BlobRefs can later be converted back into a Blob.

Blobs within the blobStore are generally persisted across sessions using the browsers buid in Cache storage.

BlobRefs have a number of advantages over directly using blobs. BlobRefs are:

• small json objects
• deterministic for each Blob based on mime type and content.
• suitable for persistence to locations with limited size.
• suitable for use as a cache key
• suitable for use as a cache value

Schemas

Contains some standard data type schemas in JSON Schema format for convenience. These types are most frequently used when describing components within the Entity Component System. Knowing the precise size and signedness of primitive numeric types allows the ECS to store these values within tightly packed typed arrays internally. This makes memory usage and processing time more efficient.

Entity Component System (ECS)

A high-performance, strongly typed ECS database for TypeScript, inspired by Flecs. All application state is modeled as composable plugins, and all mutations flow through observable, undoable transactions.

For a complete guide covering plugins, transactions, observability, composition, and transient/ephemeral semantics, see the ECS README.

import { Database } from "@adobe/data/ecs";

const myPlugin = Database.Plugin.create({
  resources: {
    score: { default: 0 as number },
  },
  transactions: {
    addPoints: (t, points: number) => {
      t.resources.score += points;
    },
  },
});

const db = Database.create(myPlugin);
db.observe.resources.score((score) => console.log("Score:", score));
db.transactions.addPoints(10);

What is an ECS?

Sanders Mertens covers this thoroughly in his ECS FAQ:

https://github.com/SanderMertens/ecs-faq?tab=readme-ov-file#what-is-ecs

In addition to the standard Entity, Component, and System definitions, we also use the term Resource — a global singleton value defined on the ECS itself, not attached to any specific entity.