@finos/legend-cube-builder

The UI and the Legend engine adapter of Legend Cube, a canvas-based visual query builder. The domain model lives in the host-free @finos/legend-cube.

It holds the page (the canvas, the palette, the side-panel editors, the source picker and the results grid), its MobX stores, the engine port with its v1/ adapter, and the bundled model catalog. Legend Analytics hosts the page at /analytics/, and Legend Studio opens it from an ingest definition's editor (see hosting).

Status: work in progress, version 0.0.x. This version has relational tables (from a model or a direct connection to H2 or DuckDB), the access points of deployed data products and the data sets of deployed ingest definitions (both beta), Join, Sort, Group, Pivot (on the databases that run it), Filter, Restrict, Rename, Distinct, Drop, Limit, Slice, Concat, Compare Column Values (Difference), window functions (Apply Window Functions), computed columns (Extend Columns, whose expressions the engine checks) and rows kept by a Pure condition (Filter by Expression, which the engine checks too), with one model, one runtime and one database, data product project or producer deployment per query. A cube is kept only through Export and Import of its spec, marked "(dev)", until the Cube store exists. A pasted model is kept in the cube. The results grid works server-side by default: a header's sort and each drill-down level run on the engine, from the run on screen; local mode groups, pivots and filters the run's rows in the browser. The grid exports the whole query as CSV through the engine and the rows shown as Excel, lists the latest runs with their SQL, and formats number columns (a number, an ISO currency or a percent). Nothing links to the page yet, there is no redo, and Cube sends no telemetry.

Layout

It depends on @finos/legend-cube, legend-graph, legend-application, legend-art, legend-lego, legend-shared, and the depot, lakehouse and storage clients, never on an application package (legend-application-*) or on legend-query-builder.

The V1 rule: V1_* protocol code lives under src/graph-manager/protocol/pure/v1/. The one product file outside it that names V1_* is the seam src/graph-manager/protocol/pure/CubeEngineBuilder.ts: buildCubeEngine is the only product code that makes a V1_LegendCubeEngine, so hosts never name a V1_* symbol. The repo's enforce-module-import-hierarchy lint rule forbids imports between stores/ or components/ and v1/, both ways. The engine-backed tests in src/__tests__/ are an exception: they need both the adapter and stores/.

Public API

src/index.ts exports:

• CubeEditor, the page, and CubeHost, what a host gives it; CubeEntrySource, a source a link asks the page to start with (initialSource), and formatCubeAccessPointEntryId with its CubeAccessPointEntry, which write a data product access point's link id (hosting.md);
• the engine port, everything in CubeEngine.ts (CubeEngine, CubeEngineError, CubeEngineErrorKind, CubeResult, CubeModelOutline, …), with buildCubeEngine and CubeEngineConfig;
• the connection explorer port, everything in CubeConnectionExplorer.ts, with buildCubeConnectionExplorer;
• data products: everything in CubeDataProduct.ts (the model kind, its runtime path, the environment types, the warehouse rule) and CubeDataProductCatalog.ts (the catalog port and its classes), with buildCubeDataProductCatalog, buildCubeLakehouseEnvironment, CubeLakehouseServices, CubeLakehouseEnvironment and getCubeRememberedWarehouse;
• ingest data sets: everything in CubeIngest.ts (the model kind, its runtime path, the saved settings) and CubeIngestCatalog.ts (the catalog port and its classes), with buildCubeIngestCatalog;
• published projects: everything in CubeProject.ts (the pointer model and its checks) and CubeProjectCatalog.ts (the catalog port), with buildCubeProjectCatalog;
• LocalModelCatalog and BUNDLED_MODELS;
• LEGEND_CUBE_COMMAND_CONFIG and LEGEND_CUBE_COMMAND_KEY, the page's shortcuts;
• LEGEND_CUBE_TEST_ID;
• helpers that 0.0.2 published and nothing outside the package uses yet: getRuntimesForDatabase, createTextModel, BundledModel, and the bundled model's CUBE_NORTHWIND_MODEL, CUBE_NORTHWIND_DATABASE and CUBE_NORTHWIND_RUNTIME.

The node-type registries (drafts, editors, help text and icons) are internal: there is no plugin API for node types yet.

Sources

A cube reads one kind of source: relational tables (from a model, or from a database through a direct connection), the access points of deployed data products, or the data sets of deployed ingest definitions (both beta; PLAN §6.7, §6.8). The "Add a source" dialog has a tab per way to find a source, each implementing CubeSourcePickerTab (src/stores/source-picker/): Model, Database connection, Data product and Ingest. A host offers the last three by giving the page a connectionExplorer, a dataProductCatalog and an ingestCatalog (see hosting).

A new kind of source needs, in the core, a SourceDefinition (node, codec, emitter and its query rules) and, in the builder, a tab, a Source panel editor, help text and an icon, and its branch in the engine's implementation, which builds the model each call runs on from the cube's saved model kind (src/graph-manager/CubeDirectConnection.ts, CubeDataProduct.ts and CubeIngest.ts are the examples).

Tables of a published project's Databases come through the Project Database tab (PLAN §6.3), which extends the Sample Data tab: the cube saves the engine's pointer to the project at a released version, and the host's projectCatalog lists projects, versions and outlines from its depot.

Samples and examples

The Sample Data tab bundles three datasets: Northwind (without Cube's test tables) and two made-up, seeded samples, Sports and Trades (src/stores/fixtures/), each an in-memory H2 database filled by its setup SQL. The Examples dialog (the header's Examples button) opens one of seven example cubes over them (src/stores/CubeExamples.ts) in place of the cube, then runs it, or starts a new cube on a dataset (PLAN §6.9). An example is built in code, so a new one needs no saved spec; add it to CUBE_EXAMPLES, and the engine test CubeExamples.engine-roundtrip-test.ts opens, types and runs it.

Documentation

• Hosting the page: what a host gives the page, the shortcuts, the stylesheet, and the dev loop.
• Testing: the test groups, the fake engine, and the tests against a real engine.
• Demo: a walkthrough on the sample data, its fast notes, and specs to import at each step.
• Adding an operation: the builder half of a new transform. The core half is in @finos/legend-cube.
• The plan and the known issues.