Harness Toolbox

Client, ClientProvider, DataSource, EnvironmentContext and ClientManager are owned by the sibling TypeScript package @compforge/harness-common. Toolbox provides protocol implementations and transports; its lifecycle entrypoints re-export the same common symbols.

Shared infrastructure for diagnostic tools, test runners and business adapters. Use the same clients to access HTTP(S), Kubernetes, MySQL, Redis, OpenSearch and S3-compatible object storage without depending on a particular command, plugin protocol, test model or verdict.

The package supports Node.js 22+ and Bun. Install with npm install @compforge/harness-toolbox. Use @compforge/harness-toolbox/duration to parse a positive duration such as 30m or 2d into milliseconds. searchAfterPages from @compforge/harness-toolbox/opensearch/search-after yields raw OpenSearch hit pages with a caller supplied query and stable sort. It streams pages without owning the client or claiming snapshot consistency; callers supply cancellation and decide how to persist or verify results. Protocol adapters have separate entry points; importing lifecycle or transport contracts does not load database or Kubernetes drivers.

HTTP requests

HttpClient borrows the same Transport as other protocol clients, keeping the logical Host and TLS identity even when the socket goes through a port-forward. Limits include the response body; statuses and redirects are returned without retries or redirect following.

import { ClientManager } from "@compforge/harness-common";
import { HttpClient } from "@compforge/harness-toolbox/http";
import { DirectTransport } from "@compforge/harness-toolbox/transport";

const clients = new ClientManager();
try {
  const route = await clients.get({ clientKey: "direct", createClient: () => new DirectTransport() });
  const http = await clients.create({ createClient: (_clients, signal) => new HttpClient(
    { endpoint: "https://service.example" }, route,
    { concurrency: 4, connectTimeoutMs: 5000, requestTimeoutMs: 10000, maxResponseBytes: 1024 * 1024 },
    { signal },
  ) });
  const response = await http.request("/health");
  console.log(response.status, new TextDecoder().decode(response.body));
  await http.dispose(); // The shared route remains available until root disposal.
} finally { await clients.dispose(); }

File sources

A FileDataSource is a caller-declared acquisition group. Use LocalFileDataSource for local paths or PodFileDataSource for paths in one explicit namespace/Pod/container. Both extend Common's FileDataSource; no additional id/kind wrapper is required.

import { ClientManager } from "@compforge/harness-common";
import { LocalFileDataSource, PodFileDataSource } from "@compforge/harness-toolbox/file";

const source = new LocalFileDataSource(["./config.yaml", "./secret/config.yaml"], { concurrency: 2 });
const clients = new ClientManager();
try {
  const file = await clients.get(source);
  const results = await file.read({ maxBytes: 1024 * 1024, maxTotalBytes: 2 * 1024 * 1024, timeoutMs: 15_000 });
  for (const result of results) {
    if (result.status === "error") console.error(result.path, result.error.message);
    else console.log(result.path, new TextDecoder("utf-8", { fatal: true }).decode(result.bytes));
  }
} finally { await clients.dispose(); }

// Resolve Workloads first. This entire group is fetched through one exec session.
const podSource = new PodFileDataSource(environment,
  { namespace: "app", pod: "worker-abc", container: "worker" },
  ["/etc/app/config.yaml", "/etc/app/secret/config.yaml"], { concurrency: 2 });

Every read reopens all declared paths and returns one ordered result per path. Empty files succeed; missing, denied, nonregular and oversized files return individual errors with no partial bytes. An error in one file does not discard sibling results. Per-file limits reset for each path; the total budget bounds retained decoded bytes, including incomplete files. Queueing and the whole group share one deadline. Cancellation, total-budget exhaustion, deadline and transport/protocol failure reject the operation. Sources share clients, never previous file contents.

Pod groups use one exec and require sh, head -c, base64 and tr. Paths remain positional arguments even when they contain shell syntax or newlines. Indexed base64 records carry raw binary bytes plus each file's exit status; truncated or malformed records reject without replay. No remote temporary files are written. Local reads allow symlinks to regular files and reject nonregular files. Keep the path and Pod target on evidence; grouping guarantees neither an atomic filesystem snapshot nor application configuration precedence. A Pod name is not a UID fence; callers needing incarnation guarantees must check discovery around collection. File clients borrow root Kubernetes access and never close it themselves.

Version 0.4.2 requires Common 0.5.4. Replace single path arguments with arrays, supply maxTotalBytes, and handle per-file results. Publish Common before Toolbox; this change has no single-file legacy adapter.

Redis sources

RedisClient owns resolved configuration and reusable node connections. Use clusterType to select single, cluster or sentinel; endpoints contains that mode's seed addresses. Sentinel configuration additionally provides sentinelMasterName and optional Sentinel credentials, separate from data-node credentials. Cluster connections require database zero.

Supply one Transport, with ordered TCP paths where needed. Only mapping or handshake network failures advance to the next transport; authentication and TLS verification failures stop immediately. connectTimeoutMs bounds each route (mapping and handshake); it defaults to the command timeoutMs. acquireTimeoutMs bounds all candidates together and defaults to the per-route budget times the number of routes. A spent total budget does not start another route. onRoute reports the selected route and the preceding failure code without credentials.

client.topology() refreshes topology using the same connections and route policy as client.access.connection(endpoint, database). A root signal cancels connection preparation and active commands. Commands and pipelines are sent once, with no offline queue or automatic replay; a subsequent acquisition replaces an unhealthy connection. The caller owns dispose() and transport cleanup, and diagnostics own sampling policy and collected observations.

Kubernetes resources

Borrow a root KubernetesClient through KubernetesEnvironment and ClientManager. Use get(ref) and list(query) for resource reads; inNamespace(namespace) binds a borrowed view to a namespace without creating a second client or pool:

const kube = await clients.get(environment);
const ns = kube.inNamespace("app");
const service = await ns.get({ apiVersion: "v1", resource: "services", name: "api" });
const pods = await ns.list({ apiVersion: "v1", resource: "pods", labelSelector: "app=api" },
  { timeoutMs: 15_000, signal });

resource is the plural REST resource name, not a Kind or kubectl alias. Explicit API versions support built-in resources and custom resources without a discovery request or local allowlist. The root uses its configured namespace unless the query overrides it; namespace views enforce their bound namespace. Resource objects and list metadata are returned intact. List requests accept label/field selectors and explicit limit/continue pages; they do not silently read all pages. Errors are KubernetesError objects carrying a stable kind and the HTTP status when available. Callers retain the query with their evidence; resource operations do not produce shell command results.

The root owns one HTTP transport with keep-alive connections shared across namespaces and resource kinds. Authentication refreshes on each request. TLS/proxy identity changes select a new pool and retire the previous one after its active reads finish. Requests have bounded bytes and a deadline covering queueing, authentication and body consumption. dispose() cancels work, waits for tracked operations, and closes connections. Views and injected borrowed resource readers have no ownership of that lifecycle. No query results are cached and failures are never replayed through another transport.

Native resource access uses kubeconfig authentication, TLS and explicit proxy-url. It does not select transports based on shell proxy environment variables. exec and forward are separate streaming operations; their implementation does not intercept resource reads. KubernetesClient has no run(argv) method. Concurrency remains caller-configured rather than inferred from pool size.

Workload discovery

Borrow a KubernetesClient through KubernetesEnvironment and the shared ClientManager lifecycle, then call client.resolveWorkload(workload, environmentId). The ID is a stable target identity from your environment registry, not a display name or kubeconfig path. The Workload's namespace overrides the client's default. Resource, Service selector and labels locations return all matching Pod incarnations without readiness filtering; successful empty discovery returns an empty array. Missing resources, selectorless Services and failed access raise KubernetesError.

The resource path uses client-node kubeconfig authentication/TLS and bounded native JSON reads, not kubectl error text. Configure request timeout, concurrency and response bytes with the client's ResourceLimits constructor argument. Exec uses the same root capacity/byte policy with a separate slot pool; its per-call timeout defaults to that policy and covers queueing, authentication, upgrade and execution. Explicit per-call timeouts support longer commands.

Output-only exec connects directly to the Pod exec WebSocket endpoint without a preliminary Pod GET and requires get pods/exec. Commands with explicit stdin (including empty input) use kubectl exec -i, which handles input EOF through SPDY/v5 on older and newer clusters. This path requires a compatible kubectl on the Host, Pod read permission and create pods/exec for SPDY (get pods/exec for WebSocket). The channel is selected before execution; failures are never retried or replayed by Toolbox. The result's command records the actual channel. Kubeconfig controls authentication, TLS and proxy access. Both paths share the root exec slot pool, deadline, retained-output byte limit and cancellation. stdout/stderr remain separate; native exec uses remote status and kubectl uses process exit status, retaining raw CLI stderr without guessing structured HTTP error kinds. Streaming callers may set collectStdout: false and enforce their own total byte budget; individual output chunks remain bounded. Root disposal cancels active and queued sessions and waits for the local kubectl process to exit. Output-only sessions accept v4/v5. Bun does not expose HTTP upgrade rejection status, so those failures retain an unknown connection error. Native exec with a kubeconfig proxy requires Node: Bun cannot reliably release proxy sockets or preserve TLS name overrides. This restriction does not apply to the kubectl stdin path. Neither path bypasses proxy or TLS checks. Port-forward also uses kubectl. See the shared Workload model.

import { ClientManager, EnvironmentContext } from "@compforge/harness-toolbox";
import { KubernetesEnvironment } from "@compforge/harness-toolbox/kubernetes/environment";

const clients = new ClientManager();
try {
  const ctx = new EnvironmentContext(
    new KubernetesEnvironment("dev",
      { kubeconfig: "/config/cluster", context: "dev", namespace: "app" },
      { timeoutMs: 10_000, concurrency: 4, maxBytes: 8 * 1024 * 1024 }),
    clients, performance.now() + 30_000,
  );
  const kube = await ctx.clients.get(ctx.environment);
  const pods = await kube.resolveWorkload({
    name: "api", platform: "kubernetes",
    location: { kind: "resource", resource_kind: "Deployment", name: "api" },
  }, "cluster-a/runtime");
} finally {
  await clients.dispose();
}

The context does not cancel work automatically. Its budget is available as remainingMs; the root owns cancellation and must join child work before disposing clients.

Catch ToolboxError from the root entry or @compforge/harness-toolbox/errors; use kind and code for decisions. Native cause may contain sensitive details and is debug-only. Connection acquisition deadlines and Redis handshake/command deadlines raise TimeoutError from @compforge/harness-toolbox/errors. It extends ToolboxError, retaining kind: "timeout" and code: "ETIMEDOUT"; OpenSearch preserves it unchanged.

For coordinated common/toolbox development, install and build common before installing toolbox:

# From sdks/typescript/toolbox
(cd ../common && bun install --frozen-lockfile && make build)
bun install --frozen-lockfile
make lint
make test

The toolbox root override resolves common from ../common only for local development; the published dependency remains a registry version. Publish common before toolbox. The Makefile builds common before typechecking or building toolbox.

A ClientProvider identifies a reusable client and constructs it; DataSource adds data-access semantics. Accessible environments implement ClientProvider directly without becoming data sources. The Client owns initialization and idempotent disposal; a ClientManager joins concurrent initialization for the same key and closes owned clients when the caller finishes. Protocol clients take configuration (or an async resolver) and one Transport; the transport selects and prepares access paths. clients.create(factory) creates a fresh client that its caller may close early; transports acquired with get stay alive until root disposal.

For shared port-forwarding, register a ServicePortForwardTransport once per cluster/namespace, then pass the borrowed transport to each protocol client. Closing an OpenSearch client leaves the forward available to the next OpenSearch, MySQL, Redis or S3 client; root disposal closes the clients before stopping the forward processes. Borrower cancellation only stops that borrower's wait. Exited forwards are recreated on the next acquisition. Transport mapping changes the dial address, while protocol clients retain the original Host, TLS and signing identity. In 0.5, replace new Client({ resolve, transports }, options) with new Client(resolve, transport, options) (S3 retains its explicit limits argument).

import { ClientManager, clientKey, type DataSource } from "@compforge/harness-toolbox";
import { MysqlClient } from "@compforge/harness-toolbox/mysql";
import { DirectTransport } from "@compforge/harness-toolbox/transport";

const target = { host: "localhost", port: 3306, database: "app", user: "reader", password: "..." };
const source: DataSource<MysqlClient> = {
  clientKey: clientKey("mysql", target),
  createClient: (_clients, signal) => new MysqlClient(target, new DirectTransport(),
    { signal, connectTimeoutMs: 5_000, queryTimeoutMs: 10_000 }),
};
const clients = new ClientManager();
try {
  const db = await clients.get(source);
  const rows = await db.database.query(db.target, "SELECT 1 AS value", []);
} finally {
  await clients.dispose();
}

Callers own credentials, authorization, query semantics, capacity and timeouts. MySQL can use TCP or a Python-capable Pod; the latter needs PyMySQL. Transport fallback occurs only when establishing a connection, never by replaying a failed SQL operation. Pod inputs use stdin.

Pod log capture shares an identical Pod/container instance and absolute time window once per client, while consumers filter independently and retain their own raw copies. Supply an explicit capture policy, shared concurrency pool and byte budget; the package does not choose product limits. Temporary capture files live until client disposal.

See toolbox contracts for ownership and cross-language semantics.

Read S3 objects

S3 uses the official AWS SDK behind the same managed lifecycle. Supply resolved connection settings and explicit limits; the package does not discover business configuration or credentials.

import { ClientManager } from "@compforge/harness-toolbox";
import { S3DataSource } from "@compforge/harness-toolbox/s3";

const clients = new ClientManager();
try {
  const s3 = await clients.get(new S3DataSource({
    endpoint: "https://s3.example.com", region: "us-east-1",
    credentials: { accessKeyId: "...", secretAccessKey: "..." },
  }, { concurrency: 4, connectTimeoutMs: 5_000, requestTimeoutMs: 15_000 }));
  const object = await s3.headObject("artifacts", "runs/example.json");
  const prefix = await s3.readObject("artifacts", "runs/example.json", { maxBytes: 65_536 });
  // prefix.truncated distinguishes a bounded prefix from the complete object.
} finally {
  await clients.dispose();
}

listObjects and listBuckets return one explicitly bounded page with continuation tokens; callers own traversal and total scan budgets. Bucket HEAD and versioning are also available. Errors preserve SDK status/code: a failed HEAD is not converted to exists=false. Reads stop at the byte cap and close the body; request deadlines include queueing and body consumption. No write operations, automatic retries or implicit region redirects are exposed.

For a mapped TCP transport, pass { key: "cluster/context identity", transport } as the third DataSource argument. The caller owns that transport's lifetime. Path-style addressing is required for mapped routes; direct endpoints also support virtual-hosted buckets. Signed Host and TLS identity remain the logical endpoint, not the forwarded address. Private CAs may be supplied explicitly with ca; certificate verification stays enabled. Initialization prepares the client and route without listing buckets or claiming that credentials have been validated.

Bounded MySQL reads

MysqlClient.database.queryReadonly(target, sql, values, { timeoutMs, maxRows, maxBytes }) uses the shared query slot and a disposable native session with NO_BACKSLASH_ESCAPES, a SELECT deadline, and a READ ONLY transaction. Prepared execution binds values and retains rows incrementally, returning rows, columns, bytes, truncated, and truncation. An extra row proves row truncation; a row exceeding the remaining byte budget is not retained.

Callers own SQL classification, function policy, and least-privilege identities: READ ONLY alone is not an arbitrary-SQL sandbox. This API requires MySQL 5.7.8+ max_execution_time and does not fall back to Pod Python. Completion, truncation, timeout, and cancellation destroy the disposable session without replay or changes to ordinary borrowed sessions. Disconnect does not prove immediate server cancellation. The byte budget limits retained JSON rows, not individual wire packets or fields.

Reusable bounded queries and read caches

For trusted, caller-owned SELECT statements, queryLimited(database, target, statement, maxRows) appends an integer LIMIT maxRows + 1 and returns { rows, truncated }. SQL must not already have LIMIT or a trailing terminator; the helper does not parse or rewrite arbitrary SQL. Business values stay bound parameters. limitedStatement and limitedRows expose the same preparation/projection for Database.queryBatch. maxRows must be a positive integer below 2147483647. These helpers bound row count only: use queryReadonly when a disposable readonly session and byte/time budgets are required.

DataLoader shares results, failures and in-flight work within one explicit read scope, similar to Guava's LoadingCache but scoped to a single collection round. Consumers must not mutate shared results. Use withReadScope to bind caller-approved database reads; do not use it for writes or polling.

import { DataLoader } from '@compforge/harness-toolbox/data-loader';
import { withReadScope, queryLimited } from '@compforge/harness-toolbox/mysql';

const scope = new DataLoader({
  maxEntries: 256, maxBytes: 4 * 1024 * 1024, signal: client.signal,
});
const reads = withReadScope(client.database, scope);
try {
  const page = await queryLimited(reads, client.target, {
    sql: 'SELECT id FROM items WHERE owner = ? ORDER BY id', values: [owner],
  }, 100);
} finally {
  await scope.close(); // Root still owns client.dispose().
}

Keys include full database identity, SQL and typed parameters as an opaque digest. Nonserializable parameters bypass caching. Batch reads reuse cached entries and send misses as one ordered native batch, or sequential queries when batching is unavailable. Failures are shared without replay; use a new scope for fresh observations. This is not a consistent database snapshot.

Entry capacity includes failures and pending reads; new keys bypass caching when full. Serialized result bytes bound retained results, excluding keys/errors and active operation memory. Oversized or nonserializable results are returned but not retained. Scope close signals cancellation and joins operations, including uncached reads. Operations must honor the signal for prompt cancellation; MySQL reads retain their underlying client's timeout/cancellation policy. Adapter close owns neither the scope nor the database. A waiter's optional signal to scope.read cancels only that waiter.

Shared Pod relay transport

PodRelayTransport opens TCP routes through an existing Running/Ready Pod. It discovers an eligible container in the selected namespace (optionally restricted by a label selector); the current implementation uses Python 3.8+ standard-library asyncio. Callers supply endpoints, not a Pod name or interpreter. It creates no Pods and installs no packages. The selected Pod must be able to resolve and reach each destination.

Create one transport in the root execution's ClientManager and lend it to all Service/protocol clients. Its key describes cluster access and relay policy, never a Service or database identity. Concurrent callers share Pod discovery, one relay process and one forward per host/port. Database credentials and connections remain owned by their protocol clients.

import { ClientManager } from '@compforge/harness-toolbox';
import { PodRelayTransport } from '@compforge/harness-toolbox/transport';

const clients = new ClientManager();
try {
  const relay = await clients.get({
    clientKey: 'transport:customer-cluster:app',
    createClient: (_clients, signal) => new PodRelayTransport({
      kubeconfig: '/config/cluster', context: 'customer', namespace: 'app',
      selector: 'diagnostic-relay=eligible',
      startupTimeoutMs: 5000, connectTimeoutMs: 5000,
      maxConnections: 16, maxTargets: 8, maxCandidatePods: 8, signal,
    }),
  });
  // Pass this same transport instance to each protocol client; clients never dispose it.
  const address = await relay.connect({host: 'database.internal', port: 3306});
  // Connect the native protocol client to address; do not dispose relay in a Service.
} finally {
  await clients.dispose();
}

The host must authorize list pods, create pods/exec and create pods/portforward, including running a temporary process in the selected container. List/discovery and each readiness operation use startupTimeoutMs; candidate Pod count, target count and aggregate live relay connections are bounded separately. List pods is namespace scoped. Relay listeners bind only to Pod loopback and local forwards only to host loopback. Control messages contain destinations, never DB credentials.

Disposal closes forwards and the exec stream; EOF closes remote listeners/connections. If an exec connection is lost without EOF, a 30-second heartbeat lease bounds remote process lifetime. A dead relay invalidates its routes and fails callers; no SQL or other protocol operation is replayed. The root should finish or cancel protocol work before disposing the transport.

OpenSearch connection preparation

openOpenSearch probes the resolved endpoint before returning. It tries the supplied TCP transports in order only for connection failures; HTTP authorization/server errors stop preparation, and failed reads are never replayed on another route. onRoute reports the selected transport and safe fallback reason. The caller owns shared transports such as PodRelayTransport and their cleanup.

Routed HTTPS preserves the original TLS server name. Bun disables HTTP keep-alive because its reused socket recovery can replay a failed request below the driver; Node retains connection pooling.

Kubernetes compute evidence

kubernetes/compute exports collectClusterCompute and collectWorkloadCompute. They borrow ResourceAccess, use explicit deadlines/item/page budgets, and retain read coverage independently. Use client.clusterResources() for Nodes and all-namespace Pods; normal client methods retain their configured namespace. A forbidden cluster read records permission_denied; it does not block a separate namespace-scoped Workload read. See compute collection.

Connection acquisition deadlines

MySQL, OpenSearch and Redis use the same acquisition lifecycle. connectTimeoutMs includes route mapping and handshake/probe; optional acquireTimeoutMs bounds all routes together (default: per-route budget × route count). Cancellation preserves the caller reason and prevents late mappings from starting a handshake. Query and command timeouts remain separate, and failed operations are never replayed on another transport.

DataLoader accepts an optional logger.debug(message) and a non-sensitive label. Diagnostics distinguish miss, wait, hit, bypass, not-retained and close using scope/read identifiers, never keys, values or errors. Logging does not change retention or retry policy.

Network connectivity

@compforge/harness-toolbox/network exports probeConnectivity(endpoint, execute, {timeoutMs, signal?}). It consumes Endpoint from @compforge/harness-common; ConnectivityTarget is removed.

import type { Endpoint } from "@compforge/harness-common";
import { endpointFromUrl, validateEndpoint } from "@compforge/harness-toolbox/endpoint";

const endpoint: Endpoint = endpointFromUrl("https://storage.example:9443/object?signature=private");
// { scheme: "https", host: "storage.example", port: 9443, secure: true }
validateEndpoint(endpoint); // Pure validation; no DNS or I/O.

The URL projection drops credentials, path, query and fragment, and unbrackets IPv6 hosts. HTTP(S), WS(S) and FTP have default ports. The tls scheme requires a port and sets secure: true. Other schemes require an explicit port and leave secure unspecified; callers supply any encryption requirement from their protocol configuration. Errors do not expose the original URL. Keep the full URL separately for resource access.

secure replaces tls in probe inputs and result targets. HTTPS/WSS/TLS require secure: true; HTTP/WS reject it. With secure: true, this probe supports direct TLS for https, wss, and explicit tls schemes. Other schemes, or a missing scheme, throw ProtocolError (a ToolboxError with kind: "protocol", code: "Unsupported") before execution: protocol-negotiated encryption such as MySQL or STARTTLS is not implemented. Without an encryption requirement, a successful TCP check does not prove application-protocol availability or encryption. TLS remains the name of the observed handshake stage.

The caller supplies an executor in the consumer environment (host, Pod or sandbox), which runs Python 3 with the supplied argv and stdin and enforces timeout/cancellation. DNS, TCP and optional verified TLS have separate results and timings. Domain names, IP literals and Kubernetes service.namespace names use the same API. There is no implicit execution on the caller host or ICMP requirement. Network failure returns evidence; missing Python, execution failure and incomplete output throw. TLS success does not establish HTTP authorization, object access or application download policy.