@harperfast/integration-testing

中文文档
1.1.0发布于 2026/11/17Apache-2.0TypeScriptES Module构建来源已验证近 30 天下载 11.8万

Integration testing utilities for Harper-based projects. Provides Harper instance lifecycle management, loopback address pooling, and a test runner script.

原始使用文档
文档目录28 节
  1. Testing Ethos
  2. Installation
  3. Framework Agnosticism
  4. Setup: Loopback Addresses
  5. Persisting across reboots (macOS)
  6. Other services listening on Harper's ports
  7. API
  8. startHarper(ctx, options?)
  9. setupHarperWithFixture(ctx, fixturePath, options?)
  10. killHarper(ctx, options?)
  11. teardownHarper(ctx)
  12. sendOperation(context, operation)
  13. createHarperContext(name?)
  14. Types
  15. Orphaned Instance Monitor (POSIX)
  16. Server Log Capture
  17. HarperStartupError
  18. ForeignListenerError
  19. LoopbackPoolExhaustedError
  20. targz(dirPath)
  21. Loopback Pool Utilities
  22. Node.js Test Runner
  23. Running Tests
  24. Runner Options
  25. Writing Tests
  26. Using Other Frameworks
  27. GitHub Actions Workflow Parallelization
  28. License

@harperfast/integration-testing

Integration testing utilities for Harper-based projects. Provides framework-agnostic Harper instance lifecycle management, a concurrent loopback address pool, directory compression utilities, and an optional preconfigured Node.js test runner script.

Testing Ethos

There are three distinct levels of testing in the Harper ecosystem:

  • Unit tests validate isolated logic against source or built code directly, relying on mocks or stubs in place of real dependencies.
  • Integration tests run against the actual Harper software, but locally and ephemerally. Harper is started as a child process for each test suite, given a temporary install directory, and torn down when the suite finishes. There is no remote infrastructure to manage.
  • End-to-end tests go further: they target a deployment that mirrors production as closely as possible — think a staging environment with real infrastructure, remote systems, and operational configuration.

This project is strictly for integration testing.

The distinction matters because integration tests give you confidence in Harper's actual behavior without the cost and complexity of managing a deployed environment. You get the real software, not mocks, but with fast local iteration.

Regardless of the framework or runner used, integration tests should strive to be:

  • Independent: No dependency on execution order or shared state with other tests
  • Hermetic: Self-contained with no external side-effects
  • Deterministic: The same input always produces the same output

These properties are what make safe concurrent execution possible — without them, parallelization produces flaky, order-dependent failures.

Installation

npm install --save-dev @harperfast/integration-testing

Framework Agnosticism

The core lifecycle and loopback pool APIs in this package — startHarper, teardownHarper, killHarper, and the loopback utilities — are not tied to any specific test runner or framework. They spawn and manage Harper child processes and allocate loopback addresses safely across concurrent processes, whatever their PID namespace or TMPDIR. This means they can be used with any testing framework that supports async setup and teardown: Node.js node:test, Vitest, Playwright, or anything else.

The key responsibility these APIs place on you is thinking critically about Harper instance scope. Starting a Harper instance has real cost — it spawns a process, allocates a loopback address from the pool, and creates a temporary install directory. How you scope instances depends on your framework and what you're testing. A single instance shared across many assertions in one logical test block may be perfectly appropriate; spinning up a fresh instance for every individual assertion would be wasteful. What matters is that concurrently executing scopes each have their own instance and loopback address — the pool handles cross-process allocation safely, but your setup and teardown hooks determine the boundaries.

This package also includes a preconfigured Node.js test runner script (harper-integration-test-run) as a recommended, batteries-included way to run tests written with the node:test API. It is documented below, and its source serves as a reference implementation for anyone integrating these utilities with a different runner.

Setup: Loopback Addresses

Running Harper integration tests concurrently requires multiple loopback addresses so that each Harper instance can bind to a distinct address without port conflicts.

Linux and Windows route the whole 127.0.0.0/8 block (127.0.0.1–127.255.255.255) to loopback by default, so they need no setup. macOS enables only 127.0.0.1.

On macOS, use the included script to configure the rest:

npx harper-integration-test-setup-loopback

This script requires sudo and respects the HARPER_INTEGRATION_TEST_LOOPBACK_POOL_COUNT environment variable (defaults to 32).

On macOS the aliases are configured with a host (/32) netmask. This matters: with the default class-A (/8) netmask every 127.0.0.x alias claims the whole 127.0.0.0/8 network, and at large pool counts the resulting set of overlapping-subnet interface addresses drives mDNSResponder's address-conflict defense into a CPU-pinning storm of mDNS announcements on port 5353. If you previously ran an older version of this script, re-run it to convert the existing aliases to /32.

Persisting across reboots (macOS)

ifconfig aliases live only in the running kernel — macOS does not persist manually added lo0 aliases, so they vanish on every reboot and the pool has to be reconfigured. For a dev machine that reboots regularly, install a launchd daemon that runs the setup script at boot. scripts/io.harperdb.loopback-setup.plist is provided for this; the script is root-aware, so it needs no sudo when launchd runs it as root.

# 1. Copy the setup script to a stable, root-owned path (the plist points here)
sudo mkdir -p /usr/local/sbin
sudo install -m 755 -o root -g wheel \
  "$(npm root)/@harperfast/integration-testing/scripts/setup-loopback.sh" \
  /usr/local/sbin/harper-loopback-setup.sh

# 2. Install the daemon (must be root:wheel and mode 644 or launchd rejects it)
sudo install -m 644 -o root -g wheel \
  "$(npm root)/@harperfast/integration-testing/scripts/io.harperdb.loopback-setup.plist" \
  /Library/LaunchDaemons/io.harperdb.loopback-setup.plist

# 3. Load and run it now (also runs at every boot via RunAtLoad)
sudo launchctl bootstrap system /Library/LaunchDaemons/io.harperdb.loopback-setup.plist

# 4. Verify
cat /var/log/harper-loopback-setup.log        # ✓ Configured ... line
ifconfig lo0 | grep '127.0.0' | tail -3       # aliases present

Edit the HARPER_INTEGRATION_TEST_LOOPBACK_POOL_COUNT value in the installed plist to change the pool size (it defaults to 32, matching the script). To remove the daemon: sudo launchctl bootout system/io.harperdb.loopback-setup and delete the plist. If you edit the installed plist, bootout then bootstrap again to reload it.

Other services listening on Harper's ports

Each node binds its fixed ports on its own loopback address, so nodes on different addresses coexist. A process listening on the same port on all interfaces is different — a local Harper instance running with its default configuration is the usual one. On macOS and Windows, that process receives a node's connections whenever the node has no listener of its own on its address: while the node starts, while its HTTP workers restart, and after teardown. Keep-alive connections then stay with it, so a suite fails, or passes, against the wrong server.

getNextAvailableLoopbackAddress, and therefore startHarper, refuses an address when another process already accepts connections on its operations or HTTP port, throwing a ForeignListenerError that names the port. Stop the other process or bind it to a specific address (lsof -nP -iTCP:9926 -sTCP:LISTEN names it). To run anyway, set HARPER_INTEGRATION_TEST_ALLOW_FOREIGN_LISTENERS=1, which turns the error into a warning.

On Linux, such a listener also stops anything else from binding that port on a single address, Harper included, so there the same error covers Harper's HTTPS, MQTT and MQTTS ports too. With the variable set on Linux, the address is handed out even though a Harper node bound to that address under the other listener cannot be ruled out.

API

The lifecycle and utility APIs below are framework-agnostic. They manage Harper child processes and a cross-process loopback address pool. Use them in the setup/teardown hooks of whichever test framework you prefer.

startHarper, setupHarperWithFixture, killHarper and teardownHarper all take the context — the object with a harper property — not the node held in ctx.harper. Passing the node throws a TypeError: the teardown pair tells you to wrap it (teardownHarper({ harper: node })), while the start pair tells you to pass the context the node came from, since starting from the node would publish a fresh one over it and abandon the instance already running. It used to be accepted silently, which left the node running until the runner exited. A context whose harper was never populated is still a no-op, so teardown after a before hook that threw stays safe.

startHarper(ctx, options?)

Allocates a loopback address from the pool, creates a temporary install directory, starts a Harper process, and waits for it to be ready. Populates ctx.harper with the instance details. Call in a setup/before() hook.

Harper runs as its own detached process group, so a runner that dies without running cleanup would otherwise leave it alive holding the fixed ports. On POSIX the instance is registered with a shared instance monitor that reaps it in that case.

The Harper binary is resolved in the following order:

  1. harperBinPath option passed directly to startHarper()
  2. HARPER_INTEGRATION_TEST_INSTALL_SCRIPT environment variable (path to dist/bin/harper.js)
  3. The nearest dist/bin/harper.js in the working directory or its ancestors — the Harper build (harper, harper-pro) your tests run inside, which takes precedence over the harper package npm auto-installs for this package's peer dependency
  4. A harper package installed as a project dependency: the one require('harper') resolves to from the working directory, located through its bin field

Options:

interface StartHarperOptions {
  startupTimeoutMs?: number;   // Idle timeout: max gap between startup output chunks. Default: 60000 (150000 under CI) or HARPER_INTEGRATION_TEST_STARTUP_TIMEOUT_MS
  startupMaxMs?: number;       // Absolute startup ceiling regardless of output. Default: 120000 (300000 under CI) or HARPER_INTEGRATION_TEST_STARTUP_MAX_MS
  config?: object;             // Harper config overrides (passed via HARPER_SET_CONFIG)
  env?: object;                // Additional environment variables for the Harper process
  harperBinPath?: string;      // Explicit path to dist/bin/harper.js
}

Startup readiness is detected by Harper printing successfully started. Rather than a single wall-clock deadline (which makes a slow-but-healthy boot indistinguishable from a hang), the watchdog uses an idle timeout that resets on every chunk of output — so the limit is time-since-last-progress — plus a generous absolute ceiling as a backstop. This is why a slow CI boot that keeps logging no longer trips the timeout.

Behavior change: startupTimeoutMs (and HARPER_INTEGRATION_TEST_STARTUP_TIMEOUT_MS) previously meant an absolute startup deadline. It now means the idle / no-output window. If you relied on it as a hard ceiling to fail slow boots quickly, set startupMaxMs (or HARPER_INTEGRATION_TEST_STARTUP_MAX_MS) instead.

Environment Variables:

  • HARPER_INTEGRATION_TEST_STARTUP_TIMEOUT_MS - Idle startup timeout: max time between chunks of startup output before Harper is treated as hung (resets on output). Default 60000 (150000 under CI).
  • HARPER_INTEGRATION_TEST_STARTUP_MAX_MS - Absolute ceiling on total startup time, regardless of ongoing output. Default 120000 (300000 under CI).
  • HARPER_INTEGRATION_TEST_INSTALL_PARENT_DIR - Parent directory for temp Harper install dirs (default: OS tmpdir)
  • HARPER_INTEGRATION_TEST_INSTALL_SCRIPT - Path to Harper CLI script
  • HARPER_INTEGRATION_TEST_ALLOW_FOREIGN_LISTENERS - Set to 1 or true to start a node on an address whose operations or HTTP port another process already accepts connections on, or that could not be checked for one, with a warning instead of an error (see Other services listening on Harper's ports).

setupHarperWithFixture(ctx, fixturePath, options?)

Like startHarper(), but copies a component directory into the Harper install before starting, so it's available on first boot without a deploy.

killHarper(ctx, options?)

Terminates Harper's whole process tree and waits for it to exit. It sends SIGTERM first, giving Harper a grace period to shut down cleanly (flush RocksDB, release ports, reap workers) before escalating to SIGKILL, then waits briefly for the actual exit. Because Harper is spawned as its own process group (detached on POSIX), the signal targets the group — parent and any child processes — rather than only the direct child; on Windows it uses taskkill /T. A dead process releases its listening sockets, so once killHarper returns the fixed ports are free (it logs a warning when it could not confirm the node stopped). Does not release the loopback address or clean up the install directory. Useful for restart scenarios where the test will call startHarper again.

It also stops a node that restarted itself. Harper's restart operation relaunches the node as a new, detached process and lets the one startHarper spawned exit, so once that process has exited killHarper terminates whichever process <dataRootDir>/hdb.pid names — the pid file harper stop uses — in the same SIGTERM, grace, SIGKILL sequence (on POSIX, waiting for its whole process group), and removes the pid file if the SIGKILL left it behind. The replacement records its pid only after its predecessor has exited, about half a second later on an idle machine. The spawned process prints its readiness line on stdout a second time just before it relaunches, so after that line killHarper waits for the pid file to appear, for up to 5 seconds (15 under CI, or HARPER_INTEGRATION_TEST_RELAUNCH_WAIT_MS) from when the announcing process exits. Harper gives that process up to 15 seconds to exit while RocksDB closes, so the wait does not start any earlier. A node that stopped without relaunching reports nothing and pays no wait. If killHarper cannot confirm the node stopped — an announced replacement never recorded its pid, or a process outlived SIGKILL — it logs a warning, since its ports may still be held. Only the first relaunch is followed this way: when a replacement restarts again, wait for that restart to finish before tearing down.

options.graceMs overrides the SIGTERM→SIGKILL grace period (default 5000, or HARPER_INTEGRATION_TEST_TEARDOWN_GRACE_MS).

teardownHarper(ctx)

Kills Harper's process tree, releases the loopback address back to the pool, and removes the install directory. Call in a teardown/after() hook.

Since killHarper waits for the process tree to exit, its fixed ports (Operations API, HTTP/S, MQTT/S) are already released by the time the address is recycled. As a safety assertion, teardown still verifies those ports are free before recycling — the pool only guarantees the address is bindable, not that these specific ports are free — and if any are still held (a sign a Harper process escaped the kill) it logs a warning and leaves both the address and the install directory in place: the slot stays parked until this process exits, so a later suite cannot co-bind the ports, and the directory is not deleted under the process still holding them.

If the kill itself could not be confirmed — Harper announced a relaunch but the replacement never recorded its pid, or a process outlived SIGKILL — teardown logs a warning and leaves both the install directory and the address in place rather than deleting the directory under a process that may still be running. This holds for a teardown after a killHarper that already warned: the stop stays unconfirmed until that replacement is gone.

Environment Variables:

  • HARPER_INTEGRATION_TEST_TEARDOWN_GRACE_MS - Grace period after SIGTERM before escalating to SIGKILL. Default 5000.
  • HARPER_INTEGRATION_TEST_PORT_RELEASE_TIMEOUT_MS - Max time teardown's safety assertion waits for Harper's ports to be free before recycling the loopback address (normally instant, since the process tree is already dead). Default 5000.
  • HARPER_INTEGRATION_TEST_RELAUNCH_WAIT_MS - After a Harper process that announced a restart relaunch exits, how long killHarper waits for the replacement to record its pid. Default 5000 (15000 under CI).

sendOperation(context, operation)

Helper to POST an operation to the Operations API and assert HTTP 200.

createHarperContext(name?)

Creates a plain object satisfying HarperTestContext, for use outside node:test (e.g. Playwright worker fixtures). The name is optional and used for log directory naming.

// Playwright example
const ctx = createHarperContext(`worker-${workerInfo.workerIndex}`);
await startHarper(ctx);

Types

There are four related TypeScript types — it helps to understand how they compose.

HarperContext is the instance data object stored at ctx.harper after startHarper() resolves. It describes a running Harper instance:

interface HarperContext {
  dataRootDir: string;          // absolute path to the Harper install directory
  admin: { username: string; password: string };
  httpURL: string;              // e.g. 'http://127.0.0.2:9926'
  operationsAPIURL: string;     // e.g. 'http://127.0.0.2:9925'
  hostname: string;             // e.g. '127.0.0.2'
  process: ChildProcess;
  logDir?: string;              // set when HARPER_INTEGRATION_TEST_LOG_DIR is configured
  startupOutput?: { stdout: string; stderr: string }; // captured startup output
}

HarperTestContext is the minimal context shape accepted by startHarper(), setupHarperWithFixture(), killHarper(), and teardownHarper(). It is intentionally loose so it can be satisfied by a plain object or a node:test context:

interface HarperTestContext {
  name?: string;                // used for log directory naming
  harper?: Partial<HarperContext>;  // populated by startHarper()
}

StartedHarperTestContext is the same as HarperTestContext but with harper guaranteed to be a fully populated HarperContext. It is the return type of startHarper() and setupHarperWithFixture(), and the required parameter type of killHarper() and teardownHarper():

interface StartedHarperTestContext extends HarperTestContext {
  harper: HarperContext;
}

ContextWithHarper is for node:test only. It extends both SuiteContext and TestContext from node:test with harper: HarperContext, so you can use it as the typed context parameter in a suite() callback:

// node:test usage
suite('my suite', (ctx: ContextWithHarper) => {
  before(async () => { await startHarper(ctx); });
  after(async () => { await teardownHarper(ctx); });

  test('example', async () => {
    const res = await fetch(ctx.harper.httpURL);
  });
});

If you are not using node:test, use createHarperContext() to create a plain HarperTestContext instead.

Orphaned Instance Monitor (POSIX)

A test runner that dies without running its teardown — SIGKILL, a hard crash, a cancelled CI job — cannot reap the Harper instances it started, because those are deliberately detached into their own process groups so that whole-tree teardown works. Left alone, they hold their loopback address's fixed ports until the machine is rebooted.

To close that gap, startHarper() registers each instance in a small on-disk registry and makes sure a single shared monitor process is running. The monitor is not a per-instance sidecar: one is started on demand per registry directory (per user on the machine, by default), every concurrent runner reuses it, and it exits once the registry has been empty for a while. Registry updates are published by renaming a complete file into place, so a runner killed mid-write leaves the previous registry — and therefore every other runner's reap targets — intact. It scans the registry on an interval and terminates — SIGTERM, then SIGKILL after a grace period — the process group of any instance whose owning runner is gone, or which has outlived its lifetime budget. Instances are matched by PID and process start time, which narrows PID reuse rather than eliminating it — a recycled PID that ps can still describe is rejected, and on a host whose ps cannot report a start time (busybox/Alpine) this degrades to a PID-only check, which the monitor logs on startup.

A record describes a process group, not one process, and the monitor is the only thing that removes one — when that group has no members left. Harper exiting is not the end of the group: a child that ignored the SIGTERM stays in it, still holding the ports, and the record is what remembers the group long enough to escalate to SIGKILL or to reap it later when its runner dies.

The runner's own exit/SIGINT/SIGTERM/SIGHUP handlers still reap instances immediately on any exit it can observe; the monitor only handles the deaths it cannot.

Registry directory layout (${TMPDIR}/harper-integration-test-monitor-${uid} by default — per-user, because signalling another user's process group fails with EPERM and could never have reaped it):

FileContents
registry.jsonThe current monitor and every registered instance (PID, start time, owning runner, loopback address, deadline)
registry.lockCross-process mutex guarding registry.json
monitor.logAppend-only record of monitor start/exit and every reap, with the reason
registry.json.*.pendingA registry update being written, renamed over registry.json once complete so no reader ever sees a partial one. Only present transiently, or left behind by a writer that was killed mid-update

Each managed Harper process also carries HARPER_IT_KIND=harper-instance, HARPER_IT_INSTANCE_ID, and HARPER_IT_OWNER_PID in its environment, and the monitor's command line contains --harper-integration-test-monitor, so both are identifiable from ps / /proc without consulting the registry.

This is POSIX-only: reaping relies on process groups, which Windows does not have. On Windows, registration is skipped and the runner-side cleanup handlers are the only protection.

Environment Variables:

  • HARPER_INTEGRATION_TEST_MONITOR - Set to off to disable registration and the monitor entirely. Default on (POSIX only).
  • HARPER_INTEGRATION_TEST_MONITOR_DIR - Registry directory. Default ${TMPDIR}/harper-integration-test-monitor-${uid}. Point separate runs at separate directories to give them separate monitors.
  • HARPER_INTEGRATION_TEST_MONITOR_INTERVAL_MS - How often the monitor rescans the registry. Default 2000.
  • HARPER_INTEGRATION_TEST_MONITOR_REAP_GRACE_MS - Grace period between the monitor's SIGTERM and SIGKILL. Default 5000.
  • HARPER_INTEGRATION_TEST_MONITOR_IDLE_MS - How long the registry must stay empty before the monitor shuts down. Default 60000.
  • HARPER_INTEGRATION_TEST_INSTANCE_MAX_LIFETIME_MS - Backstop lifetime after which an instance is reaped even if its runner still looks alive. Default 14400000 (4h).

Server Log Capture

When HARPER_INTEGRATION_TEST_LOG_DIR is set, each Harper instance writes its logs to a per-suite subdirectory. Logs are preserved for the lifetime of the log directory. In CI, combine with artifact upload steps that run on failure to capture logs from failing runs.

HARPER_INTEGRATION_TEST_LOG_DIR=/tmp/harper-test-logs npx harper-integration-test-run "integrationTests/**/*.test.ts"

Additionally, startupOutput on HarperContext provides the captured stdout/stderr from Harper startup for programmatic access (e.g. attaching to Playwright test results).

On a test failure, the runner also prints a tail of each involved Harper node's hdb.log to the console. Runs of consecutive lines that are identical once their timestamp is stripped (e.g. a repeated poll) are collapsed into one line plus a (repeated N times) marker before the tail is taken, so noisy repeated lines don't push the actual error out of the printed window. Log files on disk are never modified.

  • HARPER_INTEGRATION_TEST_LOG_TAIL_LINES - Number of trailing (post-collapse) hdb.log lines printed per failed test (one more when the boundary would otherwise split a (repeated N times) marker from the line it counts). Default 200. 0 (or any non-positive value) prints the entire log, uncollapsed.

HarperStartupError

When Harper fails to start or times out, a HarperStartupError is thrown. It extends Error and includes structured stdout and stderr properties for diagnostics:

import { HarperStartupError } from '@harperfast/integration-testing';

try {
  await startHarper(ctx);
} catch (error) {
  if (error instanceof HarperStartupError) {
    console.error('Startup failed:', error.message);
    console.error('stdout:', error.stdout);
    console.error('stderr:', error.stderr);
  }
}

ForeignListenerError

Thrown by getNextAvailableLoopbackAddress, and so by startHarper, when another process already accepts connections on the operations or HTTP port of a newly allocated address, or on Linux on any of its Harper ports through a listener on all interfaces (see Other services listening on Harper's ports). It extends Error with name set to 'ForeignListenerError', plus loopbackAddress and port properties naming what was reached. The address is released before the error is thrown. Set HARPER_INTEGRATION_TEST_ALLOW_FOREIGN_LISTENERS=1 (or true) to get a warning instead.

LoopbackPoolExhaustedError

Thrown by getNextAvailableLoopbackAddress, and so by startHarper, when no address became free within HARPER_INTEGRATION_TEST_LOOPBACK_POOL_TIMEOUT_MS (default 600000, 10 minutes). It extends Error with name set to 'LoopbackPoolExhaustedError'. The message gives the pool range and, for each holding process, the addresses it holds: its pid, the PID namespace when that differs from this process's, and the host name when that differs. It also lists any address held by a process that did not identify itself, and any unclaimed address a Harper node still has bound. The same listing is logged as a [loopback-pool] warning once a wait reaches 30 seconds (or a quarter of the timeout, if shorter). See Loopback Pool Utilities.

targz(dirPath)

Packs and compresses a directory into a base64-encoded tar.gz string. Useful for deploy_component Operations API calls.

Loopback Pool Utilities

These are used internally by startHarper and teardownHarper, but are exported for advanced use cases or custom runner integrations.

  • validateLoopbackAddressPool(): Promise<{ successful: string[]; failed: { loopbackAddress: string; error: Error }[] }> - Validates all pool addresses can be bound to
  • getNextAvailableLoopbackAddress(): Promise<string> - Allocates an address from the pool, waiting for one to be released if all are held; throws a ForeignListenerError if another process already accepts connections on its operations or HTTP port (see Other services listening on Harper's ports), or a LoopbackPoolExhaustedError if the wait outlasts HARPER_INTEGRATION_TEST_LOOPBACK_POOL_TIMEOUT_MS
  • releaseLoopbackAddress(address: string): Promise<void> - Returns an address to the pool
  • releaseAllLoopbackAddressesForCurrentProcess(): Promise<void> - Releases all addresses held by this process

A process holds an address by listening on its claim port (HARPER_INTEGRATION_TEST_LOOPBACK_CLAIM_PORT, default 9924). The kernel allows one listener per address and port, so while one process holds an address, no other process can claim it. The kernel also closes the listener when its process exits, however it exits. This covers every process in one network namespace, which is where two Harper nodes on one address would collide, whatever their PID namespace or TMPDIR. Processes in separate network namespaces, such as most containers, have separate loopback interfaces and cannot collide. Every run on a machine must use the same claim port. Nothing is written to disk.

Allocation takes the first address whose claim port is free, so an address left by a process that died is reused at once. It then skips an address on which Harper's operations, HTTP, HTTPS, MQTT or MQTTS port is still bound, such as by a node that outlived a killed test runner, and tries the next. Replication is not checked: harper-pro's default replication port has no host, so every node binds it on all interfaces. Releasing an address only frees a claim this process holds.

When every address is held, allocation waits up to HARPER_INTEGRATION_TEST_LOOPBACK_POOL_TIMEOUT_MS (default 600000), then throws a LoopbackPoolExhaustedError naming the holders, rather than waiting forever. Raise the timeout if suites legitimately queue for longer than that. To name its holder, a claim answers a connection with one JSON line: its process's pid, PID namespace and host name. Allocation itself never connects to a claim.

Versions 1.0.5 and earlier kept the pool as a file of process IDs in ${TMPDIR}/harper-integration-test-loopback-pool.json. They do not see claims, and this version does not read that file. Until every run on a machine uses this version, the two can hand out one address to two runs before either has started its node. Let runs on the old version finish before switching. The old file can then be deleted.

Node.js Test Runner

Tests executed by this runner must use the node:test API. If you're using a different test framework, use the lifecycle APIs directly and refer to the runner source as a reference implementation.

The included harper-integration-test-run script is the recommended way to run Harper integration tests. It is built on the Node.js node:test API and handles the concurrency configuration described in the original parallelization analysis: because each integration test spawns a resource-intensive Harper process, the safe default concurrency is roughly half of available system parallelism rather than Node.js's default of availableParallelism() - 1.

The runner executes each test file in its own process, which means files are the unit of isolation. For this to work safely and predictably, each test file must adhere to the independent, hermetic, and deterministic properties described in the Testing Ethos. Strictly following these properties enables files to run concurrently without interfering with each other. When a test breaks, that file can be run in isolation to iterate on a fix without re-running everything.

The Harper integration tests are an open source example of this runner and framework in production use.

Running Tests

npx harper-integration-test-run "integrationTests/**/*.test.ts"

Or add to your package.json:

{
  "scripts": {
    "test:integration": "harper-integration-test-run integrationTests/**/*.test.ts"
  }
}

Runner Options

All CLI arguments can be overridden using the associative HARPER_INTEGRATION_TEST_* environment variable.

CLI ArgumentEnvironment VariableDefaultDescription
--concurrency=NHARPER_INTEGRATION_TEST_CONCURRENCYfloor(parallelism/2)+1Number of concurrent test processes
--isolation=modeHARPER_INTEGRATION_TEST_ISOLATIONprocessprocess or none
--shard=index/totalHARPER_INTEGRATION_TEST_SHARD1/1Test sharding for CI
--onlyHARPER_INTEGRATION_TEST_ONLYfalseRun only tests marked with .only

Example - run sequentially without loopback pool:

npx harper-integration-test-run --isolation=none "integrationTests/**/*.test.ts"

Example - CI sharding across 4 jobs:

npm run test:integration -- --shard=1/4
npm run test:integration -- --shard=2/4
npm run test:integration -- --shard=3/4
npm run test:integration -- --shard=4/4

Writing Tests

Test files should be written in ESM TypeScript, end in .test.ts, and begin with a comment block describing what they verify. File names should be short and hyphen-separated — e.g., install.test.ts or application-management.test.ts.

Tests should use suite/describe, test/it, and lifecycle methods (before, beforeEach, after, afterEach) from node:test, with assertions from node:assert/strict.

A test file can contain multiple suites. Suites always run sequentially. Tests within a suite run sequentially by default, but can be made concurrent with { concurrency: true } — in which case each individual test within that suite must itself be independent, hermetic, and deterministic.

Since these tests interact with a running Harper instance, they often validate actual application output: standard streams (stdout/stderr), network responses, or file system state.

Suite Concurrency Example

This file contains two suites with two 1-second tests each. The first suite enables { concurrency: true }, the second does not.

import { suite, test } from 'node:test';
import { setTimeout as sleep } from 'node:timers/promises';

suite('Concurrency Enabled', { concurrency: true }, () => {
  test('1 second', async () => {
    await sleep(1000);
  });

  test('1 second', async () => {
    await sleep(1000);
  });
});

suite('Concurrency Disabled', () => {
  test('1 second', async () => {
    await sleep(1000);
  });

  test('1 second', async () => {
    await sleep(1000);
  });
});
  • The first suite's tests run concurrently → ~1 second
  • The second suite's tests run sequentially → ~2 seconds
  • Suites always run sequentially → total ~3 seconds
▶ Concurrency Enabled
  ✔ 1 second (1001.359083ms)
  ✔ 1 second (1001.860375ms)
✔ Concurrency Enabled (1002.26125ms)
▶ Concurrency Disabled
  ✔ 1 second (1001.122166ms)
  ✔ 1 second (1000.5495ms)
✔ Concurrency Disabled (2001.850625ms)
ℹ tests 4
ℹ duration_ms 3087.786041

Template

/**
 * Description of what this test file verifies.
 */
import { suite, test, before, after } from 'node:test';
import { strictEqual } from 'node:assert/strict';
import { startHarper, teardownHarper, type ContextWithHarper } from '@harperfast/integration-testing';

suite('short description', (ctx: ContextWithHarper) => {
  before(async () => {
    await startHarper(ctx);
  });

  after(async () => {
    await teardownHarper(ctx);
  });

  test('test description', async () => {
    const response = await fetch(ctx.harper.httpURL);
    strictEqual(response.status, 200);
  });
});

Using Other Frameworks

The lifecycle APIs work with any framework that supports async setup and teardown. The runner source is a concrete reference for how the Node.js node:test integration is wired up, including concurrency configuration and shard support. Use it as a starting point when building an integration for another framework.

GitHub Actions Workflow Parallelization

Integration tests should run efficiently and reliably on CI, not just locally. As the number of test files grows — further compounded by verifying multiple operating systems or runtime versions — even a modest test suite can become a large CI operation. The goal is to strike a balance between cost and thoroughness. Tests must always run reliably; correctness is never compromised for speed. Overloaded runners cause flaky failures.

Default GitHub Actions runners have limited performance. They can reasonably handle a single integration test, but parallelizing many of them will often exceed the runner's capabilities. A recommended strategy is to parallelize across multiple runners using workflow matrix jobs combined with test sharding — distributing test files across runners rather than running them all on one.

Larger runners are available but come at a meaningful cost increase. The right trade-off depends on your organization's total CI job volume, since parallel job execution is typically capped at the org level. When wait time for available runners starts to exceed the time saved by parallelization, fewer jobs on larger runners may be more effective.

It's also worth being deliberate about when the full integration suite runs. Running every test across all supported configurations on every commit of every open PR can quickly exhaust concurrency limits. Strategies like file path filter triggers, manually triggered workflows, and limiting the full matrix to merge-time checks can keep CI responsive without sacrificing coverage.

A merge queue is worth considering for this last point — running the complete matrix only after a PR passes initial checks (e.g., a single configuration), then verifying the full matrix before merge. This allows for faster PR iteration while keeping the main branch thoroughly tested.

The Harper integration tests workflow is a real-world example of these strategies in practice.

License

Apache-2.0

版本与发布记录

@harperfast/integration-testing 共发布 24 个版本,首次发布于 2026/04/11;过去 12 个月发布了 24 个版本。最新版本附带 npm provenance 构建来源证明,可追溯到公开的 CI 构建记录。

版本发布日期距今
1.1.0latest2026/11/17今天
1.0.52026/10/141 个月前
1.0.42026/10/141 个月前
1.0.32026/10/091 个月前
1.0.22026/10/091 个月前
1.0.12026/10/071 个月前

其他发布通道:experimental → 0.3.0-0。

关于 @harperfast/integration-testing 的常见问题

@harperfast/integration-testing 是什么?

Integration testing utilities for Harper-based projects. Provides Harper instance lifecycle management, loopback address pooling, and a test runner script.

如何安装 @harperfast/integration-testing?

在项目目录运行 npm install @harperfast/integration-testing,或使用 pnpm add / yarn add / bun add。它还需要宿主项目提供 peer 依赖:harper(^5.0.0)。npm 7 及以上与 pnpm 8 及以上默认会自动安装必需的 peer 依赖,Yarn 需要手动安装。它还提供命令行工具 harper-integration-test-run、harper-integration-test-setup-loopback,可以用 npx harper-integration-test-run 直接运行。

@harperfast/integration-testing 支持 TypeScript 吗?

支持。包内自带 TypeScript 类型声明(package.json 声明了 types 入口),安装后即可获得类型提示,无需再安装 @types 包。

@harperfast/integration-testing 支持 ESM 还是 CommonJS?

以 ES Module 形式发布,推荐使用 import 引入。在 CommonJS 项目中,Node.js 20.19+ / 22.12+ 可以直接 require() 不含顶层 await 的 ES Module,更早的版本需要改用动态 import()。

@harperfast/integration-testing 需要什么版本的 Node.js?

package.json 的 engines 字段要求 Node.js >=20。版本不符时 npm 默认只给出警告,开启 engine-strict 后会拒绝安装。

@harperfast/integration-testing 可以商用吗?

@harperfast/integration-testing 使用 Apache-2.0 许可。宽松许可:允许商业使用、修改和再分发,并附带明确的专利授权;分发时需保留许可与 NOTICE 文件,修改过的文件要注明改动。以上为概括,不构成法律意见,请以许可原文为准。

@harperfast/integration-testing 的下载量有多少?

根据 npm 官方下载统计,@harperfast/integration-testing 在 2026/10/18 – 2026/11/16 这 30 天内被下载 118,298 次,日均约 3,943 次。下载量会把 CI 构建、镜像同步和作为其他包依赖的安装一并计入,适合横向比较热度,不能直接等同于使用人数。

@harperfast/integration-testing 还在维护吗?

最新版本 1.1.0 发布于 2026/11/17(今天);至今共发布 24 个版本,其中过去 12 个月发布了 24 个;首次发布于 2026/04/11。发布频率只是参考信号:功能稳定的工具可能很少需要更新,建议结合源码仓库的 issue 和提交记录判断。