@harness-engineering/local-models

中文文档
0.8.1发布于 2026/11/17MITTypeScriptESM + CJS构建来源已验证近 30 天下载 9624

Hardware-aware local-model recommender, pool manager, and proposal engine for Harness Engineering (LMLM)

原始使用文档
文档目录3 节
  1. Status
  2. Goals (recap)
  3. License

@harness-engineering/local-models

Hardware-aware local-model recommender, pool manager, and proposal engine for Harness Engineering's Local Model Lifecycle Manager (LMLM).

See docs/changes/local-model-lifecycle-manager/proposal.md for the full spec.

Status

Phase 3c — PoolManager orchestrator (with Phase 2d RankedModel orchestrator).

Public surface so far:

  • HardwareDetector / detectHardware (Phase 1) — Apple Silicon, NVIDIA, CPU profiles with fallback warnings.
  • HuggingFaceClient (Phase 2a) — typed wrapper over /api/models and /api/models/:repo with stable error codes and an injected fetcher DI seam.
  • HuggingFaceCache (Phase 2a) — in-memory + atomically-persisted on-disk cache (TTL 24h) for HF responses.
  • loadFrozenSnapshot (Phase 2a) — bundled benchmark snapshot loader the orchestrator falls back to when live sources are unreachable (S4).
  • normalizeQuantId / QUANT_BITS_PER_WEIGHT (Phase 2b) — canonical GGUF + MLX quant table with case-insensitive alias resolution and a conservative fallback for unknown ids.
  • estimateVram (Phase 2b) — four-term VRAM decomposition (weights + KV cache + activations + framework overhead) for any (sizeB, activeB?, quant, contextTokens, kvCacheQuant) tuple. MoE keeps all weights resident; activeB is echoed for the speed estimator.
  • estimateSpeed (Phase 2b) — bandwidth-bound token throughput projection with backend-efficiency multipliers, MoE active-params handling, partial-offload blending toward a CPU floor, and a hard-zero short-circuit for won't-fit candidates. Never throws.
  • gradeEvidence / EVIDENCE_CONFIDENCE (Phase 2c) — five-rung evidence ladder (direct, variant, base, interpolated, self-reported) with calibrated confidence multipliers; self-reported observations are absorbed.
  • applyRecencyDecay (Phase 2c) — exponential age decay (halflife 9 months) plus an optional lineage step penalty (× 0.6 per generation behind the target). Weights clamp at MIN_RECENCY_WEIGHT = 0.05.
  • openLlmLeaderboardSource / huggingFacePopularitySource (Phase 2c) — two seed adapters behind the BenchmarkSource interface. Both take an injected Fetcher so CI never touches the network; every failure path surfaces as a structured SourceWarning rather than throwing.
  • mergeBenchmarks (Phase 2c) — folds evidence × recency × source weight into a single { score (0–100), confidence: 'high' | 'medium' | 'low', contributions } per candidate. Empty input short-circuits to confidence: 'low'; never throws.
  • PoolStateStore (Phase 3a) — atomic on-disk persistence of PoolState to ~/.harness/local-models/pool.json (tmp + rename, O2). Versioned schema with graceful degradation to EmptyPoolState() on missing / malformed / version-mismatched files. Single mutation path (update) always recomputes derived diskUsedGb from the entry sum.
  • planEviction (Phase 3a) — pure lowest-score-LRU planner. Sorts pool entries by (currentScore, lastUsedAt, installedAt) ascending (treating lastUsedAt: null as oldest) and accumulates evictions until the requested freeBudgetGb is met or the pool is exhausted.
  • InstallAdapter / OllamaInstallAdapter / AdvisoryInstallAdapter (Phase 3b) — transport-agnostic install contract plus two concrete implementations. The Ollama adapter speaks /api/pull (NDJSON streaming with typed InstallEvents), /api/delete, /api/tags, and /api/show via an injected Fetcher. The advisory adapter renders copy-paste commands (lms get …, vllm serve …, llama-server -m …) for backends whose lifecycle is operator-driven (D4); install/evict/inspect reject with InstallError('advisory_only', …). InstallError.code is the stable taxonomy higher layers branch on: advisory_only, failed_target_missing (D13), installer_unavailable (S6), install_failed (S7), not_in_pool (D12), parse_failed.
  • nullInstallAdapter (Phase 3b) — InstallAdapter whose methods reject with installer_unavailable. Test seam and the manager's default when LMLM is disabled.
  • rankModels / RankedModel (Phase 2d) — hardware-aware orchestrator that composes estimateVram + estimateSpeed + mergeBenchmarks into a sorted RankedModel[]. Won't-fit candidates are filtered by default (F3 / Q3); options.includeUnfit keeps them at the bottom with score: 0. Per-row evidence reflects the weakest grade among contributions so a single self-reported observation flags the row, while the full per-contribution breakdown lives on benchmarkScore.contributions for the dashboard tooltip. Deterministic sort: score desc → estimatedTokPerSec desc → hfRepoId code-point asc. scaleScore folds the merge's confidence label and the speed estimator's confidence band into the orchestrator-level score so a 'high'-confidence fit outranks a 'low'-confidence one at equal raw value (Q4, Q5). Never throws; degraded paths surface as RankerWarning[] (e.g. snapshot_unavailable for empty-snapshot calls, S4).
  • LiveObservation (Phase 2d) — BenchmarkObservation extended with hfRepoId so the orchestrator can pair source-adapter output with a candidate without re-stitching the model dimension. Phase 6's scheduler will refactor the source adapters to emit this shape directly.
  • Parity fixtures (Phase 2d) — tests/ranker/parity/m3-max-36gb.json and rtx-4090-24gb.json pin the top-1 model id + a [scoreMin, scoreMax] band for the hardware called out in spec success criteria Q1 + Q2. The bundled seed benchmark snapshot is the source of truth for the assertions; CI never invokes whichllm.
  • PoolManager (Phase 3c) — high-level orchestrator composing PoolStateStore + planEviction + an InstallAdapter. install runs the full slot pipeline in one call (allowlist gate → idempotent short-circuit → optional installer.inspect for size → capacity check → pre-commit installer.evict in lowest-score-LRU order → installer.install → append + persist). Budget enforcement is hard at the engine layer (S5: budget_exceeded short-circuits before the installer is invoked). install_failed triggers best-effort partial-byte cleanup via installer.evict (S7); installer_unavailable does not (S6); failed_target_missing does not (D13). evict invokes the installer first, mutates pool state only after a successful or D12-reconciled reply (not_in_pool collapses to silent reconciliation). reconcile() prunes pool entries the installer no longer reports (D12 primitive; F10's scheduler wires the timer in Phase 6). Bookkeeping seams markUsed and updateScores persist once per call; configurePool(partial) is the Phase 7 CLI seam for pool {set-budget, allow-org, allow-family}; snapshot() + isAllowed({ hfRepoId, family? }) are the read-only seams for the resolver / proposal engine / dashboard.

The LocalModelResolver integration (Phase 4), proposal engine + schema generalization (Phase 5), background scheduler (Phase 6), and HTTP / WS / CLI / dashboard surfaces (Phases 7–8) ship per the spec.

Goals (recap)

  • Detect the operator's hardware (Apple Silicon / NVIDIA / CPU) and rank Hugging Face models for that hardware.
  • Manage a disk-budget-bounded pool of installed Ollama models within an operator-approved org/family allowlist.
  • Propose pool changes through the existing hermes-phase-4 review queue with single approve/reject UX.
  • Drive recommendations via the live HuggingFace API with a frozen-snapshot fallback for offline environments.

License

MIT

版本与发布记录

@harness-engineering/local-models 共发布 26 个版本,首次发布于 2026/06/03;过去 12 个月发布了 26 个版本。最新版本附带 npm provenance 构建来源证明,可追溯到公开的 CI 构建记录。

版本发布日期距今
0.8.1latest2026/11/17今天
0.8.02026/11/125 天前
0.7.122026/10/2027 天前
0.7.112026/09/092 个月前
0.7.102026/09/082 个月前
0.7.92026/09/052 个月前

关于 @harness-engineering/local-models 的常见问题

@harness-engineering/local-models 是什么?

Hardware-aware local-model recommender, pool manager, and proposal engine for Harness Engineering (LMLM)

如何安装 @harness-engineering/local-models?

在项目目录运行 npm install @harness-engineering/local-models,或使用 pnpm add / yarn add / bun add。

@harness-engineering/local-models 支持 TypeScript 吗?

支持。包内自带 TypeScript 类型声明(package.json 声明了 types 入口),安装后即可获得类型提示,无需再安装 @types 包。

@harness-engineering/local-models 支持 ESM 还是 CommonJS?

同时提供 ES Module 与 CommonJS 入口,import 和 require() 都可以使用,打包工具会自动选择合适的版本。

@harness-engineering/local-models 可以商用吗?

@harness-engineering/local-models 使用 MIT 许可。宽松许可:允许商业使用、修改和再分发,通常只需在分发时保留原版权与许可声明。以上为概括,不构成法律意见,请以许可原文为准。

@harness-engineering/local-models 的下载量有多少?

根据 npm 官方下载统计,@harness-engineering/local-models 在 2026/10/17 – 2026/11/15 这 30 天内被下载 9,624 次,日均约 321 次。下载量会把 CI 构建、镜像同步和作为其他包依赖的安装一并计入,适合横向比较热度,不能直接等同于使用人数。

@harness-engineering/local-models 还在维护吗?

最新版本 0.8.1 发布于 2026/11/17(今天);至今共发布 26 个版本,其中过去 12 个月发布了 26 个;首次发布于 2026/06/03。发布频率只是参考信号:功能稳定的工具可能很少需要更新,建议结合源码仓库的 issue 和提交记录判断。