@happyvertical/smrt-images

中文文档
0.62.1发布于 2026/11/17MITTypeScriptES Module近 30 天下载 1.9万

Image asset management with AI-powered categorization, search, editing, and metadata extraction for SMRT framework

原始使用文档
文档目录13 节
  1. Photographic character setup
  2. Installation
  3. Usage
  4. API
  5. Models
  6. Collections
  7. Services
  8. Key Types
  9. Optional vocabulary decisions
  10. Dependencies
  11. Contributor guide
  12. Local photographic head isolation
  13. Image processor dependencies

@happyvertical/smrt-images

Image asset management with AI-powered categorization, editing, and metadata extraction for the s-m-r-t framework. Extends Asset from smrt-assets via cross-package STI -- stored in the same assets table with a distinct _meta_type.

Photographic character setup

The staged setup surface keeps uploaded pixels local during head isolation. The optional SDK browser segmenter runs its pinned CPU model on the device, selects hair and face-skin confidence masks, and exports a tightly cropped transparent PNG while preserving the original RGB bytes. After the user reviews the head and explicitly continues, the vision request receives only that crop and returns mouth corners and chin. The s-m-r-t package derives the Canadian horizontal split rig deterministically, while the animation package owns contour validation and motion. This is semantic segmentation, not fine-hair matting; future motion styles are separate. The split has no artificial mouth-interior fill: opening it reveals the host's transparent background through the actual photograph seam.

The expressive preview keeps the neutral rig unchanged. While an approved speech envelope is playing, the host maps it through a bounded gain to the jaw gap and small head/jaw tilts; silence, cancellation, replacement, completion, and reduced-motion all restore the neutral pose. The Canadian split's vertical travel is derived from the measured mouth-to-chin span and the canvas receives matching transparent bottom padding, so a larger opening never clips the PNG.

The mouth split is derived locally from MediaPipe FaceLandmarker lip-corner and chin landmarks on the isolated browser image. The pinned model is integrity checked before CPU inference; one clear face is required, cancellation is honored between loading and detection, and no image bytes are sent to a vision provider for this stage.

PhotoCutoutProfileStore is the persistence seam. A consuming server supplies its authenticated profile, tenant, AssetRuntime, profile resolver, and authorization callback; the store saves PNG bytes as an owner-bound canonical asset, links it to the profile as character_cutout, and stores a versioned rig manifest that refers to the stable character_cutout asset key. The host maps that key to returned durable bytes; UUIDs and expiring URLs are never baked into the rig. It rejects another tenant, declined authorization, malformed or unsupported selected manifests, invalid PNG dimensions, and missing saved bytes. The store compensates only the just-created owned asset if an unlinked profile write fails. Authorized readers can list stable saved-setup summaries without loading PNG bytes; malformed linked manifests are omitted from that gallery so valid offerings remain recoverable. Readers can then select only an existing owner- and tenant-bound character_cutout asset; an omitted asset id still loads the latest setup. PhotoCutoutSetup accepts host-provided save/load callbacks and never uses browser storage as an asset catalog. The chat workbench's local callback bridge is deliberately loopback-only and fixed to an externally provisioned temporary profile; it demonstrates canonical AssetRuntime/Profile storage but is not a production authorization mechanism.

Persistence test design: focused unit coverage exercises authorized and denied actor cases, tenant mismatch, PNG/frame rejection, malformed/versioned manifest omission from galleries and rejection on selection, selected/latest gallery loads, failed-link compensation, and an adapter that links before throwing. The SQLite integration test creates the AssetRuntime, Profile, and ProfileAsset link inside one transaction executor, then reloads after commit. The service is adapter-neutral; PostgreSQL execution belongs to the consuming application's database suite because this package owns no deployment database.

For mouth landmark analysis, configure a vision-capable model through the server-only AI configuration (for example SMRT_CHAT_DEV_MODEL). Review the result against the actual crop: valid geometry does not guarantee accurate mouth placement. The UI never exposes provider credentials or model selection.

The helper release consumes the published SDK family through the workspace catalog. s-m-r-t keeps the SDK catalog and overrides aligned atomically; the bash scripts/check-sdk-versions.sh release gate enforces that boundary.

Installation

pnpm add @happyvertical/smrt-images

Usage

import {
  Image, ImageCollection,
  ImageCategorizer, ImageEditor, ImageDeriver,
  ImageMetadataExtractor, ImageSearch, UpstreamManager,
} from '@happyvertical/smrt-images';

// Create and query images
const images = await ImageCollection.create({ db });
const image = await images.create({
  name: 'hero.jpg',
  url: 'https://cdn.example.com/hero.jpg',
  mimeType: 'image/jpeg',
  width: 1920,
  height: 1080,
});
await image.save();

// Computed properties from dimensions
image.isLandscape;  // true
image.aspectRatio;  // 1.778
image.isHighResolution();  // false (below 4K)

// Metadata categorization (returns tags, description, confidence, subjects)
const categorizer = new ImageCategorizer({ ai: aiConfig });
const result = await categorizer.categorize(image);
// Auto-tag: applies tags and sets description/alt text
await categorizer.autoTag(image, assetCollection);

// AI alt text generation via this.do()
const altText = await image.generateAltText();

// Standard editing (resize, crop, convert, thumbnail)
// Each operation creates a new derivative Image linked via parentId
const editor = new ImageEditor(store, images, { ai: aiConfig });
const thumb = await editor.thumbnail(image, 256);
const resized = await editor.resize(image, 800, 600);
const webp = await editor.convert(image, 'webp');

// AI-powered editing and variation generation
const edited = await editor.edit(image, 'add warm sunset tones');
const variations = await editor.generateVariation(image, 'winter theme', { count: 3 });

API

Models

ExportDescription
ImageSTI subclass of Asset with width, height, alt, and computed aspectRatio/isLandscape/isPortrait/isSquare

Collections

ExportDescription
ImageCollectionDimension/orientation filters: getByMinDimensions(), getByAspectRatio(), getLandscape(), getPortrait(), getSquare(), getHighResolution(), getMissingAltText()

Services

ExportDescription
ImageCategorizerMetadata categorization returning tags, description, confidence, subjects. autoTag() applies results to the image.
ImageDeriverDerive new images from sources + AI prompts with generic provenance links via AssetAssociation
ImageEditorResize, crop, convert, thumbnail (via @happyvertical/images) + AI editing. Creates derivative Image records linked via parentId.
ImageMetadataExtractorExtract dimensions, format, EXIF from image buffers
ImageSearchText search across name/description/alt with orientation filters
UpstreamManagerImport from external providers with provenance tracking

Key Types

ImageOptions, CategoryResult, DeriveOptions, ImageMetadataResult, ImageSearchOptions, AssetSourceAdapter, SourceAsset, SourceAssetMetadata

Optional vocabulary decisions

ImageCategorizer remains generative unless both decisions and a tag or subject vocabulary are supplied. A vocabulary is a readonly array of { name, description? } entries or an image resolver. Offered tag and subject labels are independent predicates: names are exact and case-sensitive, and only probabilities above 0.5 are selected. Tags and subjects can be constrained independently; an absent vocabulary keeps that generated field, while an explicit empty vocabulary returns no labels for that field without a decision provider call. Vocabulary entries are limited to 64 entries, 128-character names, and 512-character descriptions. Resolver and configured-provider errors propagate.

Use categorizeDetailed() for selected-label probabilities and decision provenance. The existing CategoryResult.confidence remains the generative categorizer's confidence and is not a calibrated decision certainty.

This path classifies only curated metadata: image name, description, MIME type, and dimensions. categorize(image, buffer) still ignores buffer; it does not send image bytes, EXIF, source URLs, or the metadata blob to either provider. A supported pixel/vision feature requires a separately scoped API.

Dependencies

  • @happyvertical/smrt-core -- ORM and code generation
  • @happyvertical/smrt-assets -- base Asset model and AssetStore
  • @happyvertical/smrt-tenancy -- multi-tenant scoping
  • @happyvertical/ai -- AI categorization and image generation
  • @happyvertical/images -- resize, crop, convert, thumbnail operations

Contributor guide

See AGENTS.md for package architecture, invariants, validation, and contributor guidance.

Local photographic head isolation

PhotoCutoutSetup isolates hair and facial skin on the user's device through the optional SDK @happyvertical/images/segmentation entry point. Original RGB values are preserved in the transparent PNG; alpha derives from a semantic pixel mask rather than a model-authored polygon. The component accepts segmentationAssets for a same-origin prepared asset directory (default /api/dev-image-segmentation). Prepare pinned runtime/model assets with SDK images prepare:segmentation; the chat workbench has a dev-only allowlisted asset route. Hair/face policy and review/retry live in s-m-r-t, generic inference and PNG encoding in SDK. Photographs are not uploaded during isolation. Mouth analysis remains the separately requested next stage.

The setup flow presents four explicit stages: choose a photo, isolate the head, set the mouth from local face landmarks, then save. Each stage exposes one next action; redo and saved-character loading remain secondary actions. On wide screens the isolated head and animated preview share a compact row, and they stack on narrow screens. Loading, cancellation, and failures preserve the recoverable prior stage.

Saving adds the reviewed character to the authorized saved gallery. It does not select or activate a helper; the consuming surface handles that explicit follow-up. The 256×256 model can soften fine strands and low-confidence boundaries. Users review the head before continuing; loading errors and cancellation retain the selected photo. Local development uses the explicitly authorized SDK #1370 sibling link, to be replaced by released family pins at coordinated delivery.

Image processor dependencies

s-m-r-t images uses its required Sharp backend for local operations. Jimp is an optional peer of the underlying SDK rather than a dependency of this package; applications that explicitly select that SDK backend install the peer themselves.

版本与发布记录

@happyvertical/smrt-images 共发布 365 个版本,首次发布于 2026/06/23;过去 12 个月发布了 365 个版本。

版本发布日期距今
0.62.1latest2026/11/17今天
0.62.02026/11/17今天
0.61.62026/11/161 天前
0.61.52026/11/161 天前
0.61.42026/11/161 天前
0.61.32026/11/152 天前

关于 @happyvertical/smrt-images 的常见问题

@happyvertical/smrt-images 是什么?

Image asset management with AI-powered categorization, search, editing, and metadata extraction for SMRT framework

如何安装 @happyvertical/smrt-images?

在项目目录运行 npm install @happyvertical/smrt-images,或使用 pnpm add / yarn add / bun add。它还需要宿主项目提供 peer 依赖:svelte(^5.57.0)。npm 7 及以上与 pnpm 8 及以上默认会自动安装必需的 peer 依赖,Yarn 需要手动安装。

@happyvertical/smrt-images 支持 TypeScript 吗?

支持。包内自带 TypeScript 类型声明(package.json 声明了 types 入口),安装后即可获得类型提示,无需再安装 @types 包。

@happyvertical/smrt-images 支持 ESM 还是 CommonJS?

以 ES Module 形式发布,推荐使用 import 引入。在 CommonJS 项目中,Node.js 20.19+ / 22.12+ 可以直接 require() 不含顶层 await 的 ES Module,更早的版本需要改用动态 import()。

@happyvertical/smrt-images 可以商用吗?

@happyvertical/smrt-images 使用 MIT 许可。宽松许可:允许商业使用、修改和再分发,通常只需在分发时保留原版权与许可声明。以上为概括,不构成法律意见,请以许可原文为准。

@happyvertical/smrt-images 的下载量有多少?

根据 npm 官方下载统计,@happyvertical/smrt-images 在 2026/10/17 – 2026/11/15 这 30 天内被下载 18,727 次,日均约 624 次。下载量会把 CI 构建、镜像同步和作为其他包依赖的安装一并计入,适合横向比较热度,不能直接等同于使用人数。

@happyvertical/smrt-images 还在维护吗?

最新版本 0.62.1 发布于 2026/11/17(今天);至今共发布 365 个版本,其中过去 12 个月发布了 365 个;首次发布于 2026/06/23。发布频率只是参考信号:功能稳定的工具可能很少需要更新,建议结合源码仓库的 issue 和提交记录判断。