@happyvertical/smrt-voice (欢呼声)
S-m-r-t生态系统中AI动力语音合成和克隆的语音剖面管理. 管理语音剖面,克隆样本,以及TTS输出,以字级计时进行唇音同步.
安装
pnpm add @happyvertical/smrt-voice
使用量
import { VoiceProfile, VoiceSample, VoiceOutput } from '@happyvertical/smrt-voice';
// Two mutually exclusive creation modes:
// 1. Voice design -- AI generates voice from a natural language prompt
const designed = new VoiceProfile({
name: 'News Anchor',
language: 'en-US',
gender: 'male',
designPrompt: 'Warm, authoritative male voice with clear enunciation',
defaultSpeed: 1.0, // 0.5 - 2.0
defaultPitch: 0, // -20 to 20 semitones
});
await designed.save();
// 2. Voice cloning -- replicate voice from audio sample(s)
const cloned = new VoiceProfile({
name: 'Custom Voice',
language: 'en-US',
sampleAssetId: 'asset-123',
});
await cloned.save();
// Add training samples for cloning (minimum 3 seconds, quality != low)
const sample = new VoiceSample({
voiceProfileId: cloned.id,
assetId: 'asset-456',
duration: 5.2,
transcription: 'Hello, this is a test recording for voice cloning.',
quality: 'high',
sampleRate: 48000,
format: 'wav',
isPrimary: true,
});
await sample.save();
// TTS output with word-level timing for lip-sync
const output = new VoiceOutput({
voiceProfileId: designed.id,
sourceText: 'Welcome to the evening news.',
audioAssetId: 'asset-789',
duration: 2.8,
wordTimings: [
{ word: 'Welcome', start: 0.0, end: 0.4 },
{ word: 'to', start: 0.4, end: 0.5 },
{ word: 'the', start: 0.5, end: 0.6 },
{ word: 'evening', start: 0.6, end: 1.0 },
{ word: 'news', start: 1.0, end: 1.3 },
],
});
// Look up which word is being spoken at a given timestamp
output.getWordAtTime(0.7); // { word: 'evening', start: 0.6, end: 1.0 }
运行时语音适配器
@happyvertical/smrt-voice 存储语音域记录。 运行时间 STT/ TTS 提供者
电话由 @happyvertical/speech。 。 。 .
import { getSpeech } from '@happyvertical/speech';
import { VoiceOutput, VoiceProfile } from '@happyvertical/smrt-voice';
const speech = await getSpeech({
synthesizer: {
type: 'qwen3-tts',
baseUrl: 'http://qwen3-tts.qwen3-tts.svc.cluster.local',
},
});
const profile = await VoiceProfile.find('voice-123');
const spoken = await speech.synthesize({
text: 'Welcome to the evening news.',
voice: profile.toSpeechVoice(),
});
const output = VoiceOutput.fromSynthesizedSpeech({
sourceText: 'Welcome to the evening news.',
voiceProfileId: profile.id,
audioAssetId: 'asset-created-by-your-asset-store',
speech: spoken,
});
API 密码
模型
| 导出 | 说明 |
|---|---|
VoiceProfile | 语音身份有两种模式: designPrompt (AI生成)或 sampleAssetId (克隆) (简体中文) |
VoiceSample | 有质量评级的语音克隆音频培训数据 |
VoiceOutput | 生成 TTS 音频( 扩展内容) , 并带有字级时间用于唇形同步 |
类型
| 导出 | 说明 |
|---|---|
VoiceProfileStatus | 生命周期状态 : pending, (中文(简体) ), processing, (中文(简体) ), ready, (中文(简体) ), failed |
VoiceGender | 性别分类: male, (中文(简体) ), female, (中文(简体) ), neutral |
SampleQuality | 音频质量评分 : low, (中文(简体) ), medium, (中文(简体) ), high |
WordTiming | 每字计时条目: { word, start, end } (单位:秒) |
VoiceOutputMetadata | 音频元数据: sampleRate,格式,频道,比特深度,提供者,模型 |
SpeechVoice, (中文(简体) ), SynthesizedSpeech, (中文(简体) ), TranscriptResult | 运行时间合同从 @happyvertical/speech |
VoiceProfileOptions | 配置创建选项 |
VoiceSampleOptions | 样本创建选项 |
VoiceOutputOptions | 输出创建选项 |
密钥计算属性
VoiceProfile.isCloned编号 :isDesigned-- 正在激活的创建模式VoiceProfile.isReady-- 地位相等readyVoiceProfile.toSpeechVoice()-- 运行时语音合同@happyvertical/speechVoiceSample.meetsMinDuration- 持续时间 3秒VoiceSample.isSuitableForCloning- 满足分钟时间和质量要求VoiceOutput.wordCount编号 :wordsPerSecond-- 从来源Text和持续时间计算VoiceOutput.getWordAtTime(seconds)- 查找在时间戳上说话的单词VoiceOutput.fromSynthesizedSpeech()-- 语音适配器正常输出元数据/模式
依赖关系
@happyvertical/smrt-core-- ORM 和代码生成@happyvertical/smrt-assets- 基础资产管理@happyvertical/smrt-config-- 配置装入@happyvertical/smrt-content-- 内容模型(声音输出扩展内容)@happyvertical/smrt-tenancy-- 多租户范围界定@happyvertical/speech-- 运行时间STT/TTS适配合同