@ copilotkit/ 语音

为 CopilotKit 提供音频复制服务.

设置

pnpm add @copilotkit/voice openai

import { CopilotRuntime, createCopilotEndpoint } from "@copilotkit/runtime";
import { TranscriptionServiceOpenAI } from "@copilotkit/voice";
import OpenAI from "openai";

const runtime = new CopilotRuntime({
  agents: { default: yourAgent },
  transcriptionService: new TranscriptionServiceOpenAI({
    openai: new OpenAI({ apiKey: process.env.OPENAI_API_KEY }),
  }),
});

一旦配置完毕,聊天UI会显示一个麦克风按钮. 用户可以记录音频,音频会被转录并被插入输入字段作为文本.

笔译服务

用途 OpenAI 低调 用于复制.

new TranscriptionServiceOpenAI({
  openai: new OpenAI({ apiKey: "..." }), // required
  model: "whisper-1", // default
  language: "en", // optional, ISO-639-1 code
  prompt: "Technical discussion context", // optional, helps with domain terms
  temperature: 0, // optional, 0 = deterministic
});

自定义提供者

扩展 TranscriptionService 从运行时间 :

import {
  TranscriptionService,
  TranscribeFileOptions,
} from "@copilotkit/runtime";

class MyTranscriptionService extends TranscriptionService {
  async transcribeFile(options: TranscribeFileOptions): Promise<string> {
    // options.audioFile, options.mimeType, options.size
    return "transcribed text";
  }
}