使用低声

反应 Hook for OpenAI Whisper API 有语音记录器,实时转录以及内置的静音去除

• 演示

• 实时复制演示

https://user-images.githubusercontent.com/2707253/224465747-0b1ee159-21dd-4cd0-af9d-6fc9b882d716.mp4 请检查isbn=值 (帮助)

• 通知

  正在开发使用Whisper for React Independent.

仓库 : https://github.com/cungsokdara/use-whiper-native 互联网档案馆的存檔,存档日期2013-12-22

进展: https://github.com/cungsokdara/use-whiper-native/issues/1 (中文(简体) )

• 安装

npm i @chengsokdara/use-whisper

yarn add @chengsokdara/use-whisper

• 使用量

import { useWhisper } from '@chengsokdara/use-whisper'

const App = () => {
  const {
    recording,
    speaking,
    transcribing,
    transcript,
    pauseRecording,
    startRecording,
    stopRecording,
  } = useWhisper({
    apiKey: process.env.OPENAI_API_TOKEN, // YOUR_OPEN_AI_TOKEN
  })

  return (
    <div>
      <p>Recording: {recording}</p>
      <p>Speaking: {speaking}</p>
      <p>Transcribing: {transcribing}</p>
      <p>Transcribed Text: {transcript.text}</p>
      <button onClick={() => startRecording()}>Start</button>
      <button onClick={() => pauseRecording()}>Pause</button>
      <button onClick={() => stopRecording()}>Stop</button>
    </div>
  )
}

• 自定义服务器( 保留 OpenAI API 令牌安全)

import { useWhisper } from '@chengsokdara/use-whisper'

const App = () => {
  /**
   * you have more control like this
   * do whatever you want with the recorded speech
   * send it to your own custom server
   * and return the response back to useWhisper
   */
  const onTranscribe = (blob: Blob) => {
    const base64 = await new Promise<string | ArrayBuffer | null>(
      (resolve) => {
        const reader = new FileReader()
        reader.onloadend = () => resolve(reader.result)
        reader.readAsDataURL(blob)
      }
    )
    const body = JSON.stringify({ file: base64, model: 'whisper-1' })
    const headers = { 'Content-Type': 'application/json' }
    const { default: axios } = await import('axios')
    const response = await axios.post('/api/whisper', body, {
      headers,
    })
    const { text } = await response.data
    // you must return result from your server in Transcript format
    return {
      blob,
      text,
    }
  }

  const { transcript } = useWhisper({
    // callback to handle transcription with custom server
    onTranscribe,
  })

  return (
    <div>
      <p>{transcript.text}</p>
    </div>
  )
}

• 实例

• 实时流线记录

import { useWhisper } from '@chengsokdara/use-whisper'

const App = () => {
  const { transcript } = useWhisper({
    apiKey: process.env.OPENAI_API_TOKEN, // YOUR_OPEN_AI_TOKEN
    streaming: true,
    timeSlice: 1_000, // 1 second
    whisperConfig: {
      language: 'en',
    },
  })

  return (
    <div>
      <p>{transcript.text}</p>
    </div>
  )
}

• 在发送到 Whisper 以节省费用之前删除沉默

import { useWhisper } from '@chengsokdara/use-whisper'

const App = () => {
  const { transcript } = useWhisper({
    apiKey: process.env.OPENAI_API_TOKEN, // YOUR_OPEN_AI_TOKEN
    // use ffmpeg-wasp to remove silence from recorded speech
    removeSilence: true,
  })

  return (
    <div>
      <p>{transcript.text}</p>
    </div>
  )
}

• 在挂载的组件上自动开始记录

import { useWhisper } from '@chengsokdara/use-whisper'

const App = () => {
  const { transcript } = useWhisper({
    apiKey: process.env.OPENAI_API_TOKEN, // YOUR_OPEN_AI_TOKEN
    // will auto start recording speech upon component mounted
    autoStart: true,
  })

  return (
    <div>
      <p>{transcript.text}</p>
    </div>
  )
}

• 只要用户在说话,就继续录制

import { useWhisper } from '@chengsokdara/use-whisper'

const App = () => {
  const { transcript } = useWhisper({
    apiKey: process.env.OPENAI_API_TOKEN, // YOUR_OPEN_AI_TOKEN
    nonStop: true, // keep recording as long as the user is speaking
    stopTimeout: 5000, // auto stop after 5 seconds
  })

  return (
    <div>
      <p>{transcript.text}</p>
    </div>
  )
}

• 自定义自定义自定义 API 配置, 当 autoTransis 属实时

import { useWhisper } from '@chengsokdara/use-whisper'

const App = () => {
  const { transcript } = useWhisper({
    apiKey: process.env.OPENAI_API_TOKEN, // YOUR_OPEN_AI_TOKEN
    autoTranscribe: true,
    whisperConfig: {
      prompt: 'previous conversation', // you can pass previous conversation for context
      response_format: 'text', // output text instead of json
      temperature: 0.8, // random output
      language: 'es', // Spanish
    },
  })

  return (
    <div>
      <p>{transcript.text}</p>
    </div>
  )
}

• 依赖关系

  • @cungsokdara/反应-呼-sync 同步反应钩
  • 记录曲c: 交叉浏览器录音机
  • 跛脚 将 wav 编码为 mp3 用于交叉浏览器支持
  • @ffmpeg/ffmpeg: (中文(简体) ) 用于消音功能
  • 汉克: 用于语音检测
  • 轴 : 因为抓取与 Whisper 结束点无关

这些依赖性大多是懒惰地装入的,所以只有在需要时才导入

• API 密码

• 配置对象

• 微声阿皮克

• 返回对象

• 语录

• 路线图

  • 反应-内在支持,将导出为 use-whiper/内在

使用 React 或 React Natural 联系我开发网络或移动应用程序
https://cungsokdara.github.io (英语)