@elg/ coerce- llm- 输出

coerceLlmOutput 使得通过将LLM输出被强迫成一个有良好型态且经过验证的JSON对象或阵列,以打字安全的方式使用LLM输出成为可能.

安装

npm install @elg/coerce-llm-output zod

或

yarn add @elg/coerce-llm-output zod

使用量

OpenAI 的基本使用

import { coerceLlmOutput } from "@elg/coerce-llm-output";
import OpenAI from "openai";
import { z } from "zod";

const openai = new OpenAI();

const User = z.object({
  id: z.string(),
  name: z.string(),
  email: z.string(),
});

async function main() {
  const chatCompletion = await openai.chat.completions.create({
    model: "gpt-3.5-turbo",
    messages: [
      {
        role: "user",
        content:
          "Generate a JSON user object with the shape " +
          "{ id: string; name: string; email: string }",
      },
    ],
  });

  const output: z.infer<User> = coerceLlmOutput(
    chatCompletion.choices[0].message,
    User,
  );
}

main();

流线

import { coerceLlmOutput } from "@elg/coerce-llm-output";
import OpenAI from "openai";
import { z } from "zod";

const openai = new OpenAI();

const User = z.object({
  id: z.string(),
  name: z.string(),
  email: z.string(),
});

async function main() {
  const stream = await openai.chat.completions.create({
    model: "gpt-3.5-turbo",
    messages: [
      {
        role: "user",
        content:
          "Generate a JSON user object with the shape " +
          "{ id: string; name: string; email: string }",
      },
    ],
    stream: true,
  });
  for await (const chunk of stream) {
    const output: z.infer<User> = coerceLlmOutput(
      chunk.choices[0].message, // TODO: Check if this works? Otherwise use chunk.choices[0].delta.content
      User,
    );
    // process.stdout.write(chunk.choices[0]?.delta?.content || '');
  }
}

main();

阵列

import { coerceLlmOutput } from "@elg/coerce-llm-output";
import OpenAI from "openai";
import { z } from "zod";

const openai = new OpenAI();

const User = z.object({
  id: z.string(),
  name: z.string(),
  email: z.string(),
});

async function main() {
  const chatCompletion = await openai.chat.completions.create({
    model: "gpt-3.5-turbo",
    messages: [
      {
        role: "user",
        content:
          "Generate a JSON array of users. Each user must have the shape " +
          "{ id: string; name: string; email: string }",
      },
    ],
  });

  const output: z.infer<User>[] = coerceLlmOutput(
    chatCompletion.choices[0].message,
    z.array(User),
  );
}

main();

职能

coerceLlmOutput数字 :

1. 提取类似JSON的内容.
2. 解析不完整的 JSON(使用极佳的 双子) (中文(简体) ).
3. 在解析的JSON对象中修复密钥,以匹配zod schema中的密钥. 我们这样做是因为LLMS有时会产生骆驼卡塞德或蛇-cased,或者出乎意料地产生出错的按键.
4. 使用提供的 Zod 方案解析 JSON 对象 .