@cogitator-ai/ test-utils (英语)

测试 Cogitator 的公用设备, 以测试自己的代理、 工具、 工作流程和后端, 而无需调用真正的 LLM 或数据库 .

安装

pnpm add -D @cogitator-ai/test-utils

@cogitator-ai/test-utils 给您定型的站台和助手测试您的代理和应用程序,而不使用网络呼叫、付费的API密钥或外部数据库。 见 测试工具 在网站上.

特征

• 锁定 LLM 后端 - 脚本 MockLLMBackend 确定性 chat() 答复和答复 chatStream() 附有请求记录和检查的块
• 模拟记忆适应器 - 进行中 MockMemoryAdapter 记录方法调用线索和条目
• 模拟嵌入服务 - 决定主义 MockEmbeddingService: 平稳的单位向量,来自一袋散列的单词,每个文本可标注,记录每个呼叫
• 测试固定器 - 预配置代理、信息、工具呼叫、工具结果、JSON Schema工具以及运行结果
• 串流帮助 - 排水和收集溪流内容、块和工具调用,或创建模拟字节流
• 同步公用事业 - 投票(waitFor, (中文(简体) ), eventually延迟,和约期援助者
• 后端合规套件 - 测试针对电线协议的自定义后端(OpenAI、Anthropic、双子座、Ollama、Bedrock) startFakeProvider
• 内存符合要求 - 持有内存适配器、内存项目存储器、内存文件存储器和内存存储器

快速启动

用 MockLLMBackend 进行测试

使用短而有效的 Vitest 示例 MockLLMBackend数字 :

import { describe, it, expect } from 'vitest';
import { MockLLMBackend, createToolCall } from '@cogitator-ai/test-utils';

describe('MockLLMBackend', () => {
  it('returns scripted responses and inspects requests', async () => {
    const backend = new MockLLMBackend();
    backend.setResponses([
      { toolCalls: [createToolCall('get_weather', { city: 'Tokyo' })] },
      { content: 'The weather in Tokyo is sunny and 22°C.' },
    ]);

    const first = await backend.chat({
      model: 'test-model',
      messages: [{ role: 'user', content: 'What is the weather in Tokyo?' }],
    });

    expect(first.finishReason).toBe('tool_calls');
    expect(first.toolCalls?.[0].name).toBe('get_weather');

    const second = await backend.chat({
      model: 'test-model',
      messages: [
        { role: 'user', content: 'What is the weather in Tokyo?' },
        { role: 'assistant', content: 'Checking weather...' },
        {
          role: 'tool',
          content: '{"temperature": 22}',
          toolCallId: first.toolCalls![0].id,
          name: 'get_weather',
        },
      ],
    });

    expect(second.content).toBe('The weather in Tokyo is sunny and 22°C.');
    expect(second.finishReason).toBe('stop');

    expect(backend.getCallCount()).toBe(2);
    expect(backend.wasCalledWith((req) => req.model === 'test-model')).toBe(true);
    expect(backend.getLastCall()?.messages).toHaveLength(3);
  });
});

使用MockMemoryAdapter进行测试

使用短而有效的 Vitest 示例 MockMemoryAdapter数字 :

import { describe, it, expect } from 'vitest';
import { MockMemoryAdapter, createUserMessage } from '@cogitator-ai/test-utils';

describe('MockMemoryAdapter', () => {
  it('stores conversation history and records calls', async () => {
    const memory = new MockMemoryAdapter();
    await memory.connect();
    expect(memory.isConnected()).toBe(true);

    const threadResult = await memory.createThread('agent-1', { topic: 'testing' }, 'thread-123');
    expect(threadResult.success).toBe(true);

    const addResult = await memory.addEntry({
      threadId: 'thread-123',
      message: createUserMessage('Remember this note'),
      tokenCount: 4,
    });
    expect(addResult.success).toBe(true);

    const entriesResult = await memory.getEntries({ threadId: 'thread-123' });
    expect(entriesResult.success).toBe(true);
    if (entriesResult.success) {
      expect(entriesResult.data).toHaveLength(1);
      expect(entriesResult.data[0].message.content).toBe('Remember this note');
    }

    expect(memory.getCallsFor('createThread')).toHaveLength(1);
    expect(memory.getCallsFor('addEntry')).toHaveLength(1);

    await memory.disconnect();
    expect(memory.isConnected()).toBe(false);
  });
});

鸡鸡

MockLLMBackend 移动端

MockLLMBackend 执行 LLMBackend (单位:千美元)provider: 'openai')和以脚本响应或流块响应呼叫,同时记录所有收到的请求.

脚本聊天响应

import { MockLLMBackend, createMockLLMBackend, createToolCall } from '@cogitator-ai/test-utils';

const backend = createMockLLMBackend();

// Set a single response for all chat() calls
backend.setResponse({ content: 'Hello, world!' });

// Set multiple responses returned in sequence (the last response repeats)
backend.setResponses([{ content: 'First response' }, { content: 'Second response' }]);

// Shortcut for scripted tool call responses
backend.setToolCallResponse([createToolCall('calculator', { expression: '2 + 2' })]);

// Set default delay (in ms) before answering
backend.setDefaultDelay(50);

流线

import { MockLLMBackend, collectStreamContent } from '@cogitator-ai/test-utils';

const backend = new MockLLMBackend();

// Script a stream for chatStream()
backend.setStreamChunks([{ content: 'Hello' }, { content: ' world' }, { content: '!' }]);

// Or script multiple distinct streams in sequence
backend.setMultipleStreamChunks([[{ content: 'First stream' }], [{ content: 'Second stream' }]]);

const stream = backend.chatStream({
  model: 'test-model',
  messages: [{ role: 'user', content: 'Hi' }],
});

const text = await collectStreamContent(stream);

请检查

// Inspect recorded requests
const count = backend.getCallCount();
const allCalls = backend.getCalls();
const lastCall = backend.getLastCall();

// Predicate match
const wasCalled = backend.wasCalledWith((req) => req.model === 'test-model');

// Reset calls, responses, and stream chunks
backend.reset();

模拟地图

MockMemoryAdapter 执行 MemoryAdapter (单位:千美元)provider: 'memory'),在记忆中使用了本土地图并记录了每次引用.

import { MockMemoryAdapter, createMockMemoryAdapter } from '@cogitator-ai/test-utils';

const memory = createMockMemoryAdapter();

// Lifecycle
await memory.connect();
const connected = memory.isConnected(); // true

// Threads
const thread = await memory.createThread('agent-id', { tag: 'session' }, 'custom-id');
const fetchedThread = await memory.getThread('custom-id');
await memory.updateThread('custom-id', { tag: 'updated' });

// Entries
await memory.addEntry({
  threadId: 'custom-id',
  message: { role: 'user', content: 'Hello' },
  tokenCount: 1,
});
const entries = await memory.getEntries({ threadId: 'custom-id', limit: 10 });
const singleEntry = await memory.getEntry(
  entries.success && entries.data[0] ? entries.data[0].id : ''
);
await memory.deleteEntry('entry-id');
await memory.clearThread('custom-id');
await memory.deleteThread('custom-id');

// Inspection
const calls = memory.getCalls(); // Array<{ method: string; args: unknown[] }>
const threadCalls = memory.getCallsFor('createThread');

// Reset all internal state and calls
memory.reset();
await memory.disconnect();

固定设备

探员

生成具有测试默认值的测试代理配置或代理形状对象 :

import { createTestAgentConfig, createTestAgent } from '@cogitator-ai/test-utils';

// AgentConfig with default name 'test-agent' and model 'mock/test-model'
const config = createTestAgentConfig({
  name: 'custom-agent',
  instructions: 'Custom instructions',
});

// Agent-shaped object implementing the Agent interface
const agent = createTestAgent({
  name: 'test-agent',
});

信件和工具调用

构造强烈的对口消息、工具呼叫和工具结果 :

import {
  createUserMessage,
  createAssistantMessage,
  createSystemMessage,
  createToolMessage,
  createMultimodalMessage,
  createImageUrlMessage,
  createToolCall,
  createToolResult,
  createConversation,
} from '@cogitator-ai/test-utils';

const userMsg = createUserMessage('Hello');
const assistantMsg = createAssistantMessage('Hi there');
const systemMsg = createSystemMessage('You are a test assistant');
const toolMsg = createToolMessage('call_1', '{"result": 42}', 'calculator');

const multimodalMsg = createMultimodalMessage([
  { type: 'text', text: 'Describe this' },
  { type: 'image_url', image_url: { url: 'https://example.com/image.png' } },
]);
const imgMsg = createImageUrlMessage('Look at this', 'https://example.com/image.png');

const call = createToolCall('calculator', { expression: '2 + 2' });
const result = createToolResult(call, { result: 4 });

const turns = createConversation(
  { role: 'user', content: 'Hello' },
  { role: 'assistant', content: 'Hi' }
);

工具

用于测试工具执行和计划通过的预定义的 JSON Schema 工具固定装置 :

import {
  createTestTool,
  createCalculatorTool,
  createWeatherTool,
  createFailingTool,
  createSlowTool,
  createToolSchema,
} from '@cogitator-ai/test-utils';

// Customizable SimpleTool
const customTool = createTestTool({
  name: 'custom_tool',
  description: 'A custom tool',
  execute: async (args) => ({ echo: args }),
});

// Built-in tools
const calc = createCalculatorTool(); // evaluates arithmetic expressions ('2 + 2')
const weather = createWeatherTool(); // returns 22°C sunny weather for city
const failing = createFailingTool('Custom error'); // always throws
const slow = createSlowTool(50); // delays for specified ms before returning

// ToolSchema fixture
const schema = createToolSchema('my_tool', 'Tool description');

运行结果

雾 RunResult 密码消耗代理执行结果的结构:

import {
  createMockRunResult,
  createRunResultWithToolCalls,
  createRunResultWithMessages,
} from '@cogitator-ai/test-utils';

const baseResult = createMockRunResult('Final answer');
const withTools = createRunResultWithToolCalls('Output', [call]);
const withHistory = createRunResultWithMessages('Output', [userMsg, assistantMsg]);

帮助者

串流帮助

帮助消耗和测试同步发电机流(AsyncGenerator<ChatStreamChunk>:

import {
  collectStreamContent,
  collectStreamChunks,
  collectStreamToolCalls,
  createMockReadableStream,
} from '@cogitator-ai/test-utils';

// Drains stream and returns concatenated text
const content = await collectStreamContent(stream);

// Drains stream and returns all chunks as an array
const chunks = await collectStreamChunks(stream);

// Drains stream and returns parsed tool calls
const toolCalls = await collectStreamToolCalls(stream);

// Creates a ReadableStream<Uint8Array> from string chunks (e.g. for SSE testing)
const byteStream = createMockReadableStream(['data: {"text":"hello"}\n\n']);

同步帮助器

处理同步计时、投票和超时的公用事业:

import { waitFor, eventually, delay, withTimeout } from '@cogitator-ai/test-utils';

// Sleep for ms
await delay(100);

// Poll condition until truthy or timeout (defaults: timeout 5000ms, interval 50ms)
const ready = await waitFor(() => checkReady(), { timeout: 2000, interval: 20 });

// Retry assertion until it passes or timeout
await eventually(() => expect(isDone()).toBe(true), { timeout: 3000, interval: 50 });

// Wrap promise with timeout
const value = await withTimeout(fetchData(), 1000);

遵约

用于验证该自定义或第三方的完整工具包 LLMBackend 执行中正确遵守提供者协议、错误处理、取消和流转惯例.

后端合规套件

运行全套符合性测试案例,针对 LLMBackend数字 :

import { describe, it } from 'vitest';
import {
  backendConformanceCases,
  openAIChatWire,
  type BackendConformanceTarget,
} from '@cogitator-ai/test-utils';

const target: BackendConformanceTarget = {
  name: 'MyCustomBackend',
  model: 'gpt-4o',
  wire: openAIChatWire,
  createBackend: (url) => new MyCustomBackend({ baseUrl: url }),
};

describe('MyCustomBackend conformance', () => {
  for (const c of backendConformanceCases(target)) {
    (c.skip ? it.skip : it)(c.name, () => c.run());
  }
});

假的提供服务器

运行一个轻量级的 HTTP 服务器模拟提供者 API :

import { startFakeProvider, sse } from '@cogitator-ai/test-utils';

const server = await startFakeProvider();

server.reply((req) => {
  if (req.path === '/v1/chat/completions') {
    return {
      type: 'stream',
      contentType: 'text/event-stream',
      chunks: [sse({ choices: [{ delta: { content: 'Hi' } }] })],
    };
  }
  return { type: 'json', status: 404, body: { error: 'Not found' } };
});

console.log(`Fake provider listening at: ${server.url}`);

await server.close();

电线协议和编码

流行LLM提供者协议的电线规格和格式化帮助器:

• 电线数字 : openAIChatWire, (中文(简体) ), openAIResponsesWire, (中文(简体) ), anthropicWire, (中文(简体) ), geminiWire, (中文(简体) ), ollamaWire, (中文(简体) ), bedrockConverseWire
• 编码数字 : sse(data, event?), (中文(简体) ), ndjson(data), (中文(简体) ), awsEventStreamMessage(headers, payload)
• 固定线数字 : conformanceRequest, (中文(简体) ), CONFORMANCE_FIXTURE

内存适配器和嵌入存储器

memoryAdapterConformanceCases(target), (中文(简体) ), memoryItemAdapterConformanceCases(target), (中文(简体) ), memoryFileAdapterConformanceCases(target) 财务报告和审定财务报表 embeddingAdapterConformanceCases(target) 检查a MemoryAdapter页:1 MemoryItemAdapter页:1 MemoryFileAdapter 或一个 EmbeddingAdapter 与合同相对 内在的内在 保存: 缺失与 null原子 updateThread,带有等分时间戳、限制、级联、副本的入场命令, listThreads 过滤和呼号,最近的第一搜索,阈值,过滤和删除.

for (const c of memoryAdapterConformanceCases({
  create: async () => {
    const adapter = new MyAdapter();
    await adapter.connect();
    return adapter;
  },
  release: (adapter) => adapter.disconnect().then(() => undefined),
})) {
  (c.skip ? it.skip : it)(c.name, () => c.run());
}

每个案件都使用线程,代理或自嵌入,所以一个数据库可以服务于他们所有人. 嵌入套件也要使用 dimensions。在内存文件套件中 file-find-heads 检查 findFileHeads:每个范围和路径的最新版本,包括删除内容,将一个版本从被忽略的分支中移出,最古老的第一,没有它就没有任何适配器 file-withdraw 检查 withdrawFileVersions:被撤回的版本保持其序列,从不是头. 在内存适配套件中 redacts-an-entry-in-place 检查 redactEntry:一个已编辑的条目保留其位置、作用和工具调用标识,不保留其文本、工具参数或工具结果、记录中的任何内容 metadata.redacted,给出 null 丢失时,一次报告为 entry-redacted,并且,在有分支的适配器上,它保留其母(也当母被删除时),并且拒绝一个事件条目(不检查任何适配器 redactEntry 或者说 onChange) (中文(简体) ). 内存项套件检查项目和实体对字段的回读(日期: Dates, 一个缺失的字段, 永远不会 null页:1, findItems 滤镜、瞄准镜滤镜(visibleTo, (中文(简体) ), scope)和顺序(最先记录的),项目被放出,除非 includeExpired 尽管后来仍然发现有人放过,但用任何普通名称发现的实体,删除和复制。 内存文件套件检查一个版本的内存文件为字段读回字段(删除的) null 在另一个地方的版本(范围、路径和 sequence) 被拒绝 MEMORY_FILE_CONFLICT,当作家竞逐一个地方时, 被拒绝的批次没有保存任何版本, 过滤器( )ids, (中文(简体) ), paths, (中文(简体) ), prefix, (中文(简体) ), visibleTo, (中文(简体) ), scope和他们的顺序(由 createdAt,则 sequence编辑(内容和内容) sha256 替换,无其他改动),删除和复制.

公共出口

公共出口分类如下: src/index.ts 作为:

许可证

麻省理工学院