@flatfile/improv

A powerful TypeScript library for building AI-powered applications with three complementary APIs: Solo for simple structured outputs, Agent for complex tool-enabled workflows, and Gig for orchestrating multi-step AI operations. Features type-safe outputs, built-in retry/fallback mechanisms, and support for multiple LLM providers.

Core Features

• 🎯 Solo: Simple, type-safe API for one-off LLM calls with structured output
• 🤖 Agent: Advanced AI agents with tool usage, knowledge bases, and multi-step reasoning
• 🎭 Gig: Orchestrate complex workflows with sequential/parallel execution and dependencies
• 🧩 Pieces: Create reusable workflow components with built-in evaluation support
• 🛠️ Tool Integration: Define and execute custom tools with type-safe parameters using Zod schemas
• 📎 Attachment Support: Handle various types of attachments (documents, images, videos)
• 🔄 Event Streaming: Built-in event system for real-time monitoring and response handling
• 💾 Memory Management: Track and persist conversation history and agent state
• 🔧 Multi-Provider Support: AWS Bedrock, OpenAI, Cohere, Gemini, Cerebras, and more

Table of Contents

• Quick Start
• 📚 Documentation
• New: Solo, Agent, and Gig
  • Solo - Structured LLM Calls
  • Agent - Tool-Enabled Workflows
  • Gig - Workflow Orchestration
  • Pieces - Reusable Components
• Core Components
  • Agent
  • Thread
  • Tool
  • Message
• Event System
• Best Practices
• License
• Contributing
• Follow-up Messages
• State Management
  • Agent State
  • Thread State
  • Tool State
  • Evaluator System
• State Management Patterns
  • Three-Keyed Lock Pattern
• AWS Bedrock Integration
• Model Drivers
• Tool Decorators
• Streaming Support
• Reasoning Capabilities

Quick Start

import { Agent, Tool, BedrockThreadDriver } from '@flatfile/improv';
import { z } from 'zod';

// Create a custom tool
const calculatorTool = new Tool({
  name: 'calculator',
  description: 'Performs basic arithmetic operations',
  parameters: z.object({
    operation: z.enum(['add', 'subtract', 'multiply', 'divide']),
    a: z.number(),
    b: z.number(),
  }),
  executeFn: async (args) => {
    const { operation, a, b } = args;
    switch (operation) {
      case 'add': return a + b;
      case 'subtract': return a - b;
      case 'multiply': return a * b;
      case 'divide': return a / b;
    }
  }
});

// Initialize the Bedrock driver
const driver = new BedrockThreadDriver({
  model: 'anthropic.claude-3-haiku-20240307-v1:0',
  temperature: 0.7,
});

// Create an agent
const agent = new Agent({
  knowledge: [
    { fact: 'The agent can perform basic arithmetic operations.' }
  ],
  instructions: [
    { instruction: 'Use the calculator tool for arithmetic operations.', priority: 1 }
  ],
  tools: [calculatorTool],
  driver,
});

// Create and use a thread
const thread = agent.createThread({
  prompt: 'What is 25 multiplied by 4?',
  onResponse: async (message) => {
    console.log('Agent response:', message.content);
  }
});

// Send the thread
await thread.send();

// Stream the response
const stream = await thread.stream();
for await (const text of stream) {
  process.stdout.write(text); // Print each chunk as it arrives
}

📚 Documentation

Comprehensive guides and API references:

• 📖 Complete Documentation - All guides and references
• 🚀 API Overview - Complete overview of all APIs
• 🎯 Event System Guide - Using events and monitoring
• 📋 Events Reference - Complete event registry
• 🔄 Structured Output - Type-safe LLM responses
• 🤖 Agent & Solo Patterns - When to use each API
• 🧩 Creating Pieces - Building reusable components

New: Solo, Agent, and Gig

Improv now provides simplified APIs for different AI use cases:

Solo - Structured LLM Calls

For simple, one-off LLM calls with structured output:

import { Solo } from '@flatfile/improv';
import { z } from 'zod';

const solo = new Solo({
  driver,
  outputSchema: z.object({
    sentiment: z.enum(["positive", "negative", "neutral"]),
    confidence: z.number().min(0).max(1)
  })
});

const result = await solo.ask("Analyze the sentiment: 'This product is amazing!'");
// result.output is fully typed: { sentiment: "positive", confidence: 0.95 }

Key Features:

• Type-safe structured output with Zod schemas
• Built-in retry logic with exponential backoff
• Fallback driver support
• Streaming support
• Simple API for classification and extraction tasks

Agent - Tool-Enabled Workflows

For complex, multi-step workflows that require tool usage:

import { Agent, Tool } from '@flatfile/improv';

const searchTool = new Tool({
  name: "search",
  description: "Search the knowledge base",
  parameters: z.object({ query: z.string() }),
  executeFn: async ({ query }) => {
    // Your search implementation
    return { results: ["Result 1", "Result 2"] };
  }
});

const agent = new Agent({
  driver,
  tools: [searchTool],
  instructions: [
    { instruction: "Always search before answering", priority: 1 }
  ]
});

const thread = agent.createThread({ 
  prompt: "What are the best practices for error handling?" 
});
await thread.send();

Gig - Workflow Orchestration

Orchestrate multiple AI operations with dependencies and control flow:

import { Gig, PieceDefinition } from '@flatfile/improv';

const gig = new Gig({ 
  label: "Customer Support Workflow", 
  driver 
});

// Add pieces sequentially (default behavior)
gig
  .add("classify", groove => 
    `Classify this support request: "${groove.feelVibe("request")}"`, {
    outputSchema: z.enum(["technical", "billing", "general"])
  })
  
  .add("sentiment", groove => 
    `Analyze sentiment: "${groove.feelVibe("request")}"`, {
    outputSchema: z.enum(["positive", "negative", "neutral"])
  })
  
  // Pieces can access previous results
  .add("research", groove => {
    const category = groove.recall("classify");
    return `Research solutions for ${category} issue`;
  }, {
    tools: [searchTool],
    driver: cheaperModel // Override driver for cost optimization
  })
  
  // Explicit parallel execution when needed
  .parallel([
    ["check_status", "Check system status"],
    ["find_similar", "Find similar resolved issues"]
  ])
  
  .add("respond", groove => {
    const sentiment = groove.recall("sentiment");
    const research = groove.recall("research");
    return `Generate ${sentiment} response using: ${research}`;
  });

// Execute the workflow
const results = await gig.perform({ 
  request: "My invoice is wrong and I'm very frustrated!" 
});

console.log(results.recordings.get("classify")); // "billing"
console.log(results.recordings.get("sentiment")); // "negative"

Key Features:

• Sequential execution by default (predictable flow)
• Explicit parallel() for concurrent operations
• Auto-detection: uses Solo for structured output, Agent for tools
• Driver overrides per piece for optimization
• Access previous results with groove.recall()
• Type-safe with full TypeScript support

Pieces - Reusable Components

Create reusable workflow components that can be shared across projects:

import { PieceDefinition } from '@flatfile/improv';
import { z } from 'zod';

// Define a reusable piece
export const sentimentAnalysis: PieceDefinition<"positive" | "negative" | "neutral"> = {
  name: "sentiment",
  play: (groove) => {
    const text = groove.feelVibe("text");
    return `Analyze sentiment of: "${text}"`;
  },
  config: {
    outputSchema: z.enum(["positive", "negative", "neutral"]),
    temperature: 0.1
  },
  meta: {
    version: "1.0.0",
    description: "Analyzes emotional tone of text"
  }
};

// Use in any Gig
gig.add(sentimentAnalysis);

Organizing Pieces with Evaluations:

src/
  pieces/
    sentiment/
      index.ts      # Piece definition (production)
      eval.ts       # Evaluation data (dev only)

This separation ensures evaluation datasets don't get bundled in production builds.

Reasoning Capabilities

Improv supports advanced reasoning capabilities through the reasoning_config option in the thread driver. This allows the AI to perform step-by-step reasoning before providing a final answer.

import { Agent, BedrockThreadDriver } from '@flatfile/improv';

const driver = new BedrockThreadDriver({
  model: 'anthropic.claude-3-7-sonnet-20250219-v1:0',
  temperature: 1,
  reasoning_config: {
    budget_tokens: 1024,
    type: 'enabled',
  },
});

const agent = new Agent({
  driver,
});

const thread = agent.createThread({
  systemPrompt: 'You are a helpful assistant that can answer questions about the world.',
  prompt: 'How many people will live in the world in 2040?',
});

const result = await thread.send();

console.log(result.last());

This example enables the AI to work through its reasoning process with a token budget of 1024 tokens before providing a final answer about population projections.

Core Components

Agent

The main agent class that manages knowledge, instructions, tools, and conversation threads.

const agent = new Agent({
  knowledge?: AgentKnowledge[],  // Array of facts with optional source and timestamp
  instructions?: AgentInstruction[],  // Array of prioritized instructions
  memory?: AgentMemory[],  // Array of stored thread histories
  systemPrompt?: string,  // Base system prompt
  tools?: Tool[],  // Array of available tools
  driver: ThreadDriver,  // Thread driver implementation
  evaluators?: Evaluator[]  // Array of evaluators for response processing
});

Thread

Manages a single conversation thread with message history and tool execution.

const thread = new Thread({
  messages?: Message[],  // Array of conversation messages
  tools?: Tool[],  // Array of available tools
  driver: ThreadDriver,  // Thread driver implementation
  toolChoice?: 'auto' | 'any',  // Tool selection mode
  maxSteps?: number  // Maximum number of tool execution steps
});

Tool

Define custom tools that the agent can use during conversations.

const tool = new Tool({
  name: string,  // Tool name
  description: string,  // Tool description
  parameters: z.ZodTypeAny,  // Zod schema for parameter validation
  followUpMessage?: string,  // Optional message to guide response evaluation
  executeFn: (args: Record<string, any>, toolCall: ToolCall) => Promise<any>  // Tool execution function
});

Message

Represents a single message in a conversation thread.

const message = new Message({
  content?: string,  // Message content
  role: 'system' | 'user' | 'assistant' | 'tool',  // Message role
  toolCalls?: ToolCall[],  // Array of tool calls
  toolResults?: ToolResult[],  // Array of tool results
  attachments?: Attachment[],  // Array of attachments
  cache?: boolean  // Whether to cache the message
});

Event System

The library uses an event-driven architecture. All major components extend EventSource, allowing you to listen for various events:

// Agent events
agent.on('agent.thread-added', ({ agent, thread }) => {});
agent.on('agent.thread-removed', ({ agent, thread }) => {});
agent.on('agent.knowledge-added', ({ agent, knowledge }) => {});
agent.on('agent.instruction-added', ({ agent, instruction }) => {});

// Thread events
thread.on('thread.response', ({ thread, message }) => {});
thread.on('thread.max_steps_reached', ({ thread, steps }) => {});

// Tool events
tool.on('tool.execution.started', ({ tool, name, args }) => {});
tool.on('tool.execution.completed', ({ tool, name, args, result }) => {});
tool.on('tool.execution.failed', ({ tool, name, args, error }) => {});

Best Practices

1. Choosing the Right API

   • Use Solo for: Classification, extraction, structured output, simple Q&A
   • Use Agent for: Tool usage, multi-step reasoning, complex decision making
   • Use Gig for: Orchestrating multiple operations, workflows with dependencies

2. Piece Design

   • Keep pieces focused on a single task
   • Use descriptive names that indicate the piece's purpose
   • Leverage groove.recall() to access previous results
   • Override drivers for cost optimization (e.g., cheaper models for simple tasks)

3. Tool Design

   • Keep tools atomic and focused on a single responsibility
   • Use Zod schemas for robust parameter validation
   • Implement proper error handling in tool execution
   • Use follow-up messages to guide response evaluation

4. Workflow Organization

   • Default to sequential execution for predictable flow
   • Use parallel() only when operations are truly independent
   • Separate evaluation data into eval.ts files
   • Create reusable pieces for common operations

5. Error Handling & Resilience

   • Configure retry logic with appropriate backoff
   • Set up fallback drivers for critical operations
   • Use onError handlers in Gig pieces
   • Monitor events for debugging and observability

6. Type Safety

   • Define output schemas with Zod for Solo operations
   • Use PieceDefinition<T> for type-safe pieces
   • Leverage TypeScript's type inference with groove.recall()

License

MIT

Contributing

Contributions are welcome! Please read our contributing guidelines for details.

Follow-up Messages

Tools can include follow-up messages that guide the AI's evaluation of tool responses. This is particularly useful for:

• Providing context for tool results
• Guiding the AI's interpretation of data
• Maintaining consistent response patterns
• Suggesting next steps or actions

const tool = new Tool({
  name: 'dataAnalyzer',
  description: 'Analyzes data and returns insights',
  parameters: z.object({
    data: z.array(z.any()),
    metrics: z.array(z.string())
  }),
  followUpMessage: `Review the analysis results:
1. What are the key insights from the data?
2. Are there any concerning patterns?
3. What actions should be taken based on these results?`,
  executeFn: async (args) => {
    // Tool implementation
  }
});

State Management

The library provides several mechanisms for managing state:

Agent State

• Knowledge base for storing facts
• Prioritized instructions for behavior guidance
• Memory system for storing thread histories
• System prompt for base context

Thread State

• Message history tracking
• Tool execution state
• Maximum step limits
• Response handlers

Tool State

• Parameter validation
• Execution tracking
• Result processing
• Event emission

Evaluator System

Evaluators provide a way to process and validate agent responses:

const evaluator: Evaluator = async ({ thread, agent }, complete) => {
  // Process the thread response
  const lastMessage = thread.last();
  
  if (lastMessage?.content.includes('done')) {
    complete();  // Signal completion
  } else {
    // Continue processing
    thread.send(new Message({
      content: 'Please continue with the task...'
    }));
  }
};

const agent = new Agent({
  // ... other options ...
  evaluators: [evaluator]
});

Evaluators can:

• Process agent responses
• Trigger additional actions
• Control conversation flow
• Validate results

State Management Patterns

Three-Keyed Lock Pattern

The three-keyed lock pattern is a state management pattern that ensures controlled flow through tool execution, evaluation, and completion phases. It's implemented as a reusable evaluator:

import { threeKeyedLockEvaluator } from '@flatfile/improv';

const agent = new Agent({
  // ... other options ...
  evaluators: [
    threeKeyedLockEvaluator({
      evalPrompt: "Are there other items to process? If not, say 'done'",
      exitPrompt: "Please provide a final summary of all actions taken."
    })
  ]
});

The pattern works through three distinct states:

stateDiagram-v2
    [*] --> ToolExecution
    
    state "Tool Execution" as ToolExecution {
        [*] --> Running
        Running --> Complete
        Complete --> [*]
    }
    
    state "Evaluation" as Evaluation {
        [*] --> CheckMore
        CheckMore --> [*]
    }
    
    state "Summary" as Summary {
        [*] --> Summarize
        Summarize --> [*]
    }
    
    ToolExecution --> Evaluation: Non-tool response
    Evaluation --> ToolExecution: Tool called
    Evaluation --> Summary: No more items
    Summary --> [*]: Complete

    note right of ToolExecution
        isEvaluatingTools = true
        Handles tool execution
    end note
    
    note right of Evaluation
        isEvaluatingTools = false
        nextMessageIsSummary = false
        Checks for more work
    end note
    
    note right of Summary
        nextMessageIsSummary = true
        Gets final summary
    end note

The evaluator manages these states through:

1. Tool Execution State

   • Tracks when tools are being executed
   • Resets state when new tools are called
   • Handles multiple tool executions

2. Evaluation State

   • Triggered after tool completion
   • Prompts for more items to process
   • Can return to tool execution if needed

3. Summary State

   • Final state before completion
   • Gathers summary of actions
   • Signals completion when done

Key features:

• Automatic state transitions
• Event-based flow control
• Clean event listener management
• Configurable prompts
• Support for multiple tool executions

Example usage with custom prompts:

const workflowAgent = new Agent({
  // ... agent configuration ...
  evaluators: [
    threeKeyedLockEvaluator({
      evalPrompt: "Review the results. Should we process more items?",
      exitPrompt: "Provide a detailed summary of all processed items."
    })
  ]
});

// The evaluator will automatically:
// 1. Let tools execute freely
// 2. After each tool completion, check if more processing is needed
// 3. When no more items need processing, request a final summary
// 4. Complete the evaluation after receiving the summary

This pattern is particularly useful for:

• Processing multiple items sequentially
• Workflows requiring validation between steps
• Tasks with dynamic tool usage
• Operations requiring final summaries

AWS Bedrock Integration

The library uses AWS Bedrock (Claude) as its default LLM provider. Configure your AWS credentials:

// Required environment variables
process.env.AWS_ACCESS_KEY_ID = 'your-access-key';
process.env.AWS_SECRET_ACCESS_KEY = 'your-secret-key';
process.env.AWS_REGION = 'your-region';

// Initialize the driver
const driver = new BedrockThreadDriver({
  model: 'anthropic.claude-3-haiku-20240307-v1:0',  // Default model
  temperature?: number,  // Default: 0.7
  maxTokens?: number,  // Default: 4096
  cache?: boolean  // Default: false
});

Model Drivers

Improv supports multiple LLM providers through dedicated thread drivers:

Available Drivers

Each driver provides a consistent interface while supporting model-specific features:

// OpenAI example
import { OpenAIThreadDriver } from '@flatfile/improv';

const driver = new OpenAIThreadDriver({
  model: 'gpt-4o',
  apiKey: process.env.OPENAI_API_KEY,
  temperature: 0.7
});

// Cohere example
import { CohereThreadDriver } from '@flatfile/improv';

const driver = new CohereThreadDriver({
  model: 'command-r-plus',
  apiKey: process.env.COHERE_API_KEY
});

// Gemini example
import { GeminiThreadDriver } from '@flatfile/improv';

const driver = new GeminiThreadDriver({
  model: 'gemini-1.5-pro',
  apiKey: process.env.GOOGLE_API_KEY
});

// Cerebras example
import { CerebrasThreadDriver } from '@flatfile/improv';

const driver = new CerebrasThreadDriver({
  model: 'llama-4-scout-17b-16e-instruct',
  apiKey: process.env.CEREBRAS_API_KEY
});

Refer to each driver's documentation for available models and specific configuration options.

Tool Decorators

The library provides decorators for creating tools directly on agent classes:

class CustomAgent extends Agent {
  @ToolName("sampleData")
  @ToolDescription("Sample the original data with the mapping program")
  private async sampleData(
    @ToolParam("count", "Number of records to sample", z.number())
    count: number,
    @ToolParam("seed", "Random seed", z.number().optional())
    seed?: number
  ): Promise<any> {
    return { count, seed };
  }
}

This provides:

• Type-safe tool definitions
• Automatic parameter validation
• Clean method-based tools
• Integrated error handling

Streaming Support

The library provides built-in support for streaming responses from the AI model. Keys features:

• Real-time text chunks as they're generated
• Automatic message management in the thread
• Event emission for stream progress
• Error handling and recovery
• Compatible with all thread features including tool calls

const thread = agent.createThread({
  prompt: 'What is 25 multiplied by 4?',
});

const stream = await thread.stream();

for await (const text of stream) {
  process.stdout.write(text);
}

// The final response is also available in the thread
console.log(thread.last()?.content);