@genkit-ai/google-genai

中文文档
1.44.0发布于 2026/11/04Apache-2.0TypeScriptESM + CJS近 30 天下载 67.4万

Genkit AI framework plugin for Google AI & Vertex APIs, including Gemini APIs.

原始使用文档
文档目录23 节
  1. Installation
  2. Configuration
  3. Using the Gemini API ( googleAI )
  4. Using Gemini Enterprise Agent Platform ( vertexAI )
  5. Using Both Gemini API ( googleAI ) and the Gemini Enterpris…
  6. Usage Examples
  7. Text Generation (Gemini)
  8. Text Embedding
  9. Video Understanding (agentic processing)
  10. Image Generation
  11. Video Generation
  12. Music Generation (Lyria)
  13. Text-to-Speech
  14. Autonomous Agents (Deep Research & Antigravity)
  15. Interactions API Architecture & response.raw
  16. Model Routing
  17. Model Routing ( vertexAI )
  18. Standard Genkit Output vs. response.raw
  19. Key Differences on the Interactions Path
  20. Key Differences
  21. Feedback
  22. Links
  23. License

Google GenAI plugin for Genkit

This plugin provides a unified interface to connect with Google's generative AI models, offering access through both the Gemini API and the Gemini Enterprise Agent Platform. It is a replacement for the previous googleAI and vertexAI plugins.

Building with a coding agent? Install the Genkit JS skill first.

npx skills add genkit-ai/skills --skill developing-genkit-js

It teaches your agent the current Genkit JS APIs and common gotchas. Source, manual install and skills for other languages: genkit-ai/skills.

Note on Naming: In April 2026, Google Cloud rebranded Vertex AI to the Gemini Enterprise Agent Platform. To maintain backward compatibility with Google Cloud's underlying API infrastructure and existing deployments, this plugin retains the vertexAI namespace, export names, and configuration keys.

Official documentation:

Installation

npm i --save @genkit-ai/google-genai

Configuration

This unified plugin exports two main initializers:

  • googleAI: Allows access to models via the Gemini API using API key authentication.
  • vertexAI: Allows access to models via the Gemini Enterprise Agent Platform (formerly Vertex AI). Authentication can be done via Google Cloud Application Default Credentials (ADC) or a simpler API Key for Express Mode.

You can configure one or both in your Genkit setup depending on your needs.

Using the Gemini API (googleAI)

Ideal for quick prototyping and access to models available in Google AI Studio.

Authentication: Requires a Google AI API Key, which you can get from Google AI Studio. You can provide this key by setting the GEMINI_API_KEY or GOOGLE_API_KEY environment variables, or by passing it in the plugin configuration.

import { genkit } from 'genkit';
import { googleAI } from '@genkit-ai/google-genai';

const ai = genkit({
  plugins: [
    googleAI(),
    // Or with an explicit API key:
    // googleAI({ apiKey: 'your-api-key' }),
  ],
});

Using Gemini Enterprise Agent Platform (vertexAI)

Suitable for applications leveraging Google Cloud's AI infrastructure.

Authentication Methods:

  • Application Default Credentials (ADC): The standard method for most platform use cases, especially in production. It uses the credentials from the environment (e.g., service account on GCP, user credentials from gcloud auth application-default login locally). This method requires a Google Cloud Project with billing enabled and the Gemini Enterprise Agent Platform API enabled.
  • Platform Express Mode: A streamlined way to try out many features using just an API key, without needing to set up billing or full project configurations. This is ideal for quick experimentation and has generous free tier quotas. Learn More about Express Mode.
import { genkit } from 'genkit';
import { vertexAI } from '@genkit-ai/google-genai';

const ai = genkit({
  plugins: [
    // Using Application Default Credentials (Recommended for full features)
    vertexAI({ location: 'us-central1' }), // Regional endpoint
    // vertexAI({ location: 'global' }),      // Global endpoint

    // OR

    // Using Express Mode (Easy to start, some limitations)
    // Get an API key from the Agent Platform Studio Express Mode setup.
    // vertexAI({ apiKey: process.env.VERTEX_EXPRESS_API_KEY }),
  ],
});

Note: When using Express Mode, you do not provide projectId and location in the plugin config.

Using Both Gemini API (googleAI) and the Gemini Enterprise Agent Platform (vertexAI)

You can configure both plugins if you need to access models or features from both services.

import { genkit } from 'genkit';
import { googleAI, vertexAI } from '@genkit-ai/google-genai';

const ai = genkit({
  plugins: [
    googleAI(),
    vertexAI()
  ],
});

Usage Examples

Access models and embedders through the configured plugin instance (googleAI or vertexAI).

Text Generation (Gemini)

With googleAI:

import { genkit } from 'genkit';
import { googleAI } from '@genkit-ai/google-genai';

const ai = genkit({
  plugins: [googleAI()],
});

const response = await ai.generate({
  model: googleAI.model('gemini-flash-latest'),
  prompt: 'Tell me something interesting about Google AI.',
});

console.log(response.text());

With vertexAI:

import { genkit } from 'genkit';
import { vertexAI } from '@genkit-ai/google-genai';

const ai = genkit({
  plugins: [vertexAI()],
});

const response = await ai.generate({
  model: vertexAI.model('gemini-3.1-pro-preview'),
  prompt: 'Explain Gemini Enterprise Agent Platform in simple terms.',
});

console.log(response.text());

Text Embedding

With googleAI:

import { genkit } from 'genkit';
import { googleAI } from '@genkit-ai/google-genai';

const ai = genkit({
  plugins: [googleAI()],
});

const embeddings = await ai.embed({
  embedder: googleAI.embedder('gemini-embedding-2'),
  content: 'Embed this text.',
});

With vertexAI:

import { genkit } from 'genkit';
import { vertexAI } from '@genkit-ai/google-genai';

const ai = genkit({
  plugins: [vertexAI()],
});

const embeddings = await ai.embed({
  embedder: vertexAI.embedder('text-embedding-005'),
  content: 'Embed this text.',
});

Video Understanding (agentic processing)

With googleAI, newer models (e.g. gemini-4-argon, gemini-3.8-flash) can explore a video and load only the parts they need, instead of reading frames at a fixed rate. This works better for long videos. Set it per video with metadata.processing ('agentic', or 'static', the default):

const response = await ai.generate({
  model: googleAI.model('gemini-4-argon'),
  prompt: [
    {
      media: { url: 'https://www.youtube.com/watch?v=QO30-W3b_vw' },
      metadata: { processing: 'agentic' },
    },
    { text: 'What happens in this video? List the main moments with timestamps.' },
  ],
});

The model's loading steps come back as custom.processingCall and custom.processingResult parts. Keep them in the history for follow-up questions. response.usage.custom.toolUseTokens counts the tokens loaded.

Image Generation

With googleAI (Gemini Image):

import { genkit } from 'genkit';
import { googleAI } from '@genkit-ai/google-genai';

const ai = genkit({
  plugins: [googleAI()],
});

const response = await ai.generate({
  model: googleAI.model('gemini-nano-banana-2.1'),
  prompt: 'A beautiful watercolor painting of a castle in the mountains.',
});

const generatedImage = response.media;

With vertexAI (Gemini Image):

import { genkit } from 'genkit';
import { vertexAI } from '@genkit-ai/google-genai';

const ai = genkit({
  plugins: [vertexAI()],
});

const response = await ai.generate({
  model: vertexAI.model('gemini-3.1-flash-image-preview'),
  prompt: 'A beautiful watercolor painting of a castle in the mountains.',
});

const generatedImage = response.media;

Video Generation

With googleAI (Gemini Omni):

import { genkit } from 'genkit';
import { googleAI } from '@genkit-ai/google-genai';

const ai = genkit({
  plugins: [googleAI()],
});

const response = await ai.generate({
  model: googleAI.model('gemini-omni-1.1-flash'),
  prompt: 'An origami butterfly flaps its wings and flies out of the french doors into the garden.',
  config: {
    videoFormat: { aspectRatio: '16:9', resolution: '720p' },
  },
});

// An MP4 video (video/mp4), returned inline.
const video = response.media;
  • Gemini Omni replaces the Veo 3.1 preview models on the Gemini API, which shut down on October 22, 2026. Unlike Veo, it isn't a long-running operation: ai.generate() returns the video directly.
  • Images and videos in the prompt are used as inputs (image to video, first and last frame, subject references, editing or extending a video).
  • config.videoFormat sets aspectRatio (16:9, the default, or 9:16) and resolution (360p, 720p (the default), 1080p or 4k). delivery: 'uri' returns a Files API URI instead of inline data, which is recommended for large videos (a 10 s 4k video is about 50 MB); it requires store: true. Download the file with your API key (x-goog-api-key header). It can take a few seconds after the response to become ACTIVE, and it expires after 48 hours.
  • config.videoConfig.task (text_to_video, image_to_video, reference_to_video, edit, extend) tells the model which kind of generation to do; without it, the model infers it from the prompt.
  • To edit a generated video in a follow-up turn, set store: true and pass the previous response's messages: the plugin continues the stored interaction.
  • Tools aren't supported.

With vertexAI (Veo):

import { genkit } from 'genkit';
import { vertexAI } from '@genkit-ai/google-genai';

const ai = genkit({
  plugins: [vertexAI()],
});

let { operation } = await ai.generate({
  model: vertexAI.model('veo-3.1-lite-generate-001'),
  prompt: 'An origami butterfly flaps its wings and flies out of the french doors into the garden.',
});

// The generation is asynchronous and returns an operation.
// You must poll the operation until it completes.
while (!operation?.done) {
  await new Promise((resolve) => setTimeout(resolve, POLL_INTERVAL_MS));
  operation = await ai.checkOperation(operation!);
}

// Once complete, the generated video is available in `operation.output`.

Music Generation (Lyria)

With googleAI:

import { genkit } from 'genkit';
import { googleAI } from '@genkit-ai/google-genai';

const ai = genkit({
  plugins: [googleAI()],
});

const response = await ai.generate({
  model: googleAI.model('lyria-3-clip-preview'),
  prompt: 'A cheerful acoustic folk song with guitar and harmonica.',
});

const generatedAudio = response.media;

With vertexAI:

import { genkit } from 'genkit';
import { vertexAI } from '@genkit-ai/google-genai';

const ai = genkit({
  plugins: [vertexAI()],
});

const response = await ai.generate({
  model: vertexAI.model('lyria-3-clip-preview'),
  prompt: 'A cheerful acoustic folk song with guitar and harmonica.',
});

const generatedAudio = response.media;

Text-to-Speech

With googleAI:

import { genkit } from 'genkit';
import { googleAI } from '@genkit-ai/google-genai';

const ai = genkit({
  plugins: [googleAI()],
});

const response = await ai.generate({
  model: googleAI.model('gemini-3.8-flash-tts'),
  prompt: [
    {
      text: 'Have a wonderful day! <laugh>',
      // Delivery goes in metadata: the text itself is read verbatim.
      metadata: { speechMetadata: { style: 'cheerful and friendly' } },
    },
  ],
  config: {
    speechConfig: {
      voiceConfig: { prebuiltVoiceConfig: { voiceName: 'Kore' } },
    },
  },
});

// A complete WAV file (audio/wav), ready to play or save.
const audio = response.media;

Audio format and streaming (gemini-3.8-flash-tts, gemini-3.8-flash-lite-tts):

  • Without audioFormat, a non-streamed request returns one audio/wav file, and a streamed request returns raw PCM: audio/l16, 24 kHz, mono, 16-bit, with no header.
  • Set config.audioFormat.mimeType to choose the format: audio/wav, audio/l16, audio/mulaw or audio/alaw. audioFormat.sampleRate is also sent, but the models currently return 24 kHz regardless.
  • When streaming, audio chunks arrive as the speech is generated, so you can start playing before it's finished. The final response joins them into one media part in the requested format. Request audio/wav when streaming to get a playable WAV file at the end (the chunks themselves are still mostly raw PCM).
const { stream, response } = ai.generateStream({
  model: googleAI.model('gemini-3.8-flash-lite-tts'),
  prompt: 'Have a wonderful day!',
  config: {
    speechConfig: {
      voiceConfig: { prebuiltVoiceConfig: { voiceName: 'Kore' } },
    },
    audioFormat: { mimeType: 'audio/wav' },
  },
});

for await (const chunk of stream) {
  // chunk.media: the next piece of audio.
}

// One complete WAV file.
const audio = (await response).media;

Autonomous Agents (Deep Research & Antigravity)

Deep Research:

import { genkit } from 'genkit';
import { googleAI } from '@genkit-ai/google-genai';

const ai = genkit({ plugins: [googleAI()] });

let { operation } = await ai.generate({
  model: googleAI.model('deep-research-preview-04-2026'),
  prompt: 'Analyze global semiconductor market trends. Include graphics showing market share changes.',
});

// Deep Research runs asynchronously. Poll until complete.
while (!operation?.done) {
  await new Promise((resolve) => setTimeout(resolve, LONG_POLL_INTERVAL_MS));
  operation = await ai.checkOperation(operation!);
}

// Once complete, the research report is available in `operation.output`.

Antigravity:

import { genkit } from 'genkit';
import { googleAI } from '@genkit-ai/google-genai';

const ai = genkit({ plugins: [googleAI()] });

const response = await ai.generate({
  model: googleAI.model('antigravity-preview-09-2026'),
  prompt: 'Read Hacker News, summarize the top 10 stories.',
  // Required: the sandbox the agent works in.
  config: { environment: 'remote' },
});

On Gemini Enterprise Agent Platform (vertexAI): Deep Research is available as vertexAI.model('deep-research-preview-04-2026') and is used the same way. It runs on the global location (used when you don't set one). It uses the server's default for store unless you set it (in the request config or vertexAI({ store })); store: false is sent as is, and the agent rejects it.

Interactions API Architecture & response.raw

For newer Gemini models accessed through the googleAI() plugin, requests are routed through Google's modern Interactions API (/v1beta/interactions) rather than the legacy generateContent API.

Model Routing

Gemini and Gemma models use the Interactions API, including the -latest aliases, agents (Deep Research, Antigravity) and any model not listed below. Only these older models still use generateContent, until they are retired:

  • gemini-3.5-flash
  • gemini-3.1-flash-lite
  • gemini-3.1-pro-preview
  • gemini-3.1-pro-preview-customtools
  • gemini-3-flash-preview
  • gemini-2.5-pro
  • gemini-2.5-flash
  • gemini-2.5-flash-lite
  • gemini-3.1-flash-image
  • gemini-3-pro-image
  • gemini-2.5-flash-image
  • gemini-2.5-flash-preview-tts
  • gemini-2.5-pro-preview-tts
  • gemini-3.1-flash-tts-preview
  • gemma-4-26b-a4b-it
  • gemma-4-31b-it

Model Routing (vertexAI)

On Gemini Enterprise Agent Platform, newer Gemini models (for example gemini-4-argon, gemini-3.8-flash), Gemini Omni video models and any Gemini model not listed below use the Interactions API (.../interactions). These models still use generateContent:

  • gemini-3.8-flash-cyber
  • gemini-3.5-flash
  • gemini-3.1-flash-lite
  • gemini-3.1-pro-preview, gemini-3.1-pro-preview-customtools
  • gemini-3-flash-preview
  • gemini-2.5-pro, gemini-2.5-flash, gemini-2.5-flash-lite
  • image models (gemini-*-image*, gemini-nano-banana-*) and text-to-speech models (gemini-*-tts*): the platform's Interactions API doesn't support them
  • tuned model endpoints

Deep Research also uses the Interactions API.

The Interactions API is available on the global location (and on the us and eu multi-regions with CMEK), and in Express mode. If you don't set a location (no vertexAI({ location }), no GCLOUD_LOCATION, no config.location), these models use global. A location you set is used as is, and the server returns an error if the model isn't available there. Interactions are stateless by default (store: false), as with googleAI; set config: { store: true } (or vertexAI({ store: true }) for every request; a request's store wins) to store them and continue a conversation from the previous response's messages. On models that use generateContent, store: true is ignored with a warning. Unlike googleAI, safetySettings and labels are supported on these models.

In a stateless conversation (store: false), earlier results of server-side tools other than code execution (Google Search, URL context, Google Maps, file search, MCP servers) are not sent back with the history, because the platform's Interactions API rejects them. The model may call those tools again; set store: true to keep the full tool history on the server.

Standard Genkit Output vs. response.raw

For common use cases, the Genkit abstraction is completely unified: response.text, response.reasoning, response.message, streaming (for await (const chunk of stream)), and tool calling work identically regardless of which API backend is used.

However, if your code directly inspects the underlying provider payload via response.raw, be aware of the architectural difference:

  • generateContent payload: Returns a GenerateContentResponse with candidates[]. Grounding metadata and citations are located on candidates[0].groundingMetadata.
  • Interactions API payload: Returns a GeminiInteraction object with id, status, and an execution timeline array: steps[].
    • Tool Invocations: Intermediate server-side tool calls (such as Google Search and Code Execution) and tool results are always represented as explicit steps (google_search_call, code_execution_call, etc.).
    • Grounding Citations: Search/Maps grounding citations are located inside content annotations within the model_output step:
      import type { GeminiInteraction } from '@genkit-ai/google-genai';
      
      const raw = response.raw as GeminiInteraction;
      const annotations = raw.steps
        ?.flatMap((step) => (step.type === 'model_output' ? step.content : []))
        .flatMap((c) => (c.type === 'text' ? (c.annotations ?? []) : []));
      

Key Differences on the Interactions Path

  1. Safety Settings: Custom safety settings are not supported on the Interactions path. These models apply no additional safety filters by default (Gemini API safety settings); built-in protections against core harms always apply. BLOCK_NONE settings are ignored, since they can't block anything beyond that default. Blocking thresholds (BLOCK_ONLY_HIGH and stricter) throw an INVALID_ARGUMENT error, so a requested filter is never silently dropped.
  2. Sampling Settings: topK isn't sent to these models (the Interactions API has no top_k); it's dropped with a warning. temperature and topP are still sent, although newer models (Gemini 3.6 Flash, 3.5 Flash-Lite and later) ignore them.
  3. Server-Side Tool Invocations: The Interactions API always includes intermediate tool invocations in the execution steps, so toolConfig.includeServerSideToolInvocations is unnecessary.
  4. Stateless by Default (Explicit Store Opt-In): The upstream Interactions API defaults to storing conversations on Google's servers (store: true). To preserve the stateless, privacy-preserving behavior of Genkit's generateContent, the plugin explicitly defaults store: false. For Gemini models, server-side storage is an explicit opt-in: store: true must be specified via request config (config: { store: true }) or plugin configuration (googleAI({ store: true })); a request's store takes precedence, so config: { store: false } turns off a plugin-wide store: true for that request. When an interaction is stored, its ID is accessible on response.message?.metadata?.interactionId. In multi-turn calls, passing messages: previousResponse.messages with config: { store: true } will automatically extract the previous interaction ID from message metadata and continue the session statefully. If setting config: { previousInteractionId: "..." } manually, store: true is also required (omitting store or passing store: false will throw an INVALID_ARGUMENT error). On models that use generateContent, store only controls whether the server logs the request for your project (viewable in AI Studio's Logs and Datasets): it's sent as is when you set it (request config or googleAI({ store })), and otherwise your project's logging setting applies.
  5. Streaming: Text, media and thought summaries stream as they arrive. A server-side tool step (Google Search, code execution, URL context, ...) is streamed as one complete part when the step ends, the same part as in the final response. A function call is streamed as a complete toolRequest when the response ends. Grounding tool use is counted in response.usage.custom (e.g. googleSearchCount).
  6. Antigravity Requires Storage: The Antigravity agent doesn't work with store: false (the server rejects it), so unlike Gemini models it doesn't default to store: false; store is only sent when you set it. If you set store: false (in the request config or googleAI({ store: false })), it's passed through and the server returns an error; to use Antigravity with plugin-level store: false, set store: true in the request config. The agent's work (files, installed packages) lives in a sandbox environment that persists across turns, so passing messages: previousResponse.messages continues the stored interaction automatically, in the same environment, sending only the new messages. Function tools work the same way: a tool result is sent as a continuation of the interaction that asked for it.
  7. JSON Output: JSON output without a schema (output: { format: 'json' }, or config: { responseMimeType: 'application/json' }) is sent with an empty schema, so the model answers in JSON of any shape. Other responseMimeType values have no equivalent on the Interactions API and are dropped with a warning.
  8. Usage: response.usage has the token totals (inputTokens, outputTokens, thoughtsTokens, ...). Token counts per modality are in response.usage.custom (e.g. inputTextTokens, inputImageTokens, outputAudioTokens), and tool-use tokens in response.usage.custom.toolUseTokens (also on generateContent models). Character and media-part counts (inputCharacters, inputImages, ...) are counted from the request and response, the same way as on generateContent models (Deep Research, a background model, doesn't have them).

Key Differences

  • googleAI: Easier setup for smaller projects, great for prototyping with Google AI Studio. Uses API keys.
  • vertexAI: Enterprise-ready, integrates with Google Cloud IAM and other core enterprise security features. Offers a broader range of models and enterprise capabilities, fine-tuning, and more robust governance. Express Mode provides a low-friction entry point using an API key.

Choose the interface based on your project's scale, infrastructure, and feature requirements.

Feedback

The sources for this package are in the main Genkit repo. Please file issues and pull requests against that repo.

License

Apache-2.0

版本与发布记录

@genkit-ai/google-genai 共发布 61 个版本,首次发布于 2025/07/24;过去 12 个月发布了 40 个版本。

版本发布日期距今
1.44.0latest2026/11/0412 天前
1.43.02026/10/2225 天前
1.43.0-rc.3next2026/10/2225 天前
1.43.0-rc.22026/10/2126 天前
1.43.0-rc.12026/10/091 个月前
1.43.0-rc.02026/10/091 个月前

关于 @genkit-ai/google-genai 的常见问题

@genkit-ai/google-genai 是什么?

Genkit AI framework plugin for Google AI & Vertex APIs, including Gemini APIs.

如何安装 @genkit-ai/google-genai?

在项目目录运行 npm install @genkit-ai/google-genai,或使用 pnpm add / yarn add / bun add。它还需要宿主项目提供 peer 依赖:genkit(^1.44.0)。npm 7 及以上与 pnpm 8 及以上默认会自动安装必需的 peer 依赖,Yarn 需要手动安装。

@genkit-ai/google-genai 支持 TypeScript 吗?

支持。包内自带 TypeScript 类型声明(package.json 声明了 types 入口),安装后即可获得类型提示,无需再安装 @types 包。

@genkit-ai/google-genai 支持 ESM 还是 CommonJS?

同时提供 ES Module 与 CommonJS 入口,import 和 require() 都可以使用,打包工具会自动选择合适的版本。

@genkit-ai/google-genai 可以商用吗?

@genkit-ai/google-genai 使用 Apache-2.0 许可。宽松许可:允许商业使用、修改和再分发,并附带明确的专利授权;分发时需保留许可与 NOTICE 文件,修改过的文件要注明改动。以上为概括,不构成法律意见,请以许可原文为准。

@genkit-ai/google-genai 的下载量有多少?

根据 npm 官方下载统计,@genkit-ai/google-genai 在 2026/10/16 – 2026/11/14 这 30 天内被下载 674,414 次,日均约 22,480 次。下载量会把 CI 构建、镜像同步和作为其他包依赖的安装一并计入,适合横向比较热度,不能直接等同于使用人数。

@genkit-ai/google-genai 还在维护吗?

最新版本 1.44.0 发布于 2026/11/04(12 天前);至今共发布 61 个版本,其中过去 12 个月发布了 40 个;首次发布于 2025/07/24。发布频率只是参考信号:功能稳定的工具可能很少需要更新,建议结合源码仓库的 issue 和提交记录判断。