Learn how to use Google Provider.
The Google provider contains language and embedding model support for the Google APIs.
Setup
The Google provider is available in the @ai-sdk/google module. You can install it with
$ pnpm add @ai-sdk/google
$ npm install @ai-sdk/google
$ yarn add @ai-sdk/google
$ bun add @ai-sdk/google
Provider Instance
You can import the default provider instance google from @ai-sdk/google:
If you need a customized setup, you can import createGoogle from @ai-sdk/google and create a provider instance with your settings:
You can use the following optional settings to customize the Google provider instance:
-
batchResultDownloads object
Settings for downloading file-based JSON Lines batch results.
-
maxLineBytes number
Maximum UTF-8 bytes per row, excluding the LF delimiter. Defaults to 64 MiB (67,108,864 bytes). Must be a positive safe integer. Oversized rows throw a
DownloadErrorand cancel the download.
-
-
baseURL string
Use a different URL prefix for API calls, e.g. to use proxy servers. The default prefix is
https://generativelanguage.googleapis.com/v1beta. -
apiKey string
API key that is being sent using the
x-goog-api-keyheader. It defaults to theGOOGLE_GENERATIVE_AI_API_KEYenvironment variable. -
headers Record<string,string>
Custom headers to include in the requests.
-
fetch (input: RequestInfo, init?: RequestInit) => Promise<Response>
Custom fetch implementation. Defaults to the global
fetchfunction. You can use it as a middleware to intercept requests, or to provide a custom fetch implementation for e.g. testing. -
generateId () => string
Optional function to generate unique IDs for each request. Defaults to the SDK's built-in ID generator.
-
name string
Custom provider name. Defaults to
'google.generative-ai'.
Language Models
You can create models that call the Google Generative AI API using the provider instance.
The first argument is the model id, e.g. gemini-3.8-flash.
The models support tool calls and some have multi-modal capabilities.
You can use Google language models to generate text with the generateText function:
Google language models can also be used in the streamText function
and support structured data generation with Output
(see AI SDK Core).
To remain forward-compatible, the provider treats unrecognized gemini-*
model IDs and -latest aliases like the newest supported Gemini generation.
Currently, this means Gemini 3 request behavior for provider-defined and mixed
tools, thinkingLevel reasoning, multimodal function responses, and thought
signatures. Known legacy Gemini model IDs keep their generation-specific
request behavior. This SDK default controls request serialization; the
selected model must still support each feature in the Google API.
Google also supports some model specific settings that are not part of the standard call settings. You can pass them as an options argument:
The following optional provider options are available for Google models:
-
cachedContent string
Optional. The name of the cached content used as context to serve the prediction. Format: cachedContents/{cachedContent}
-
structuredOutputs boolean
Optional. Enable structured output. Default is true.
You can disable structured outputs when a schema is not supported or exceeds Google's schema complexity limits. The model will still be instructed to return JSON, but the schema will not be sent to Google.
See Troubleshooting: Schema Limitations for more details.
-
safetySettings Array<{ category: string; threshold: string }>
Optional. Safety settings for the model.
-
category string
The category of the safety setting. Can be one of the following:
HARM_CATEGORY_UNSPECIFIEDHARM_CATEGORY_HATE_SPEECHHARM_CATEGORY_DANGEROUS_CONTENTHARM_CATEGORY_HARASSMENTHARM_CATEGORY_SEXUALLY_EXPLICITHARM_CATEGORY_CIVIC_INTEGRITY
-
threshold string
The threshold of the safety setting. Can be one of the following:
HARM_BLOCK_THRESHOLD_UNSPECIFIEDBLOCK_LOW_AND_ABOVEBLOCK_MEDIUM_AND_ABOVEBLOCK_ONLY_HIGHBLOCK_NONEOFF
-
-
responseModalities string[] The modalities to use for the response. The following modalities are supported:
TEXT,IMAGE. When not defined or empty, the model defaults to returning only text. -
thinkingConfig { thinkingLevel?: 'minimal' | 'low' | 'medium' | 'high'; thinkingBudget?: number; includeThoughts?: boolean }
Optional. Configuration for the model's thinking process. Only supported by specific Google models.
-
thinkingLevel 'minimal' | 'low' | 'medium' | 'high'
Optional. Controls the thinking depth for Gemini 3 and later models. Gemini 3.1 Pro supports 'low', 'medium', and 'high', Gemini 3 Pro supports 'low' and 'high', while Gemini 3 Flash supports all four levels: 'minimal', 'low', 'medium', and 'high'.
-
thinkingBudget number
Optional. Gives the model guidance on the number of thinking tokens it can use when generating a response. Setting it to 0 disables thinking, if the model supports it. For more information about the possible value ranges for each model see Google thinking documentation.
This option is for Gemini 2.5 models. Gemini 3 and later models should use
thinkingLevelinstead. -
includeThoughts boolean
Optional. If set to true, thought summaries are returned, which are synthesized versions of the model's raw thoughts and offer insights into the model's internal reasoning process.
-
-
imageConfig { aspectRatio?: string, imageSize?: string }
Optional. Configuration for the models image generation. Only supported by specific Google models.
-
aspectRatio string
Model defaults to generate 1:1 squares, or to matching the output image size to that of your input image. Can be one of the following:
- 1:1
- 2:3
- 3:2
- 3:4
- 4:3
- 4:5
- 5:4
- 9:16
- 16:9
- 21:9
-
imageSize string
Controls the output image resolution. Defaults to 1K. Can be one of the following:
- 1K
- 2K
- 4K
-
-
audioTimestamp boolean
Optional. Enables timestamp understanding for audio-only files. See Google Cloud audio understanding documentation.
-
mediaResolution string
Optional. If specified, the media resolution specified will be used. Can be one of the following:
MEDIA_RESOLUTION_UNSPECIFIEDMEDIA_RESOLUTION_LOWMEDIA_RESOLUTION_MEDIUMMEDIA_RESOLUTION_HIGH
-
labels Record<string, string>
Optional. Defines labels used in billing reports. Available on Vertex AI only. See Google Cloud labels documentation.
-
serviceTier 'standard' | 'flex' | 'priority'
Optional. The service tier to use for the request. Set to
'flex'for 50% cheaper processing at the cost of increased latency. Set to'priority'for ultra-low latency at a 75-100% price premium over'standard'.Because Priority can be gracefully downgraded to Standard under load, the tier the request actually ran on is surfaced on
result.providerMetadata.google.serviceTier. See Priority inference and Flex inference. -
threshold string
Optional. Standalone threshold setting that can be used independently of
safetySettings. Uses the same values as thesafetySettingsthreshold.
Thinking
The Gemini 2.5 and Gemini 3-and-later series models use an internal "thinking process" that significantly improves their reasoning and multi-step planning abilities, making them highly effective for complex tasks such as coding, advanced mathematics, and data analysis. For more information see Google thinking documentation.
Gemini 3 and Later Models
For Gemini 3 and later models, use the thinkingLevel parameter to control the depth of reasoning:
Gemini 2.5 Models
For Gemini 2.5 models, use the thinkingBudget parameter to control the number of thinking tokens:
File Inputs
The Google provider supports file inputs, e.g. PDF files.
You can also use YouTube URLs directly:
The AI SDK will automatically download URLs if you pass them as data, except
for https://generativelanguage.googleapis.com/v1beta/files/ and YouTube
URLs. You can use the Google Generative AI Files API to upload larger files to
that location. YouTube URLs (public or unlisted videos) are supported directly
- you can specify one YouTube video URL per request.
See File Parts for details on how to use files in prompts.
Cached Content
Google supports both explicit and implicit caching to help reduce costs on repetitive content.
Implicit Caching
Gemini models automatically provide cache cost savings without needing to create an explicit cache. When you send requests that share common prefixes with previous requests, you'll receive a 75% token discount on cached content.
To maximize cache hits with implicit caching:
- Keep content at the beginning of requests consistent
- Add variable content (like user questions) at the end of prompts
- Ensure requests meet minimum token requirements:
- Gemini 3 series: 4096 tokens minimum
- Gemini 2.5 series: 2048 tokens minimum
Usage metadata was added to providerMetadata in @ai-sdk/google@1.2.23. If
you are using an older version, usage metadata is available in the raw HTTP
response body returned as part of the return value from generateText.
Explicit Caching
For guaranteed cost savings, you can still use explicit caching with Gemini 2.5 and 3 series models. See the models page to check if caching is supported for the used model:
Code Execution
With Code Execution, certain models can generate and execute Python code to perform calculations, solve problems, or provide more accurate information.
You can enable code execution by adding the code_execution tool to your request.
The response will contain the tool calls and results from the code execution.
Google Search
With Google Search grounding, the model has access to the latest information using Google Search.
The googleSearch tool accepts the following optional configuration options:
-
searchTypes object
Enables specific search types. Both can be combined.
webSearch: Enable web search grounding (pass{}to enable). This is the default.imageSearch: Enable image search grounding (pass{}to enable).
-
timeRangeFilter object
Restricts search results to a specific time range. Both
startTimeandendTimeare required.startTime: Start time in ISO 8601 format (e.g.'2025-01-01T00:00:00Z').endTime: End time in ISO 8601 format (e.g.'2025-12-31T23:59:59Z').
When Google Search grounding is enabled, the model will include sources in the response.
Additionally, the grounding metadata includes detailed information about how search results were used to ground the model's response. Here are the available fields:
-
webSearchQueries(string[] | null)- Array of search queries used to retrieve information
- Example:
["What's the weather in Chicago this weekend?"]
-
searchEntryPoint({ renderedContent: string } | null)- Contains the main search result content used as an entry point
- The
renderedContentfield contains the formatted content
-
groundingSupports(Array of support objects | null)- Contains details about how specific response parts are supported by search results
- Each support object includes:
segment: Information about the grounded text segmenttext: The actual text segmentstartIndex: Starting position in the responseendIndex: Ending position in the response
groundingChunkIndices: References to supporting search result chunksconfidenceScores: Confidence scores (0-1) for each supporting chunk
Example response:
Enterprise Web Search
With Enterprise Web Search, the model has access to a compliance-focused web index designed for highly-regulated industries such as finance, healthcare, and public sector.
Enterprise Web Search is only available on Vertex AI. You must use the Google
Vertex provider (@ai-sdk/google-vertex) instead of the standard Google
provider (@ai-sdk/google) to use this feature. Requires Gemini 2.0 or newer
models.
Enterprise Web Search provides the following benefits:
- Does not log customer data
- Supports VPC service controls
- Compliance-focused web index for regulated industries
File Search
The File Search tool lets Gemini retrieve context from your own documents that you have indexed in File Search stores. The provider enables this tool for Gemini 2.5 and Gemini 3-and-later model IDs, including unrecognized future Gemini IDs.
File Search responses include citations via the normal sources field and expose raw grounding metadata in providerMetadata.google.groundingMetadata.
Custom metadata attached to the indexed documents is available in
groundingMetadata.groundingChunks[].retrievedContext.customMetadata.
URL Context
Google provides a provider-defined URL context tool.
The URL context tool allows you to provide specific URLs that you want the model to analyze directly in from the prompt.
The URL context metadata includes detailed information about how the model used the URL context to generate the response. Here are the available fields:
urlMetadata({ retrievedUrl: string; urlRetrievalStatus: string; }[] | null)- Array of URL context metadata
- Each object includes:
retrievedUrl: The URL of the contexturlRetrievalStatus: The status of the URL retrieval
Example response:
With the URL context tool, you will also get the groundingMetadata.
The URL context tool is only supported for Gemini 2.0 Flash models and above. Check the supported models for URL context tool.
Combine URL Context with Search Grounding
You can combine the URL context tool with search grounding to provide the model with the latest information from the web.
Google Maps Grounding
With Google Maps grounding, the model has access to Google Maps data for location-aware responses. This enables providing local data and geospatial context, such as finding nearby restaurants.
The optional retrievalConfig.latLng provider option provides location context for queries about nearby places. This configuration applies to any grounding tools that support location context, including Google Maps and Google Search.
When Google Maps grounding is enabled, the model's response will include sources pointing to Google Maps URLs. The grounding metadata includes maps chunks with place information:
RAG Engine Grounding
With RAG Engine Grounding, the model has access to your custom knowledge base using the Vertex RAG Engine. This enables the model to provide answers based on your specific data sources and documents.
RAG Engine Grounding is only supported with Vertex Gemini models. You must use
the Google Vertex provider (@ai-sdk/google-vertex) instead of the standard
Google provider (@ai-sdk/google) to use this feature.
When RAG Engine Grounding is enabled, the model will include sources from your RAG corpus in the response.
Additionally, the grounding metadata includes detailed information about how RAG results were used to ground the model's response. Here are the available fields:
-
groundingChunks(Array of chunk objects | null)- Contains the retrieved context chunks from your RAG corpus
- Each chunk includes:
retrievedContext: Information about the retrieved contexturi: The URI or identifier of the source documenttitle: The title of the source document (optional)text: The actual text content of the chunk
-
groundingSupports(Array of support objects | null)- Contains details about how specific response parts are supported by RAG results
- Each support object includes:
segment: Information about the grounded text segmenttext: The actual text segmentstartIndex: Starting position in the responseendIndex: Ending position in the response
groundingChunkIndices: References to supporting RAG result chunksconfidenceScores: Confidence scores (0-1) for each supporting chunk
Example response:
Configuration Options
The vertexRagStore tool accepts the following configuration options:
-
ragCorpus(string, required)- The RagCorpus resource name in the format:
projects/{project}/locations/{location}/ragCorpora/{rag_corpus} - This identifies your specific RAG corpus to search against
- The RagCorpus resource name in the format:
-
topK(number, optional)- The number of top contexts to retrieve from your RAG corpus
- Defaults to the corpus configuration if not specified
Image Outputs
Gemini models with image generation capabilities (e.g. gemini-3.1-flash-image-preview) support generating images as part of a multimodal response. Images are exposed as files in the response.
If you primarily want to generate images without text output, you can also use
Gemini image models with the generateImage() function. See Gemini Image
Models for details.
Safety Ratings
The safety ratings provide insight into the safety of the model's response. See Google AI documentation on safety settings.
Example response excerpt:
Troubleshooting
Schema Limitations
The Google Generative AI API supports a subset of JSON Schema. Google notes that very large or deeply nested schemas may be rejected. When a schema exceeds Google's complexity limit, the API may return the following generic error:
Request contains an invalid argument.
You can work around this by simplifying the schema sent to Google. For example,
remove unnecessary constraints or reduce nesting while preserving the fields
and types the model needs to generate. If you need stricter client-side
validation than the schema sent to Google, use jsonSchema
with a custom validation function or validate the generated output separately.
By default, structured outputs are enabled (and for tool calling they are required). You can also disable provider-enforced structured outputs for object generation:
When structuredOutputs is false, Google still receives the JSON response MIME type
but not the schema. Describe the required schema in your prompt. AI SDK
will still validate the response against the schema in Output.object.
Model Capabilities
| Model | Image Input | Object Generation | Tool Usage | Tool Streaming | Google Search | URL Context |
|---|---|---|---|---|---|---|
gemini-nano-banana-2.1 | ||||||
gemini-3.8-flash | ||||||
gemini-3.7-flash | ||||||
gemini-3.6-flash | ||||||
gemini-3.5-flash | ||||||
gemini-3.5-flash-lite | ||||||
gemini-3.1-pro-preview | ||||||
gemini-3.1-flash-image-preview | ||||||
gemini-3.1-flash-lite-preview | ||||||
gemini-3-pro-preview | ||||||
gemini-3-pro-image-preview | ||||||
gemini-3-flash-preview | ||||||
gemini-2.5-pro | ||||||
gemini-2.5-flash | ||||||
gemini-2.5-flash-lite | ||||||
gemini-2.5-flash-lite-preview-06-17 | ||||||
gemini-2.0-flash |
The table above lists popular models. Please see the Google Generative AI docs for a full list of available models. The table above lists popular models. You can also pass any available provider model ID as a string if needed.
Batch
Batch support is experimental and the API may change in patch releases.
The Google provider supports asynchronous text generation through the Gemini Batch API. Pass the Google provider to the AI SDK's Batch API for the complete workflow, including polling, persistence, and result handling.
Each request specifies its type and model. Google requires every text
request in a batch to use the same model and throws before submission when the
models differ.
Webhooks
You can pass a webhookUrl to receive a notification when the batch reaches a terminal state:
Google sends a thin event to the webhook URL. After receiving and verifying the event, use the persisted batch reference with getBatchStatus or getBatchResults. Your application is responsible for handling the webhook and verifying Google's dynamic webhook signature.
Large Batches
When the serialized batch creation body is under 20 MB, the provider sends the requests inline. At 20 MB or more, it uploads a JSONL input file through the Gemini Files API.
Realtime Models
You can create models that call the Gemini Live API
using the .experimental_realtime() factory method.
Realtime sessions run in the browser and require a short-lived token created on
your server with google.experimental_realtime.getToken():
Google realtime models may require provider-specific audio formats, depending on the model and modality. See Realtime for the complete setup and tool calling pattern.
Gemini lifecycle signals that do not have provider-neutral equivalents are
emitted as custom events. Inspect rawType for goAway,
sessionResumptionUpdate, and generationComplete and use the event's raw
payload for provider-specific details.
With input audio transcription enabled, parseServerEvent emits
input-transcription-completed events with a stable itemId for each user
utterance. Each transcript contains the accumulated text so far. If you consume
these events directly, replace the text for that itemId with transcript on
each event. The realtime session handles this automatically.
Google's finished transcription marker ends an utterance even when the marker
contains no text. For payloads without this marker, response completion and
interruption provide fallback boundaries. These fallback boundaries cannot
unambiguously associate all delayed transcription fragments. Input item IDs use
their own sequence and need not match the model's response IDs.
Live Translation
Use gemini-3.5-live-translate-preview for low-latency speech-to-speech
translation. Configure the target language with
providerOptions.google.translationConfig in the realtime session config you
use when creating the token:
Set targetLanguageCode to a BCP-47 language code (defaults to en). When
echoTargetLanguage is true, input audio already in the target language is
echoed instead of producing silence.
Gemini Live Translation accepts audio input and produces translated audio output. Text input, tools, and custom instructions are not supported by this model.
Gemini 3.8 Live
gemini-3.8-live and gemini-3.8-live-extended-thinking are audio-only
realtime models: request outputModalities: ['audio'], and add
outputAudioTranscription: {} if you want the assistant's transcript as
audio-transcript-delta events. Server-side voice activity detection and
proactive audio are always on.
gemini-3.8-live-extended-thinking reasons in the background while streaming
audio. Google requires exactly one of thinkingLevel (low, medium, high)
or thinkingBudget for it; the provider sends thinkingLevel: 'low' when you
set neither. Override it through providerOptions.google.thinkingConfig:
Tools declared in the session config run asynchronously (NON_BLOCKING) by
default on these models, so the model keeps talking while a tool call is in
flight. Set providerOptions.google.defaultToolBehavior: 'BLOCKING' on
gemini-3.8-live to make it wait for the result instead.
On the extended thinking model, turnComplete (the response-done event) does
not mean the model is idle. Watch the interactionStatus custom event: its
raw.interactionStatus is IN_PROGRESS, IDLE, or WAITING_FOR_INPUT. A
waitingForInput custom event is emitted when the model has stopped and needs
user input to continue.
Translation Models
For server-side streaming translation, you can create models that call the
Gemini Live API using the
.translation() factory method. Translation models are streaming-only and are
used with
experimental_streamTranslate.
The first argument is the model id e.g. gemini-3.5-live-translate-preview.
The Gemini Live API auto-detects the source language (sourceLanguage is not
supported) and always outputs 24kHz 16-bit PCM audio (outputAudioFormat is
not supported). Unsupported settings surface as call warnings.
The following provider options are available:
- echoTargetLanguage boolean Whether input audio already in the target language should be echoed instead of producing silence. Optional.
Model Capabilities
| Model | Translated Audio | Translated Text | Source Transcript |
|---|---|---|---|
gemini-3.5-live-translate-preview |
Interactions API
The Gemini Interactions API
(POST /v1beta/interactions) is a separate Google endpoint with server-side
state, unified content blocks, first-class built-in tools, agent presets,
managed agents that run in a sandboxed Linux environment, and native
multimodal image and video output. It is reached via the
google.interactions(...) factory:
google.interactions(...) accepts a model ID string (e.g.
'gemini-3.8-flash', 'gemini-3.1-pro-preview'), { agent: <name> } to use
a Gemini agent preset, or { managedAgent: <name> } to
invoke a managed agent you created on Google's side.
The returned model can be passed to generateText and streamText like
any other AI SDK language model.
Use google(...) for the standard :generateContent /
:streamGenerateContent endpoints, and google.interactions(...) for the new
Interactions endpoint. Pick one per model instance — they target different
request bodies and SSE event vocabularies.
Provider Options
The Interactions model reads its options from the shared
providerOptions.google.* namespace. Validate them with the
GoogleLanguageModelInteractionsOptions type:
The following optional provider options are available:
-
previousInteractionId string
Server-side interaction id from a prior turn. When set, the server pulls prior context from its own state and only the new user message is sent on the wire. Pair with the default
store: trueto chain stateful conversations. See Stateful chaining. -
store boolean
Whether the server should persist the interaction. Defaults to
true. Set tofalsefor stateless multi-turn conversations where the full message history is re-sent on every turn. -
agent string
Name of a Gemini agent preset (e.g.
'deep-research-pro-preview-12-2025').Prefer the factory form
google.interactions({ agent: '...' })over settingagentin provider options — the factory is type-checked against the supported agent names. -
agentConfig object
Per-agent configuration. Currently supports
{ type: 'dynamic' }and{ type: 'deep-research', thinkingSummaries?, visualization?, collaborativePlanning? }. -
thinkingLevel 'minimal' | 'low' | 'medium' | 'high'
Controls reasoning depth for thinking-enabled models. Mapped onto the Interactions request's
thinking_level. -
thinkingSummaries 'auto' | 'none'
Whether the model returns synthesized thought summaries on reasoning parts. Defaults to the API default.
-
responseFormat Array<ResponseFormatEntry>
Output-format entries that map directly to the API's
response_formatarray. Entries have the following shape:Use this for fine-grained control over image, audio, video, or non-JSON text outputs. Video entries can control aspect ratio, resolution, duration, and inline or URI delivery; set
gcsUriwhen delivering to Google Cloud Storage. Inline videos are returned inresult.filesbygenerateTextand emitted asfileparts bystreamText. The AI SDK call-levelresponseFormat: { type: 'json', schema }still drives JSON-mode automatically and prepends a matching text entry; entries listed here are appended.For image entries,
aspectRatioaccepts1:1,2:3,3:2,3:4,4:3,4:5,5:4,9:16,16:9,21:9,1:8,8:1,1:4,4:1. -
imageConfig { aspectRatio?: string; imageSize?: '1K' | '2K' | '4K' | '512' } (deprecated)
Use responseFormat with a
{ type: 'image', ... }entry instead. Retained for backwards compatibility; the SDK translatesimageConfiginto a matchingresponse_formatimage entry and emits a warning when set. Ignored whenresponseFormatalready supplies an image entry. -
mediaResolution 'low' | 'medium' | 'high' | 'ultra_high'
Media resolution applied to image inputs / outputs.
-
serviceTier 'flex' | 'standard' | 'priority'
Service tier for the request. Mirrored back on
result.providerMetadata.google.serviceTierfor observability. -
systemInstruction string
Alternative to the AI SDK
systemmessage. If both are set, the AI SDKsystemmessage wins and a warning is emitted. -
background boolean
Run the interaction in the background. Required for agents whose server-side workflow cannot complete within a single request/response; rejected by agents that only support synchronous calls. When
true, the POST returns a non-terminal status and the SDK pollsGET /interactions/{id}until the work completes. -
environment string | object
Sandbox environment configuration for managed agents. Pass
'remote'to provision a fresh sandbox, anenvironment_idstring to reuse an existing one, or an object of the form{ type: 'remote', sources?, network? }to preload files and/or constrain outbound traffic. Only applies to agent calls. -
pollingTimeoutMs number
Maximum time, in milliseconds, to poll a background interaction before giving up. Defaults to 30 minutes (1,800,000 ms). Long-running agents may need longer.
Provider Metadata
result.providerMetadata.google (typed via GoogleInteractionsProviderMetadata)
exposes:
-
interactionId string
Server-side interaction id. Pass this back as
previousInteractionIdon the next turn to chain. -
serviceTier string
Service tier the request actually ran on.
-
signature string
Per-block signature hash, set by the SDK on output reasoning and tool-call or agentic video processing parts. Round-tripped automatically on the next turn.
Agentic video custom parts additionally expose:
-
processingId string
ID of an agentic video
processing_call. Present ongoogle.processing_callcustom parts. -
processingCallId string
ID of the matching agentic video processing call. Present on
google.processing_resultcustom parts.
Stateful chaining
With the default store: true, the server retains the prior turn so the
next request only needs to send the new user message and the
previousInteractionId:
For stateless multi-turn conversations, set store: false and re-send the
full message history on every turn (no previousInteractionId):
Agentic video understanding
Gemini 3.7 Flash, 3.6 Flash, and 3.5 Flash Lite support agentic video understanding through the Interactions API. Instead of sampling the entire video at a fixed frame rate, the model dynamically navigates the timeline and loads the transcripts, frames, or audio needed for the prompt.
Set processing: 'agentic' in the Google provider options of an individual
video file part. You can validate the per-file options with
GoogleInteractionsVideoOptions:
Agentic navigation is exposed as custom content parts:
google.processing_callmarks a request by the model to inspect part of the video. ItsproviderMetadata.google.processingIdcontains the processing ID.google.processing_resultmarks completion of that request. ItsproviderMetadata.google.processingCallIdlinks it to the processing call.
These parts are informational and do not require a tool response. They are
available from both generateText and streamText.
For static processing, use 'static' or provide clipping and frame-rate
options. startOffset and endOffset are measured in seconds:
With stateful conversations, pass the returned interactionId as
previousInteractionId; Gemini retains the video context server-side. With
store: false, append result.responseMessages and re-send the full history.
The processing custom parts and their signatures are then round-tripped as
native processing_call and processing_result steps so the video context is
preserved.
Built-in Tools
The Interactions API ships a built-in tool catalog. The provider-defined
tools under google.tools.* map onto Interactions tool descriptors:
| AI SDK tool | Interactions tool type | Notes |
|---|---|---|
google.tools.googleSearch | google_search | Web / image search grounding. |
google.tools.codeExecution | code_execution | Server-side Python execution. |
google.tools.urlContext | url_context | Fetch URLs referenced in the prompt. |
google.tools.fileSearch | file_search | Retrieval from File Search stores. |
google.tools.googleMaps | google_maps | Maps grounding for nearby-places queries. |
provider tool google.computer_use | computer_use | Computer use (browser environment). |
provider tool google.mcp_server | mcp_server | Remote MCP server passthrough. |
provider tool google.retrieval | retrieval | Vertex AI Search retrieval. |
Function tools (type: 'function') defined with the AI SDK tool(...)
helper are translated to Interactions function tool descriptors. Other
tool kinds emit a warning and are dropped.
Function tools work the same way as on the standard provider:
Image output via Interactions
Add a { type: 'image' } entry to responseFormat on a Gemini
image-capable model to get images as LanguageModelV4FilePart files in
the response. No tool wrapping is required, and the entry doubles as the
place to set aspectRatio, imageSize, and mimeType.
To control aspect ratio, image size, or output mime type, add those fields to the same image entry:
For multimodal output, list one entry per modality. The model returns
text in result.text and the accompanying image(s) in result.files:
Iterative image editing pairs naturally with stateful chaining — keep
previousInteractionId set across turns and the model edits its prior
output:
Agent presets
Pass { agent: <name> } to target a Gemini agent preset. The factory
type-checks the agent name against the supported set:
Whether an agent runs synchronously or in the background depends on the
agent. Long-running presets (such as the deep-research-* family)
require background: true — the POST returns a non-terminal status and
the SDK polls GET /interactions/{id} internally until the interaction
completes. Other agents accept synchronous calls only and will reject
background: true. Set the flag explicitly via
providerOptions.google.background.
The default polling timeout is 30 minutes; raise it via
pollingTimeoutMs for slower agents:
Agents also chain through previousInteractionId like model-id calls.
Managed Agents
Managed agents run inside a
sandboxed Linux environment provisioned per interaction. Pass the environment
provider option to control how the sandbox is set up; the option is only
accepted on agent calls.
The simplest form provisions a fresh sandbox:
environment accepts three shapes:
'remote'— provision a fresh sandbox for this call.- any other string — an
environment_idto reuse, forking the previous sandbox so its filesystem and installed packages persist. - an object — provision a fresh sandbox and optionally preload
sourcesand/or constrain outbound traffic vianetwork:
Three source types are supported: inline (write a string into the
sandbox at target), repository (clone a git repository — pass the
URL as source), and gcs (mount a Google Cloud Storage prefix).
The network field accepts the string 'disabled' to block all
outbound traffic, or an object with an allowlist array whose entries
each carry a domain plus an optional transform array of header
objects to inject into matching requests.
Custom managed agents
For user-defined agents that you created on Google's side via the
Gemini API's /v1beta/agents endpoint, pass the agent's name through the dedicated
managedAgent factory shape instead of agent (which only accepts
known preset names):
Streaming
streamText is supported. The stream's finish part exposes
interactionId on providerMetadata.google so callers can chain.
Gemma Models
You can use Gemma models with the Google Generative AI API. The following Gemma models are available:
gemma-3-27b-itgemma-3-12b-it
Gemma models don't natively support the systemInstruction parameter, but the provider automatically handles system instructions by prepending them to the first user message. This allows you to use system instructions with Gemma models seamlessly:
The system instruction is automatically formatted and included in the conversation, so Gemma models can follow the guidance without any additional configuration.
Embedding Models
You can create models that call the Google Generative AI embeddings API
using the .embedding() factory method.
The Google provider sends API calls to the right endpoint based on the type of embedding:
- Single embeddings: When embedding a single value with
embed(), the provider uses the single:embedContentendpoint, which typically has higher rate limits compared to the batch endpoint. - Batch embeddings: When embedding multiple values with
embedMany()or multiple values inembed(), the provider uses the:batchEmbedContentsendpoint.
Google embedding models support additional settings. You can pass them as an options argument:
When using embedMany, provide per-value multimodal content via the content option. Each entry corresponds to a value at the same index; use null for text-only entries:
The following optional provider options are available for Google embedding models:
-
outputDimensionality: number
Optional reduced dimension for the output embedding. If set, excessive values in the output embedding are truncated from the end.
-
taskType: string
Optional. Specifies the task type for generating embeddings. Supported task types include:
SEMANTIC_SIMILARITY: Optimized for text similarity.CLASSIFICATION: Optimized for text classification.CLUSTERING: Optimized for clustering texts based on similarity.RETRIEVAL_DOCUMENT: Optimized for document retrieval.RETRIEVAL_QUERY: Optimized for query-based retrieval.QUESTION_ANSWERING: Optimized for answering questions.FACT_VERIFICATION: Optimized for verifying factual information.CODE_RETRIEVAL_QUERY: Optimized for retrieving code blocks based on natural language queries.
-
content: array
Optional. Per-value multimodal content parts for embedding non-text content (images, video, PDF, audio). Each entry corresponds to the embedding value at the same index — its parts are merged with the text value in the request. Use
nullfor entries that are text-only. The array length must match the number of values being embedded. Each non-null entry is an array of parts, where each part can be{ text: string },{ inlineData: { mimeType: string, data: string } }for inline base64 data, or{ fileData: { fileUri: string, mimeType: string } }to reference remote content via HTTP URL or Google Cloud Storage URI (gs://...). Supported bygemini-embedding-2-preview.
Model Capabilities
| Model | Default Dimensions | Custom Dimensions | Multimodal |
|---|---|---|---|
gemini-embedding-001 | 3072 | ||
gemini-embedding-2 | 3072 | ||
gemini-embedding-2-preview | 3072 |
Image Models
You can create image models that call the Google Generative AI API using the .image() factory method.
For more on image generation with the AI SDK see generateImage().
The .image() factory supports Gemini multimodal language models with image
output capabilities through the :generateContent API.
Gemini Image Models
Gemini image models (e.g. gemini-3.1-flash-image-preview) are technically multimodal output language models, but they can be used with the generateImage() function for a simpler image generation experience. Internally, the provider calls the language model API with responseModalities: ['IMAGE'].
Gemini image models also support image editing by providing input images:
You can also use URLs for input images:
Gemini image models do not support the size or n parameters. Use
aspectRatio instead of size. Mask-based inpainting is also not supported.
For more advanced use cases where you need both text and image outputs, or
want more control over the generation process, you can use Gemini image models
directly with generateText(). See Image Outputs for
details.
Google Search Grounding
Gemini image models support Google Search grounding through providerOptions.google.googleSearch. The value matches the args of google.tools.googleSearch(...); pass {} to enable with defaults, or { searchTypes: { imageSearch: {} } } to ground on reference photos.
Gemini Image Model Capabilities
| Model | Image Generation | Image Editing | Aspect Ratios |
|---|---|---|---|
gemini-2.5-flash-image | 1:1, 2:3, 3:2, 3:4, 4:3, 4:5, 5:4, 9:16, 16:9, 21:9 | ||
gemini-3-pro-image-preview | 1:1, 2:3, 3:2, 3:4, 4:3, 4:5, 5:4, 9:16, 16:9, 21:9 | ||
gemini-3.1-flash-image-preview | 1:1, 2:3, 3:2, 3:4, 4:3, 4:5, 5:4, 9:16, 16:9, 21:9 | ||
gemini-nano-banana-2.1 | 1:1, 2:3, 3:2, 3:4, 4:3, 4:5, 5:4, 9:16, 16:9, 21:9, 1:8, 8:1, 1:4, 4:1 |
gemini-3-pro-image-preview supports additional features including up to 14
reference images for editing (6 objects, 5 humans), resolution options (1K,
2K, 4K via providerOptions.google.imageConfig.imageSize), and Google Search
grounding.
Speech Models
You can create models that call the Gemini text-to-speech API
using the .speech() factory method.
The first argument is the model ID, such as gemini-3.8-flash-tts or
gemini-3.8-flash-lite-tts. Speech generation uses the generateContent API.
The voice argument can be set to one of Gemini's 30 prebuilt voices
e.g. Kore, Puck, Zephyr, or Charon. Voice names are case-sensitive.
The default voice is Kore. Custom voice IDs and replication keys are not supported.
By default the generated audio is returned as a playable WAV file
(result.audio.mediaType is audio/wav). Gemini 3.8 returns a complete WAV file;
the provider passes those bytes through unchanged. Older models return raw PCM,
which the provider wraps in a WAV container. Write result.audio.uint8Array
directly to disk without adding another WAV header.
Gemini 3.8 supports the following outputFormat values:
| Format | Values | Result |
|---|---|---|
| WAV | wav, audio/wav | WAV with a RIFF header |
| PCM | pcm, audio/l16 | Headerless signed 16-bit PCM |
| Mu-law | mulaw, audio/mulaw | Headerless mu-law audio |
| A-law | alaw, audio/alaw | Headerless A-law audio |
These map to generationConfig.responseFormat.audio.mimeType in
generateContent. Older models support wav and pcm. The source MIME type and
sample rate are available in result.providerMetadata.google.
Gemini 2.5 and 3.1 model IDs retain the legacy request format. Other model IDs, including custom aliases, use the structured speech format.
Speech directions in Gemini 3.8
Gemini 3.8 treats text as a verbatim transcript. The provider sends instructions
as speechMetadata.style, leaving the text unchanged. You can also supply
providerOptions.google.speechMetadata with style and speaker fields.
An explicit style overrides instructions, including an empty style string.
Put sustained delivery directions such as whispering, speaking slowly, or being
out of breath in metadata. Keep momentary vocal events and pauses such as
<laugh>, <sigh>, <cough>, <breath>, and <short pause> in the transcript.
Avoid sound-effect tags such as applause or thuds. The provider preserves inline
tags as written.
For Gemini 3.1 and earlier, the provider retains the existing behavior of
prepending instructions to single-speaker text. It ignores instructions with a
warning for older multi-speaker requests. Structured turns and speechMetadata
require Gemini 3.8.
Multi-speaker audio
For Gemini 3.8 dialogue, pass multiSpeakerVoiceConfig and turns through
providerOptions.google. Every turn must include speechMetadata.speaker
matching a configured speaker; missing or unknown speakers produce an error
before the request is sent. Speaker labels belong in metadata, not the text.
The turns array replaces the top-level text, so pass text: ''. The combined
turn text must be non-empty. Passing
nonempty top-level text with turns produces a warning. Put metadata on each turn
instead of combining turns with top-level speechMetadata. Each turn's style
overrides instructions; turns without a style inherit instructions.
multiSpeakerVoiceConfig overrides the top-level voice. Each voiceConfig
uses prebuiltVoiceConfig: { voiceName: 'Kore' }. Google supports up to two
prebuilt voices in a multi-speaker request.
Older models continue to use labelled transcripts, such as
text: 'Joe: How are you? Jane: Doing great, thanks!', with
multiSpeakerVoiceConfig and without turns.
Gemini TTS models do not support the speed or language options; passing
them adds a warning to result.warnings. Language is detected automatically
from the input text.
Model Capabilities
| Model | Multi-speaker | Style via instructions |
|---|---|---|
gemini-2.5-flash-preview-tts | ||
gemini-2.5-pro-preview-tts | ||
gemini-3.1-flash-tts-preview | ||
gemini-3.8-flash-tts | ||
gemini-3.8-flash-lite-tts |
Decision Models
Create an experimental decision model with google.decisionModel(modelId).
It uses Gemini structured output for Choice, Score, and Boolean questions. Choose a model
that supports structured output, such as gemini-3.5-flash-lite.
The factory respects createGoogle settings and forwards providerOptions.google,
including thinking configuration. Thinking controls depend on the selected model;
use options supported by that model. The existing Gemini implementation sends the
answer schema as generationConfig.responseJsonSchema.
Choice labels are returned exactly, and Scores are finite fractional positions within the ordered rubric. The adapter validates complete answers after parsing and rejects safety-blocked, truncated, or invalid output. It preserves usage (including reasoning tokens), warnings, response data, and provider metadata.
Choice and Score answers do not include probability distributions. Boolean
answers contain prompted estimates of P(true), validated to be finite and in
[0, 1]. These estimates are not guaranteed to be calibrated. Apply thresholds
in application code, for example answers.requestsRefund.probability >= 0.5. See Decisions.
Decision models can also be accessed through customProvider aliases or
createProviderRegistry().decisionModel('provider:model'). Direct string IDs
use Gateway by default, or an explicitly configured default provider with an
decisionModel method. See
model aliases and registries.