AI Gateway
The AI Gateway provider connects you to models from multiple AI providers through a single interface. Instead of integrating with each provider separately, you can access OpenAI, Anthropic, Google, Meta, xAI, and other providers and their models.
Features
- Access models from multiple providers without having to install additional provider modules/dependencies
- Use the same code structure across different AI providers
- Switch between models and providers easily
- Automatic authentication when deployed on Vercel
- View pricing information across providers
- Observability for AI model usage through the Vercel dashboard
Setup
The Vercel AI Gateway provider is part of the AI SDK.
Basic Usage
For most use cases, you can use the AI Gateway directly with a model string:
The AI SDK automatically uses the AI Gateway when you pass a model string in the creator/model-name format.
Provider Instance
The gateway provider instance is available from the ai package in version
5.0.36 and later.
You can also import the default provider instance gateway from ai:
You may want to create a custom provider instance when you need to:
- Set custom configuration options (API key, Vercel access token, base URL, headers)
- Use the provider in a provider registry
- Wrap the provider with middleware
- Use different settings for different parts of your application
To create a custom provider instance, import createGateway from ai:
You can use the following optional settings to customize the AI Gateway provider instance:
-
batchResultDownloads object
Settings for downloading JSON Lines batch results.
-
maxLineBytes number
Maximum UTF-8 bytes per row, excluding the LF delimiter. Defaults to 64 MiB (67,108,864 bytes). Must be a positive safe integer. Oversized rows throw a
DownloadErrorand cancel the download.
-
-
baseURL string
Use a different URL prefix for API calls. The default prefix is
https://ai-gateway.vercel.sh/v4/ai. -
apiKey string | PromiseLike<string> | (() => string | PromiseLike<string>)
Pass a callback to resolve the API key before each request. The callback can return a string or a promise; token caching and renewal are handled by the caller. A callback or promise selects API key authentication and must resolve to a non-empty string; lookup failures do not switch authentication methods.
API key or Vercel access token that is being sent using the
Authorizationheader. It defaults to theAI_GATEWAY_API_KEYenvironment variable. Supports AI Gateway API keys, Vercel personal access tokens, and Vercel app access tokens. -
teamIdOrSlug string
Vercel team ID or slug used to scope model requests when authenticating with multi-team Vercel access tokens.
-
headers Record<string,string>
Custom headers to include in the requests.
-
fetch (input: RequestInfo, init?: RequestInit) => Promise<Response>
Custom fetch implementation. Defaults to the global
fetchfunction. You can use it as a middleware to intercept requests, or to provide a custom fetch implementation for e.g. testing. -
metadataCacheRefreshMillis number
How frequently to refresh the metadata cache in milliseconds. Defaults to 5 minutes (300,000ms).
Authentication
The Gateway provider supports the following authentication methods:
API Key Authentication
Set your API key via environment variable:
Or pass it directly to the provider:
Vercel Access Token Authentication for Model Requests
For dynamic runtime authentication of model requests, pass a Vercel personal access token or Vercel app access token to a custom Gateway provider instance:
apiKey takes precedence over AI_GATEWAY_API_KEY and OIDC.
Vercel access tokens with teamIdOrSlug are supported for model requests.
Gateway helper methods such as getCredits, getSpendReport, and
getGenerationInfo require an AI Gateway API key or OIDC authentication.
OIDC Authentication (Vercel Deployments)
When deployed to Vercel, the AI Gateway provider supports authenticating model requests using OIDC (OpenID Connect) tokens without API Keys.
How OIDC Authentication Works
-
In Production/Preview Deployments:
- OIDC authentication is automatically handled
- No manual configuration needed
- Tokens are automatically obtained and refreshed
-
In Local Development:
- First, install and authenticate with the Vercel CLI
- Run
vercel env pullto download your project's OIDC token locally - For automatic token management:
- Use
vercel devto start your development server - this will handle token refreshing automatically
- Use
- For manual token management:
- If not using
vercel dev, note that OIDC tokens expire after 12 hours - You'll need to run
vercel env pullagain to refresh the token before it expires
- If not using
If an API key or Vercel access token is present (either passed directly or via environment), it will always be used instead of OIDC, even if invalid.
Read more about using OIDC tokens in the Vercel AI Gateway docs.
Bring Your Own Key (BYOK)
You can connect your own provider credentials to use with Vercel AI Gateway. This lets you use your existing provider accounts and access private resources.
To set up BYOK, add your provider credentials in your Vercel team's AI Gateway settings. Once configured, AI Gateway automatically uses your credentials. No code changes are needed.
For providers like Azure where you can use custom deployment names, you can configure model mappings to map gateway model slugs to your deployment names. See model mappings for details.
Learn more in the BYOK documentation.
Language Models
You can create language models using a provider instance. The first argument is the model ID in the format creator/model-name:
AI Gateway language models can also be used in the streamText function and support structured data generation with Output (see AI SDK Core).
When an upstream provider reports a well-formed error after streaming has
started, AI SDK Core exposes it as a
StreamProviderError.
Use its type, code, statusCode, and isRetryable metadata to classify
the failure without inspecting the Gateway's serialized error payload.
Image Models
AI Gateway supports image generation through generateImage. To reduce peak
memory when receiving large images, opt in to streaming base64 decoding:
The option defaults to false, preserving base64 strings in provider results.
When enabled, canonical base64 images of at least 64 KiB are decoded while the
response arrives. The model returns these images as Uint8Array; if a response
mixes extracted bytes with smaller images, all images in that result become
bytes. Images that cannot be extracted retain the regular decoding path.
This experimental option changes what image-model middleware sees in
result.images. GeneratedFile still provides both uint8Array and base64,
but reading base64 can require re-encoding. Validation errors contain short
placeholders for extracted data. The option is handled by the client and is
not sent to the Gateway service.
Batch
Batch support is experimental and the API may change in patch releases.
AI Gateway supports durable text batches through experimental_startBatch,
experimental_getBatchStatus, experimental_getBatchResults, and experimental_cancelBatch. See
Batch for the provider-agnostic workflow,
and the AI Gateway batch processing guide
for supported models, limits, and Gateway-specific behavior.
Pass a publicly reachable HTTPS webhookUrl when starting a batch to receive a terminal notification instead of polling. AI Gateway sends batch.completed, batch.failed, or batch.cancelled. The event contains terminal status information but not batch results. Completion webhooks are an AI Gateway capability; direct Anthropic and OpenAI batch providers return an unsupported warning when this option is provided.
The signing secret is returned only in the start response. Reserve the callback token before submitting the batch, then persist the secret and minimal batch reference before the process exits. The stable providerOptions.gateway.idempotencyKey makes retrying an ambiguous start safe. Reuse the reserved token when retrying; do not generate a new one. Delete the reservation after a definitive start failure. While the reservation is still pending, the receiver should return a retryable response so an early delivery is sent again after setup completes.
The receiver must verify the x-ai-gateway-signature header against a bounded raw request body before trusting the event:
The signature header has the form t=<unix seconds>,v1=<hex digest>, where v1 is the HMAC-SHA256 digest of "<t>.<raw body>". Verify a size-limited raw request body rather than a re-serialized object, use a timing-safe comparison, reject stale timestamps, and reject events whose data.jobId does not match the persisted batch.id.
AI Gateway expects a 2xx response within 10 seconds and retries failed deliveries. It does not follow redirects, so the callback endpoint itself must return the 2xx response. Retries carry the same x-ai-gateway-idempotency-key value (<jobId>-<status>), which receivers can use for deduplication. After verification, pass the persisted batch reference to experimental_getBatchStatus and experimental_getBatchResults. Delivery is best-effort, so periodically reconcile persisted batches by status as a fallback for a notification that exhausts its retries.
Each request specifies its type and model. AI Gateway currently requires
every text request in a batch to use the same model and throws before submission
when the models differ.
Cancelling a batch
Pass the batch reference returned by experimental_startBatch to request cancellation:
Cancellation is asynchronous. Continue checking experimental_getBatchStatus until the batch is terminal, then read experimental_getBatchResults for any partial results. A batch may finish before cancellation takes effect, and successful requests remain billable. Cancellation returns acknowledgement metadata, not confirmation that provider execution has stopped.
Reranking Models
You can create reranking models using the rerankingModel method on the provider instance:
Reranking models are useful for improving search results in retrieval-augmented generation (RAG) pipelines by re-scoring candidate documents after an initial retrieval step.
Streaming Speech Models
Streaming speech is experimental and the API may change in patch releases.
Speech models that support streaming can be used with
experimental_streamSpeech. The gateway provider connects to AI Gateway over a
WebSocket, forwards text deltas as they arrive, and streams audio chunks back.
Gateway WebSocket auth is carried in the subprotocols, so no header-capable
WebSocket implementation is needed.
Streaming speech works only with models that the AI Gateway model catalog lists with streaming speech support. The model ID below is illustrative; check the catalog for current availability.
Streaming Speech in the Browser
To stream speech from browser code without exposing your Gateway credential,
mint a short-lived client secret on your server with
gateway.experimental_speech.getToken() and pass the token to the browser.
The token is bound to the streaming speech surface and the requested model.
Realtime Models
For a trusted server-owned connection, getServerWebSocketConfig() reuses Gateway HTTP authentication, protocol, team and custom headers. Only the conversation intent is supported. The identity codec passes through normalized usage only when the Gateway route includes it in its events; credentials in connection settings remain on your server.
Realtime support is experimental and the API may change in patch releases.
You can create realtime models for bidirectional audio/text WebSocket sessions using the experimental_realtime method on the provider instance. The first argument is the model ID in the format creator/model-name:
The Gateway normalizes realtime the same way it normalizes every other modality: your client speaks the normalized AI SDK realtime protocol and the Gateway translates to and from the upstream provider server-side. This means the same client code works regardless of which provider backs the model.
Realtime sessions run in the browser and require a short-lived Gateway client secret created on your server with gateway.experimental_realtime.getToken(). The method uses your Gateway credential (apiKey, AI_GATEWAY_API_KEY, or Vercel OIDC token) to mint a vcst_ client secret and returns the WebSocket URL for the model.
Use the matching Gateway realtime model in the browser. Creating the model is
safe in browser code; only getToken() is server-side.
Do not expose your Gateway API key or OIDC token to browser clients. Only return
the short-lived setup response from getToken().
The Gateway WebSocket route transports the short-lived auth token via the
versioned Sec-WebSocket-Protocol subprotocol and the model id via the
?ai-model-id= query - the WebSocket transport of the Authorization and
ai-model-id headers the HTTP routes use. Subprotocol values must fit the
WebSocket token grammar, and the complete Sec-WebSocket-Protocol header
should stay compact (under an 8 KiB safe header budget).
Provider Options
Gateway provider options — tags, user, byok, compliance flags, and so on — are set under providerOptions.gateway in the session configuration, mirroring how they ride the request body on the non-realtime routes. Include them in the session configuration you send to the Gateway, and the Gateway applies them server-side:
The GatewayProviderOptions type is exported from @ai-sdk/gateway for stable client-facing fields. The type also accepts service-owned option keys so the Gateway service can add or change server-side options without requiring an SDK release for every schema change. Runtime validation is owned by the Gateway service.
Provider options are sent in the initial session update after the socket
opens, so connect-time options such as byok and quota selection require a
Gateway that resolves them from that update. Routing knobs like order /
only do not apply to realtime, where the Gateway selects the
WebSocket-capable provider.
Available Models
The AI Gateway supports models from OpenAI, Anthropic, Google, Meta, xAI, Mistral, DeepSeek, Amazon Bedrock, Cohere, Perplexity, Alibaba, and other providers.
For the complete list of available models, see the AI Gateway documentation.
Dynamic Model Discovery
You can discover available models programmatically:
Credit Usage
You can check your team's current credit balance and usage:
The getCredits() method returns your team's credit information based on the authenticated API key or OIDC token:
- balance number - Your team's current available credit balance
- total_used number - Total credits consumed by your team
Generation Lookup
Look up detailed information about a specific generation by its ID, including cost, token usage, latency, and provider details. Generation IDs are available in providerMetadata.gateway.generationId on both generateText and streamText responses.
When streaming, the generation ID is injected on the first content chunk, so you can capture it early in the stream without waiting for completion. This is especially useful in cases where a network interruption or mid-stream error could prevent you from receiving the final response — since the gateway records the final status server-side, you can use the generation ID to look up the results (including cost, token usage, and finish reason) later via getGenerationInfo().
With streamText, you can capture the generation ID from the first chunk via stream:
You can also capture the generation ID for every completed model call with
onLanguageModelCallEnd. The callback runs before client-side tools execute,
so the metadata remains available even if the operation is later aborted while
a tool is running:
The getGenerationInfo() method accepts:
- id string - The generation ID to look up (format:
gen_<ulid>, required)
It returns a GatewayGenerationInfo object with the following fields:
- id string - The generation ID
- totalCost number - Total cost in USD
- marketCost number (optional) - Market cost in USD, excluding surcharges
- surchargeCost number (optional) - Total surcharges applied in USD
- gatewayCost number (optional) - Total amount debited in USD
- upstreamInferenceCost number - Upstream inference cost in USD (relevant for BYOK)
- usage number - Usage cost in USD (same as totalCost)
- createdAt string - ISO 8601 timestamp when the generation was created
- model string - Model identifier used
- isByok boolean - Whether Bring Your Own Key credentials were used
- providerName string - The provider that served this generation
- streamed boolean - Whether streaming was used
- finishReason string - Finish reason (e.g.
'stop') - latency number - Time to first token in milliseconds
- generationTime number - Total generation time in milliseconds
- promptTokens number - Number of prompt tokens
- completionTokens number - Number of completion tokens
- reasoningTokens number - Reasoning tokens used (if applicable)
- cachedTokens number - Cached tokens used (if applicable)
- cacheCreationTokens number - Cache creation input tokens
- billableWebSearchCalls number - Number of billable web search calls
The optional cost breakdown fields contain the values reported by AI Gateway.
Fields that are missing or null in the API response are omitted from the result,
so unavailable costs remain distinct from a reported cost of zero. The SDK does
not calculate these fields from other costs.
Examples
Basic Text Generation
Streaming
Tool Usage
Provider-Executed Tools
Some providers offer tools that are executed by the provider itself, such as OpenAI's web search tool. To use these tools through AI Gateway, import the provider to access the tool definitions:
Some provider-executed tools require account-specific configuration (such as Claude Agent Skills) and may not work through AI Gateway. To use these tools, you must bring your own key (BYOK) directly to the provider.
Gateway Tools
The AI Gateway provider includes built-in tools that are executed by the gateway itself. These tools can be used with any model through the gateway.
Perplexity Search
The Perplexity Search tool enables models to search the web using Perplexity's search API. This tool is executed by the AI Gateway and returns web search results that the model can use to provide up-to-date information.
You can also configure the search with optional parameters:
The Perplexity Search tool supports the following optional configuration options:
-
maxResults number
The maximum number of search results to return (1-20, default: 10).
-
maxTokensPerPage number
The maximum number of tokens to extract per search result page (256-2048, default: 2048).
-
maxTokens number
The maximum total tokens across all search results (default: 25000, max: 1000000).
-
searchLanguageFilter string[]
Filter search results by language using ISO 639-1 language codes (e.g.,
['en']for English,['en', 'es']for English and Spanish). -
country string
Filter search results by country using ISO 3166-1 alpha-2 country codes (e.g.,
'US'for United States,'GB'for United Kingdom). -
searchDomainFilter string[]
Limit search results to specific domains (e.g.,
['reuters.com', 'bbc.com']). This is useful for restricting results to trusted sources. -
searchRecencyFilter 'day' | 'week' | 'month' | 'year'
Filter search results by relative time period. Useful for always getting recent results (e.g., 'week' for results from the last week).
The tool works with both generateText and streamText:
Exa Search
The Exa Search tool enables models to search the web using Exa's Search API. This tool is executed by the AI Gateway and returns token-efficient web excerpts for agent workflows.
You can also configure the search with optional parameters:
The Exa Search tool supports the following optional configuration options:
-
type 'auto' | 'fast' | 'instant'
Search mode.
'auto'is the default balance of speed and quality. -
numResults number
Maximum number of results to return (1-100, default: 10).
-
category 'company' | 'people' | 'research paper' | 'news' | 'personal site' | 'financial report'
Focus results on a specific content type.
-
includeDomains / excludeDomains string[]
Restrict or exclude search results by domain.
-
startPublishedDate / endPublishedDate string
Filter results by ISO 8601 publication date.
-
contents object
Control result extraction and freshness:
text- Return full page text, optionally capped withmaxCharactershighlights- Return token-efficient excerptsmaxAgeHours- Control cached content freshnesslivecrawlTimeout- Livecrawl timeout in millisecondssubpages/subpageTarget- Crawl related subpagesextras.links/extras.imageLinks- Extract links from pages
The tool works with both generateText and streamText.
This initial Gateway integration supports Exa's plain Search modes and token-efficient content controls. Deep synthesis modes and generated summaries are intentionally not exposed yet because they have separate pricing from standard Search.
Browserbase Search
The Browserbase Search tool enables models to search the web using Browserbase's Search API. It is executed by AI Gateway and returns fast, structured search results without opening a browser session.
You can configure the maximum number of results returned by each search:
The Browserbase Search tool supports this optional configuration option:
-
numResults number
Maximum number of search results to return (1-25, default: 10).
The tool works with both generateText and streamText.
Browserbase Fetch
The Browserbase Fetch tool enables models to retrieve page content using Browserbase's Fetch API. It is a lightweight option for pages that do not require JavaScript execution or browser interaction, and it supports raw, Markdown, and schema-driven JSON output.
You can configure the output format and request behavior. For example, use JSON extraction to return content matching a JSON Schema:
The Browserbase Fetch tool supports these optional configuration options:
-
format 'raw' | 'markdown' | 'json'
Output format for the fetched content.
rawreturns the upstream response body unchanged and is the default.markdownconverts the page to Markdown.jsonperforms structured extraction and requiresschema. -
schema object
JSON Schema describing the desired structured content. Only used when
formatisjson. -
allowRedirects boolean
Follow HTTP redirects. Defaults to
false. -
proxies boolean
Route the request through Browserbase's proxy network. Defaults to
false. -
allowInsecureSsl boolean
Bypass TLS certificate verification. Defaults to
false; only enable it for trusted hosts.
The Fetch API does not execute JavaScript. Use a browser session for interactive or JavaScript-rendered pages. The tool works with both generateText and streamText.
Tako Search
The Tako Search tool enables models to search the web and Tako's curated knowledge graph in a single call using Tako's Search API. This tool is executed by the AI Gateway and returns token-efficient web excerpts, plus knowledge-graph results that carry structured data, premium source attribution, and an embed-ready visualization.
Configure the search with effort, source selection, and localization defaults. These defaults override values the model includes in a tool call. variableCost, maxVariableCostUsd, and extraBody are developer-only: the model can't see or set them.
The Tako Search tool supports these optional configuration options:
- effort 'fast' | 'instant' | 'deep' - Search effort.
'fast'is the balanced default,'instant'favors cached low-latency results, and'deep'broadens retrieval with reranking at higher cost and latency. - sources object - Omit to search both curated Tako data and the live web. When set, only source keys present are searched.
- sources.web object - Configure web results:
- count number - Maximum web results, 1-20.
- category 'finance' | 'news' | 'sports' - Restrict web results to one category.
- includeDomains / excludeDomains string[] - Only return, or drop, results from these bare domains. Up to 20 each.
- publishedAfter / publishedBefore string - Keep results published on or after, or on or before, this
YYYY-MM-DDdate. - snippetMaxChars number - Character cap on each result's text excerpt. Defaults to 4,000; maximum 20,000.
- highlights boolean - Return query-relevant passages as each result's snippet instead of the opening text of the page. The snippet can hold multiple passages joined by
' ... ', and a page with no highlight returnssnippet: null. Defaults totruein the AI Gateway. - includeContents boolean - Inline each page's extracted full text in
content. articleContentMaxChars caps it, defaulting to 30,000 characters (maximum 1,000,000).
- sources.data object - Configure data results:
- count number - Maximum data results, 1-20. Defaults to 5.
- includeContents boolean - Inline each card's underlying rows in
content.datasetas typed, unit-labeled columns. This adds a data surcharge based on the row count and dataset source. To estimate cost, first search withincludeContents: falseand inspectcards.content.export_pricing. Because this applies to every returned card, usecountandmaxRowsto control cost. - contentFormat 'json_compact' | 'json_records' | 'csv' | 'card_json' - Serialization for inlined card data. Defaults to
'json_compact'. - maxRows number - Row cap per result. Omit to use the allowance in
cards.content.export_pricing. A data surcharge applies per 1,000 exported rows; lower values reduce cost. - nodeIds string[] - Data Graph node IDs to prioritize. Up to 20.
- strict boolean - Only return cards matching
nodeIds. Requires at least onenodeIdsvalue.
- location object - End-user
{ latitude, longitude }coordinates for localized results. - countryCode string - ISO 3166-1 alpha-2 country code, such as
'US'. - locale string - BCP-47 locale, such as
'en-US'. - timezone string - IANA timezone, such as
'America/New_York'. - outputSettings object - Card rendering controls.
forceRefreshis available only with'instant'effort. - includeRelated number - Number of related search suggestions to include (1-20).
- variableCost boolean - Serve data from sources priced per unit, such as web traffic and SEO. Each such card adds a charge, and the total per call is capped by
maxVariableCostUsd. Defaults totrue. Set it tofalseto opt out; those cards then come back as placeholders. Only searches that include the data index (sources.dataset, orsourcesomitted) return these cards. - maxVariableCostUsd number | null - The most one search call can be charged in variable cost, in USD.
- Defaults to
0.25, andnullremoves the cap. - Tako checks each card's maximum charge (
content.export_pricing.variable_cost_max_usd) in rank order. A card that doesn't fit comes back as a placeholder withreason: 'over_cap', even if its actual charge would have fit. 0returns only the cards whose charge the base price already covers.- It can't be combined with
variableCost: false.
- Defaults to
- extraBody Record<string, unknown> - Extra Tako request fields, in snake_case, deep-merged into the request body last. Objects merge, and scalars and arrays replace. Use it for Tako parameters this tool doesn't model yet. Its limits:
- It can't set a field the tool already models, including the
sources.datavariable-cost fields. The gateway answersinvalid_input. - Tako rejects unknown keys, so a typo comes back as an
api_error. - It's limited to 16,384 serialized characters.
- It can't set a field the tool already models, including the
When Tako withholds a card's data, it returns a placeholder card instead. A placeholder keeps its title and a description but carries no data. placeholder.withheld_source names the source, and placeholder.reason is 'not_served' (variable cost is off, or the account isn't served that source) or 'over_cap'. usage.variable_cost.cost_usd reports the variable-cost charge for the call:
For precise entity matching, quote a phrase in the query and optionally append a label: "Tesla":PRODUCT price. Tako treats the quoted phrase as one entity before searching.
The tool works with both generateText and streamText:
Parallel Search
The Parallel Search tool enables models to search the web using Parallel AI's Search API. The model provides a research objective and a few keyword queries, and the tool returns relevant excerpts from web pages, optimized for LLM consumption.
You can also configure the search with optional parameters:
The Parallel Search tool supports the following optional configuration options:
-
mode 'turbo' | 'fast' | 'basic' | 'advanced'
Search mode. Defaults to
'basic'. See the AI Gateway web search docs for pricing.'turbo'- Lowest latency, shorter excerpts. Supports English and Japanese only.'fast'- Low latency, shorter excerpts.'basic'- Balanced latency and excerpt depth (default).'advanced'- Most thorough retrieval, higher latency.
'one-shot'and'agentic'are deprecated but still accepted:'one-shot'runs as'basic'and'agentic'runs as'advanced'. -
maxResults number
Maximum number of results to return (1-20). Defaults to 10 if not specified.
-
sourcePolicy object
Source policy for controlling which domains to include/exclude:
includeDomains- Domains or domain/path prefixes to include, such as'wikipedia.org'or'docs.python.org/3'. When set,excludeDomainsis ignored.excludeDomains- Domains or domain/path prefixes to excludeafterDate- Only include results published on or after this date (YYYY-MM-DD)
The include and exclude lists can hold up to 200 entries combined.
-
maxCharsTotal number
Maximum total characters across all result excerpts.
-
excerpts object
Excerpt configuration for controlling result length:
maxCharsPerResult- Maximum characters per resultmaxCharsTotal- Deprecated; use the top-levelmaxCharsTotal
-
fetchPolicy object
Fetch policy for controlling content freshness:
maxAgeSeconds- Maximum age of cached content, in seconds, before a live fetch is attempted. Minimum 600; lower values are raised to 600.timeoutSeconds- Timeout for a live fetch, in seconds. The gateway caps it to stay within its own tool timeout.disableCacheFallback- Return an error instead of older cached content when a live fetch fails. Defaults tofalse.
-
location string
ISO 3166-1 alpha-2 country code to localize results, such as
'us'. Invalid codes are ignored with a warning.
Each tool result contains a search_id and results, ordered by decreasing relevance, and may include a session_id. Each result has a url, a title, a publish_date (both may be null), and excerpts (an array of markdown strings). The result can also include warnings, for example when an invalid location is ignored.
The tool works with both generateText and streamText:
DeepSeek Search and Fetch
Use gateway.tools.deepseekSearch() to search the web and
gateway.tools.deepseekFetch() to retrieve a page as Markdown. Both tools call
DeepSeek's standalone Search/Fetch APIs from AI Gateway and work with supported
tool-calling models from any provider. They are separate from a model's built-in
web search. No additional model provider package is required.
The Search tool supports the following optional configuration options:
-
query string
Pin the search query; otherwise the model chooses it. Queries contain 1–128 Unicode characters and cannot be all whitespace.
-
count number
Number of results, 1–50 (default: 10).
-
needContent boolean
Include available page content (default: false).
-
oriQuestion string | null
Original question, 1–128 Unicode characters.
nullmeans not provided. -
localization string
IP address used to localize results; developer configuration only.
The model-facing input uses query, count, need_content, and ori_question.
Queries support search operators including site:, -site:, lang:, after:,
before:, and quoted phrases. Search returns { results }, containing ranked
URLs, titles, snippets, summaries, optional page content, and nullable site/date
metadata. An empty results array is a successful search with no matches.
Fetch accepts an HTTP(S) url up to 8,192 characters, without a username or
password, and returns { content, title? }. The content is Markdown. You can
pin the URL with gateway.tools.deepseekFetch({ url: 'https://example.com' }).
The Fetch tool supports the following optional configuration option:
-
url string
Pin the HTTP(S) URL; otherwise the model chooses it.
Tool failures appear as tool-error parts in result.content (for
generateText) or result.stream (for streamText). Inspect part.error
there; failures are not included in result.toolResults.
The tools also work with streamText:
See the DeepSeek API guide for the underlying APIs.
Custom Reporting
Track usage per end-user and categorize requests with tags, then query the data through the reporting API.
Usage Tracking with User and Tags
This allows you to:
- View usage and costs broken down by end-user in your analytics
- Filter and analyze spending by feature or use case using tags
- Track which users or features are driving the most AI usage
Querying Spend Reports
Use the getSpendReport() method to query usage data programmatically. The reporting API is only available for Vercel Pro and Enterprise plans. For pricing, see the Custom Reporting docs.
The getSpendReport() method accepts the following parameters:
- startDate string - Start date in
YYYY-MM-DDformat (inclusive, required) - endDate string - End date in
YYYY-MM-DDformat (inclusive, required) - groupBy string - Aggregation dimension:
'day'(default),'user','model','tag','provider', or'credential_type' - datePart string - Time granularity when
groupByis'day':'day'or'hour' - userId string - Filter to a specific user
- model string - Filter to a specific model (e.g.
'anthropic/claude-sonnet-5.5') - provider string - Filter to a specific provider (e.g.
'anthropic') - credentialType string - Filter by
'byok'or'system'credentials - tags string[] - Filter to requests matching these tags
Each row in results contains a grouping field (matching your groupBy choice) and metrics:
- totalCost number - Total cost in USD
- marketCost number - Market cost in USD
- inputTokens number - Number of input tokens
- outputTokens number - Number of output tokens
- cachedInputTokens number - Number of cached input tokens
- cacheCreationInputTokens number - Number of cache creation input tokens
- reasoningTokens number - Number of reasoning tokens
- requestCount number - Number of requests
You can combine tracking and querying to analyze spend by tags you defined:
Provider Options
The AI Gateway provider accepts provider options that control routing behavior and provider-specific configurations.
Gateway Provider Options
You can use the gateway key in providerOptions to control how AI Gateway routes requests. Without any routing options, AI Gateway picks providers based on recent uptime and latency and falls back to other providers that serve the same model, so most applications don't need to set them.
The following gateway provider options are available:
-
order string[]
Specifies the sequence of providers to attempt when routing requests. The gateway will try providers in the order specified. If a provider fails or is unavailable, it will move to the next provider in the list.
Example:
order: ['bedrock', 'anthropic']will attempt Amazon Bedrock first, then fall back to Anthropic. -
only string[]
Restricts routing to only the specified providers. When set, the gateway will never route to providers not in this list, even if they would otherwise be available.
Because unlisted providers are never tried, the request fails instead of falling back when the listed providers are unavailable. When the requirement is about data handling or a model capability,
zeroDataRetention,disallowPromptTraining, orhasfilter by that requirement while keeping every qualifying provider eligible.Example:
only: ['anthropic', 'vertex']will only allow routing to Anthropic or Vertex AI. -
sort 'cost' | 'ttft' | 'tps'
Sorts available providers by a performance or cost metric before routing. The gateway will try the best-scoring provider first and fall back through the rest in sorted order. If unspecified, providers are ordered using the gateway's default system ranking.
'cost'— lowest cost first'ttft'— lowest time-to-first-token first'tps'— highest tokens-per-second first
When combined with
order, the user-specified providers are promoted to the front while remaining providers follow the sorted order.Example:
sort: 'ttft'will route to the provider with the fastest time-to-first-token.When
sortis active, the response'sproviderMetadata.gateway.routing.sortobject contains the sort option used, the resulting execution order, per-provider metric values, and any providers that were deprioritized. -
models string[] | [GatewayModelFallback, ...string[]]
Specifies fallback models in attempt order. String entries keep their existing behavior: the gateway tries them when the primary model fails or is unavailable. Decision requests can also start the list with one conditional
{ model, when }entry to retry a successful but uncertain decision with another model. Only the first entry can be conditional, and only one conditional entry is allowed. If the primary model fails, the gateway also advances to the conditional entry under its existing error fallback behavior. Conditional entries are rejected on non-decision requests.Example:
models: ['openai/gpt-5.4-nano', 'google/gemini-4-hydrogen']will try the fallback models in order if the primary model fails.A direct condition uses
{ question, confidenceBelow }for Choice and Score questions, or{ question, probabilityBetween: [minimum, maximum] }for Boolean questions. Leave outquestionto check every question of that type:{ confidenceBelow: 0.6 }matches when any Choice or Score answer is below 0.6, and a request with no Choice or Score questions never matches it. Boolean probability is P(true), not confidence in the selected Boolean outcome. Bounds are inclusive, finite, ordered, and within[0, 1].Combine conditions with
{ any: [...] },{ all: [...] }, or{ atLeast: { count, conditions: [...] } }. Each condition list holds 1 to 20 conditions.atLeast.countmust be an integer from1through the number of conditions. Conditions nest at most 5 levels deep. Question IDs are 1 to 256 characters, and the conditionalmodelmust not be empty. The SDK checks these bounds before sending the request, and the gateway also checks that each question exists and has a kind that matches its condition. String error fallbacks may follow the conditional entry. -
user string
Optional identifier for the end user on whose behalf the request is being made. This is used for spend tracking and attribution purposes, allowing you to track usage per end-user in your application.
Example:
user: 'user-123'will associate this request with end-user ID "user-123" in usage reports. -
tags string[]
Optional array of tags for categorizing and filtering usage in reports. Useful for tracking spend by feature, prompt version, or any other dimension relevant to your application.
Example:
tags: ['chat', 'v2']will tag this request with "chat" and "v2" for filtering in usage analytics. -
byok Record<string, Array<Record<string, unknown>>>
Request-scoped BYOK (Bring Your Own Key) credentials to use for this request. When provided, any cached BYOK credentials configured in the gateway system are not considered. Requests may still fall back to use system credentials if the provided credentials fail.
Each provider can have multiple credentials (tried in order). The structure is a record where keys are provider slugs and values are arrays of credential objects.
Each credential can optionally include a
modelMappingsarray to map AI Gateway model slugs to your deployment names (for example, custom Azure deployment names). If a BYOK request fails, the gateway falls back to system credentials using the default model name.Examples:
- Single provider:
byok: { 'anthropic': [{ apiKey: 'sk-ant-...' }] } - Multiple credentials:
byok: { 'vertex': [{ project: 'proj-1', googleCredentials: { privateKey: '...', clientEmail: '...' } }, { project: 'proj-2', googleCredentials: { privateKey: '...', clientEmail: '...' } }] } - Multiple providers:
byok: { 'anthropic': [{ apiKey: '...' }], 'bedrock': [{ accessKeyId: '...', secretAccessKey: '...' }] } - With model mappings:
byok: { 'azure': [{ apiKey: '...', resourceName: '...', modelMappings: [{ gatewayModelSlug: 'openai/gpt-5.4-nano', customModelId: 'my-deployment' }] }] }
- Single provider:
-
zeroDataRetention boolean
Restricts routing to providers with zero data retention agreements with Vercel for AI Gateway. BYOK credentials are skipped by default, since your provider agreements differ from Vercel's. When this filter is on, AI Gateway routes only to providers Vercel has ZDR agreements with for the model. If you have BYOK keys marked as ZDR, those keys are tried first, then AI Gateway falls back to its system credentials. You are responsible for the accuracy of that marking. Applies to both account-wide and request-level ZDR. The request fails if no ZDR-eligible credentials are available. Request-level ZDR is only available for Vercel Pro and Enterprise plans.
-
disallowPromptTraining boolean
Restricts routing to providers that have agreements with Vercel for AI Gateway to not use prompts for model training. When using BYOK credentials, this filter is not applied. If BYOK credentials fail and the request falls back to system credentials, only providers that do not train on prompt data will be used. If there are no providers available for the model that disallow prompt training, the request will fail.
-
headquarters string[]
Restricts routing to providers whose parent company is headquartered in the listed countries, given as ISO 3166-1 alpha-2 codes (for example
['US', 'DE']). To exclude countries instead, prefix every code with!, as in['!US']. A list can't mix allowed and excluded countries. The filter applies to both BYOK and system credentials, since it's about the provider rather than the credential. Providers without a recorded headquarters country are always excluded. If no provider for the model is headquartered in an allowed country, the request fails. -
quotaEntityId string
The unique identifier for the entity against which quota is tracked. Used for quota management and enforcement purposes.
-
has Array<'implicit-caching' | 'reasoning' | 'structured-output' | 'tool-use' | 'vision' |
quantization:${string}|!quantization:${string}>Restricts routing to provider models that have all of the specified capabilities. Applies to both BYOK and system credentials, since the capability is a property of the model rather than the credential. If no provider model for the requested model satisfies the capabilities, the request fails. Unsupported values are rejected.
Supported capabilities:
'implicit-caching'— models that perform automatic (implicit) prompt caching.'reasoning'— models that support reasoning.'structured-output'— models that support schema-constrained output.'tool-use'— models that support tool calling.'vision'— models that accept image input.
Weight-format conditions route on the serving provider's recorded weight format:
'quantization:fp8'requires a provider serving fp8 weights, while'!quantization:fp8'excludes fp8 providers. A negated condition also matches providers whose weight format is not recorded, so exclusions remain usable while the catalog is populated.Example:
has: ['implicit-caching']will only route to models that support implicit caching. Example:has: ['vision']will only route to models that accept image input. Example:has: ['tool-use']will only route to models that support tool calling. Example:has: ['!quantization:fp8']will only route to providers that do not serve fp8 weights. -
providerTimeouts object
Per-provider timeouts for BYOK credentials in milliseconds. Controls how long to wait for a provider to start responding before falling back to the next available provider.
Example:
providerTimeouts: { byok: { openai: 5000, anthropic: 2000 } }For full details, see Provider Timeouts.
-
serviceTier 'flex' | 'priority'
Unified service tier intent. The gateway translates it into whichever per-provider option each provider expects, and overrides any tier also set in the per-provider options. Leave unset for provider default. See the OpenAI, Google Generative AI, and Google Vertex AI provider docs for tier semantics, model availability, and graceful downgrade behavior on each provider.
You can combine these options to have fine-grained control over routing and tracking. For example, prefer Vertex AI while keeping fallbacks, and require zero data retention:
Model Fallbacks Example
The models option enables automatic fallback to alternative models when the primary model fails:
Conditional Decision Fallbacks Example
Conditional model entries are available only for decision requests. Keep the
configuration under providerOptions.gateway.models. experimental_decide
does not have a top-level fallback option.
When the condition matches, result.response.modelId names the fallback model,
and providerMetadata.gateway.routing.modelAttempts lists the primary and
fallback attempts. The successful attempt of each stage carries its own
generationId, usage, and cost fields, and the first fallback attempt lists
the questions that triggered it in triggeredBy.
Zero Data Retention Example
Set zeroDataRetention to true to route requests to providers with zero data retention agreements with Vercel for AI Gateway. BYOK credentials are skipped by default, since your provider agreements differ from Vercel's. When this filter is on, AI Gateway routes only to providers Vercel has ZDR agreements with for the model. If you have BYOK keys marked as ZDR, those keys are tried first, then AI Gateway falls back to its system credentials. You are responsible for the accuracy of that marking. Applies to both account-wide and request-level ZDR. The request fails if no ZDR-eligible credentials are available. When zeroDataRetention is false or not specified, there is no enforcement of restricting routing. Request-level ZDR is only available for Vercel Pro and Enterprise plans.
Disallow Prompt Training Example
Set disallowPromptTraining to true to route requests to providers that have agreements with Vercel for AI Gateway to not use prompts for model training. When using BYOK credentials, this filter is not applied. If BYOK credentials fail and the request falls back to system credentials, only providers that do not train on prompt data will be used. If there are no providers available for the model that disallow prompt training, the request will fail. When disallowPromptTraining is false or not specified, there is no enforcement of restricting routing.
Quota Entity ID Example
Set quotaEntityId to track and enforce quota against a specific entity. This is useful for multi-tenant applications where you need to manage quota at the entity level (e.g., per organization or team).
Filtering by Model Capability
Set has to restrict routing to provider models that have the specified capabilities. This applies to both BYOK and system credentials, since the capability is a property of the model rather than the credential. 'implicit-caching' limits routing to models that perform automatic prompt caching, 'vision' limits routing to models that accept image input, 'reasoning' limits routing to models that support reasoning, 'structured-output' limits routing to models that support schema-constrained output, and 'tool-use' limits routing to models that support tool calling. Weight-format conditions ('quantization:fp8' to require, '!quantization:fp8' to exclude) limit routing by the serving provider's recorded weight format. If no provider model for the requested model satisfies the capabilities, the request fails.
Use 'vision' when the request sends image input, so the gateway only routes to
provider models that can accept it:
Provider-Specific Options
When using provider-specific options through AI Gateway, use the actual provider name (e.g. anthropic, openai, not gateway) as the key:
This works with any provider supported by AI Gateway. Each provider has its own set of options - see the individual provider documentation pages for details on provider-specific options.
Available Providers
AI Gateway supports routing to 20+ providers.
For a complete list of available providers and their slugs, see the AI Gateway documentation.
Model Capabilities
Model capabilities depend on the specific provider and model you're using. For detailed capability information, see:
- AI Gateway provider options for an overview of available providers
- Individual AI SDK provider pages for specific model capabilities and features