Batch | AI SDK

Batch support is experimental and the API may change in patch releases.

Batches let you submit multiple independent requests for asynchronous processing. The provider processes the batch in the background, so your application does not need to keep the request open while the model generates the results. This is useful for workloads such as classification, summarization, and content generation that do not need an immediate response.

The batch API supports text generation with type: 'text' and image generation with type: 'image'.

The AI SDK provides five functions for the batch lifecycle:

experimental_startBatch submits a batch and returns its initial status and a serializable reference.
experimental_getBatchStatus retrieves the latest status and request counts.
experimental_getBatchResults asynchronously iterates over the terminal result for each request.
experimental_cancelBatch requests cancellation of a batch.
experimental_listBatches lists batches and their latest statuses.

All five functions are exported from ai. The examples below use aliases so the shorter names can be used in application code:

import {
  experimental_cancelBatch as cancelBatch,
  experimental_getBatchResults as getBatchResults,
  experimental_getBatchStatus as getBatchStatus,
  experimental_listBatches as listBatches,
  experimental_startBatch as startBatch,
} from 'ai';

Cancellation and listing are optional provider capabilities. Calling either function with a provider that does not implement it throws an UnsupportedFunctionalityError. In the cancellation and listing examples below, provider represents a batch provider that implements the respective capability.

Supported providers

Batch processing requires a provider that implements the batch interface. Support is provider- and model-specific. The first-party providers with support are:

Provider	Provider value	Request types	Provider API
Anthropic	`anthropic`	text	Message Batches API
Google	`google`	text, image	Gemini Batch API
OpenAI	`openai`	text	Batch API
xAI	`xai`	text, image	Batch API
AI Gateway	global default	text	Batch processing

See the provider documentation for the supported models, limits, and native batch behavior. For example, OpenAI batch support is available through the Responses API, not openai.chat().

Starting a batch

Pass a provider and one or more uniquely identified requests to startBatch. Each text request must specify type: 'text', a model ID, and either a text prompt or a messages array. Requests can use different model IDs when supported by the provider. Each request can also use the usual text-generation settings such as instructions, maxOutputTokens, temperature, topP, topK, presencePenalty, frequencyPenalty, stopSequences, seed, and reasoning.

Tool definitions and tool settings are provided on individual requests. A tool with the same name must have the same definition in every request that uses it.

import { anthropic } from '@ai-sdk/anthropic';
import { experimental_startBatch as startBatch } from 'ai';

const provider = anthropic;

const batch = await startBatch({
  provider,
  requests: [
    {
      id: 'capital-france',
      type: 'text',
      model: 'claude-haiku-5-5',
      prompt: 'What is the capital of France?',
    },
    {
      id: 'capital-germany',
      type: 'text',
      model: 'claude-haiku-5-5',
      prompt: 'What is the capital of Germany?',
    },
  ],
});

console.log(batch.id, batch.status);

Request IDs must be non-empty and unique within the batch. They are the link between an input request and its result. Results are not guaranteed to arrive in input order, so use the ID rather than an array position when associating a result with application data.

You can pass provider-specific settings at the batch level with providerOptions, or on an individual request with that request's providerOptions. Settings supported by the provider can differ between the two levels. The warnings property on the start result contains warnings about unsupported settings.

The batch reference returned by startBatch is serializable. Persist it before the process exits if the batch will be completed by another process or at a later time. When retrieving a batch, pass the same provider. The AI SDK uses the reference to ensure that a batch is not read through an incompatible provider.

Image requests

Image requests use the same prompt and generation settings as generateImage: prompt, n, size, aspectRatio, seed, and providerOptions. A prompt can also contain input images and a mask for providers that support image editing. Provider batch endpoints may support only a subset of these options; unsupported options produce warnings or errors.

import { google } from '@ai-sdk/google';
import { experimental_startBatch as startBatch } from 'ai';

const batch = await startBatch({
  provider: google,
  requests: [
    {
      id: 'red-panda',
      type: 'image',
      model: 'gemini-3.1-flash-image-preview',
      prompt: 'A red panda reading beside a cabin window',
      aspectRatio: '16:9',
    },
  ],
});

Successful image items have an images array of GeneratedFile values, plus image warnings, response metadata, usage, and provider metadata when the provider supplies them. Successful text items have text and content properties. Use the type property to narrow mixed results:

for await (const item of getBatchResults({ provider: google, batch })) {
  if (item.type === 'image' && item.status === 'succeeded') {
    console.log(item.images);
  }
}

Batch tools

Batch text requests support both client-defined tools and provider-defined tools. Tool definitions are sent with each request that includes them, so the model can call them independently for each request.

Client-defined tools are definition-only in a batch. Their execute functions are never invoked, and the AI SDK does not submit tool results or start a follow-up generation after retrieving a tool call. Pass the same tool set to getBatchResults to validate and normalize returned tool calls:

import { anthropic } from '@ai-sdk/anthropic';
import {
  experimental_getBatchResults as getBatchResults,
  experimental_startBatch as startBatch,
  tool,
} from 'ai';
import { z } from 'zod';

const tools = {
  get_weather: tool({
    description: 'Get the current weather for a location.',
    inputSchema: z.object({ location: z.string() }),
    execute: async ({ location }) => {
      // This function is not called by batch processing.
      return { location, temperature: 21, condition: 'sunny' };
    },
  }),
};

const batch = await startBatch({
  provider: anthropic,
  requests: [
    {
      id: 'weather-san-francisco',
      type: 'text',
      model: 'claude-haiku-4-5',
      prompt: 'Call get_weather for San Francisco, California.',
      tools,
      toolChoice: { type: 'tool', toolName: 'get_weather' },
    },
  ],
});

for await (const item of getBatchResults({
  provider: anthropic,
  batch,
  tools,
})) {
  if (item.status === 'succeeded') {
    console.log(item.id, item.content);
  }
}

Provider-defined tools, such as web search or code execution, can execute on the provider when supported by that provider's batch API. Their tool calls and results are returned as normalized content parts. See the provider documentation for the tools and models available in batches.

Checking batch status

The start result includes the initial status. Use getBatchStatus to retrieve the latest status while the provider is processing the batch:

import { setTimeout } from 'node:timers/promises';

let status = batch.status;
let error = batch.error;

while (status === 'pending') {
  await setTimeout(10_000);
  const latestStatus = await getBatchStatus({
    provider: anthropic,
    batch,
  });
  status = latestStatus.status;
  error = latestStatus.error;
}

if (status === 'failed') {
  throw new Error(error?.message ?? 'The batch failed.');
}

The normalized batch status is one of:

pending — the provider is still processing the batch.
completed — the batch reached a terminal state and results can be retrieved.
failed — the batch could not be completed. The error property may contain additional details.

Status responses can also include requestCounts, createdAt, expiresAt, rawStatus, and provider metadata. requestCounts reports the total, pending, completed, and failed requests known by the provider.

For webhook-capable providers, pass webhookUrl to startBatch to receive a notification when the batch reaches a terminal state. Webhook support and payloads are provider-specific. The provider pages linked above describe their webhook behavior; providers that do not support webhooks return an unsupported warning and continue without one.

Cancelling a batch

Use cancelBatch to ask the provider to stop processing a batch:

const result = await cancelBatch({
  provider,
  batch,
});

console.log(result.providerMetadata);

A successful call means that the provider accepted the cancellation request. It does not guarantee that cancellation has finished or that every pending request will be cancelled. Use getBatchStatus to retrieve the latest status after requesting cancellation.

The result can include provider metadata with additional information from the cancellation response.

Listing batches

Use listBatches to retrieve batches from the provider. Results are paginated. Pass the returned nextCursor as cursor to retrieve the next page:

let cursor: string | undefined;

do {
  const page = await listBatches({
    provider,
    limit: 20,
    cursor,
  });

  for (const batch of page.batches) {
    console.log(batch.id, batch.status);
  }

  cursor = page.nextCursor;
} while (cursor != null);

Each item is a serializable batch reference with its latest normalized status, so it can be passed directly to getBatchStatus, getBatchResults, or cancelBatch. Cursors are opaque and provider-specific; applications should store or pass them unchanged rather than inspect their contents.

Retrieving results

After the batch is complete, getBatchResults returns an async iterable. Each item contains the ID and terminal status for one input request:

for await (const item of getBatchResults({ provider: anthropic, batch })) {
  if (item.status === 'succeeded') {
    console.log(item.id, item.text);
  } else {
    console.error(item.id, item.status, item.error);
  }
}

Successful items include:

text — the concatenated text content. This can be an empty string when the result contains no text parts.
content — ordered, normalized content parts, including text, reasoning, files, sources, tool calls, tool results, and provider content where supported.
finishReason and optional rawFinishReason.
usage and optional response metadata.
Optional providerMetadata.

Failed, cancelled, and expired items include their id and status. Failed items include an error; cancelled and expired items may also include one. One failed request does not necessarily mean that every request in the batch failed, so handle each item independently.

Batch retrieval does not run an AI SDK tool loop or invoke client-defined execute functions. Provider-defined tools may execute on the provider as part of batch processing. Treat result content and provider metadata as untrusted model output, and avoid logging it indiscriminately because it can contain sensitive data.

Request controls

All batch lifecycle functions accept providerOptions, headers, timeout, and abortSignal for the current operation. getBatchStatus, getBatchResults, and listBatches also accept maxRetries:

maxRetries controls retries for status, result retrieval, and listing. It does not retry batch creation or cancellation. It defaults to 2; set it to 0 to disable retries.
abortSignal cancels the current API request.
timeout limits the current HTTP operation.

These controls affect communication with the provider. They do not change the provider's processing deadline or cancel a batch that has already been submitted.