HeyGen
The HeyGen provider supports heygen-video-1, a video generation model with text-to-video, image-to-video, and reference-to-video modes. Generated clips include audio.
It also supports HeyGen Decision models, which classify, check, and score text, images, and video.
You can use HeyGen models through AI Gateway (recommended) or connect directly with @ai-sdk/heygen.
Using AI Gateway
The AI SDK uses AI Gateway by default for model strings. Gateway provides access to models across providers through a single API key, with usage and cost tracking. Install ai:
$ pnpm add ai
$ npm install ai
$ yarn add ai
$ bun add ai
Set an AI Gateway API key in your environment:
Vercel deployments can also use OIDC authentication. See the AI Gateway provider documentation for configuration and the model catalog for available models.
Gateway feature and provider-option support depends on the model and the provider serving the request. See Gateway provider options.
Using @ai-sdk/heygen
For direct access to HeyGen, install ai and @ai-sdk/heygen:
$ pnpm add ai @ai-sdk/heygen
$ npm install ai @ai-sdk/heygen
$ yarn add ai @ai-sdk/heygen
$ bun add ai @ai-sdk/heygen
Set your HeyGen API credential in the HEYGEN_API_KEY environment variable:
AI Gateway does not expose every API in @ai-sdk/heygen. Use the direct provider for provider-specific APIs or custom connection settings.
The following sections document the direct @ai-sdk/heygen provider,
including its model factories and settings. Gateway model and feature
availability can differ.
This provider supports video generation and decision models. HeyGen's avatar, voice, translation, and video editing APIs are not included.
Provider Instance
Import the default instance:
Or create a customized instance:
The following settings are optional:
-
apiKey string | PromiseLike<string> | (() => string | PromiseLike<string>): The key sent in the
x-api-keyheader. Defaults to theHEYGEN_API_KEYenvironment variable.Pass a function to resolve credentials before each request, including when keys rotate. The function can return a string or a promise. Credential caching and refresh are managed by the function.
-
baseURL string: The API URL prefix. Defaults to
https://api.heygen.com. -
headers Record<string, string>: Additional request headers.
-
fetch typeof fetch: A custom fetch implementation.
Video Models
Create a model with heygen.video('heygen-video-1') or heygen.videoModel('heygen-video-1').
Text to Video
Image to Video
Pass an image in the prompt to use it as the first frame. The video follows the image's aspect ratio; a supplied aspectRatio is ignored with a warning.
A single first_frame in frameImages is also supported. Last-frame conditioning and multiple first frames are rejected.
Reference to Video
Use inputReferences to condition a new scene on images or videos. Include a mediaType for explicit URL references so the provider can select the reference kind.
References are grouped by media type, preserving order within each group. Address them in the prompt as <Picture 1>, <Video 1>, and <Audio 1>. Provider-option references are appended after standard references of the same kind.
Reference-to-video requires at least one image or video. Audio-only conditioning is not supported. The adapter accepts up to 9 image references, 3 video references, and 3 audio references, with a combined maximum of 12. Reference videos share a 15-second input budget across the request. Footage within that budget is used in full; longer references are allocated and trimmed by HeyGen. See the reference-video rules.
Provider Options
- mode 'text_to_video' | 'image_to_video' | 'reference_to_video': Inferred from the inputs when omitted. Explicit modes must match the supplied inputs. First-frame images cannot be combined with references.
- resolution '480p' | '768p' | '1080p' | '2k': Output size class. Defaults to
768p. Takes precedence over the standardresolutionoption. - promptEnhancement 'turbo' | 'quality' | 'ultraquality' | 'disabled': HeyGen's prompt enhancement setting. The API defaults to
quality. - image HeyGenVideoAsset: First-frame input, including an existing asset ID. Cannot be combined with a standard image or
frameImages. - referenceImages HeyGenVideoAsset[]: Additional image references.
- referenceVideos HeyGenVideoAsset[]: Additional video references.
- referenceAudio HeyGenVideoAsset[]: Additional audio references.
HeyGenVideoAsset supports these representations:
Asset IDs must already exist in the workspace associated with the API key. This provider does not upload or manage HeyGen assets. Standard file inputs are sent inline as base64. HeyGen permits up to 5 MB per inline image and 16 MB per inline video or audio input. URL inputs must use HTTPS and be directly accessible without redirects; HeyGen permits up to 16 MB per image URL and 32 MB per video or audio URL.
Resolution and Aspect Ratio
Prefer providerOptions.heygen.resolution to select a size class. The standard resolution option accepts the exact frame sizes below and infers the corresponding aspect ratio when aspectRatio is omitted:
| Aspect ratio | 480p | 768p | 1080p | 2k |
|---|---|---|---|---|
21:9 | 960x416 | 1536x672 | — | — |
16:9 | 832x480 | 1344x768 | 1890x1080 | 2688x1536 |
4:3 | 640x480 | 1024x768 | — | — |
1:1 | 480x480 | 768x768 | — | — |
3:4 | 480x640 | 768x1024 | — | — |
9:16 | 480x832 | 768x1344 | 1080x1890 | 1536x2688 |
Text-to-video defaults to 16:9. Reference-to-video defaults to adaptive, inheriting the first reference image or video's shape. At 1080p or 2k, text-to-video and reference-to-video require 16:9 or 9:16. Image-to-video always inherits the first frame's shape, even when a standard frame size is supplied; only its size class is used.
Other Constraints
- Prompts must contain between 1 and 32,000 characters.
- The SDK accepts positive whole-second durations and forwards them unchanged. HeyGen currently supports 5–30 seconds for text-to-video and image-to-video with
qualityorturboprompt enhancement, and 5–15 seconds for reference-to-video orultraquality/disabledenhancement. The default duration is 5. The API enforces these limits; see duration limits. - Seeds must be unsigned 32-bit integers.
- One video is generated per provider call.
- Output is MP4 at 24 fps with generated audio. Unsupported
fpsvalues andgenerateAudio: falseproduce warnings.
Asynchronous Generation
The provider implements doStart and doStatus. The AI SDK orchestrates polling for generateVideo. Native webhook completion is not implemented; polling is used.
HeyGen jobs continue running after submission. Aborting a local request does not cancel the remote generation. The returned video URL is signed; retrieving the same operation's status again refreshes the URL.
Pass an Idempotency-Key through the standard headers option when retrying the same submission. HeyGen retains keys for 24 hours; concurrent duplicates return an HTTP 409 error.
Provider Metadata
providerMetadata.heygen.videos contains one entry per completed video. The AI SDK combines these entries when multiple videos are requested. Each entry contains:
videoId: The HeyGen video identifier.mode: The generation mode submitted to HeyGen.resolution: The resolved size class submitted to HeyGen, retained across asynchronous status calls. This is separate from the actual output dimensions.duration,width,height,aspectRatio, andseed: Values reported by HeyGen on completion, when available.timings.inference: The inference timing reported by HeyGen, when available.
Submission and in-progress operation metadata exposes videoId, mode, and resolution directly under providerMetadata.heygen. Failed or cancelled operations also expose failureCode when HeyGen reports it.
The current HeyGen video endpoint does not document token counts or credit consumption in its generation response. The provider returns the reported duration and dimensions without estimating those quantities. Reported inference timing is preserved per video.
Decision Models
Use heygen.decisionModel() with
experimental_decide to call the
HeyGen Decision API.
This API is experimental and may change in patch releases.
heygen-decision-1 and heygen-decision-1-turbo are available. The provider
maps SDK boolean questions to HeyGen's native noul type and returns Choice
and Score probability distributions without rounding. Box questions are
unsupported.
Images and video
Add image and video file parts to state. The provider sends them as HeyGen
assets and keeps text and JSON parts in the native state. HeyGen reads
images first, then videos, then the state, each in request order, so describe
each input's role in the text by its order:
Inline file data is sent as base64. When calling the model directly, HTTPS URL
file data is passed to HeyGen as a URL, and a provider reference such as
{ type: 'reference', reference: { heygen: assetId } } is sent as an uploaded
HeyGen asset. experimental_decide downloads URL file state before calling the
provider.
Confidence, legends, and response metadata
HeyGen's confidence values and Score legends are available in
providerMetadata.heygen, keyed by question ID. Confidence is distinct from the
selected option's probability. Per-answer diagnostic warnings returned by HeyGen
are available at providerMetadata.heygen.answerWarnings.
Decision limits
The provider validates these limits before sending the decision request:
| Input | Limit |
|---|---|
| Questions | 1–400; names nonempty, without colons, newlines, or surrounding whitespace |
| Choice options | 1–255 |
| Score levels | 1–10, all non-null |
| Media | At most 4 images and 1 video; HTTPS URLs only |
See HeyGen's media limits
and context limits
for API-enforced file-size and token limits. HeyGen decision models do not accept
provider-specific options, maxOutputTokens, or reasoning; these produce
unsupported warnings and are not sent to the API.
Decision Model Capabilities
| Model | Boolean | Choice | Score | Box | Image Input | Reasoning | Max Output Tokens |
|---|---|---|---|---|---|---|---|
heygen-decision-1 | |||||||
heygen-decision-1-turbo |
Model Capabilities
| Model | Image Input | First Frame | Last Frame | Reference Images | Reference Video | Audio | Aspect Ratio | Resolution | Duration | FPS | Seed |
|---|---|---|---|---|---|---|---|---|---|---|---|
heygen-video-1 | Conditional: Audio is always generated; disabling it is unsupported. | Conditional: Supported outside reference-to-video mode. | Conditional: Supported outside reference-to-video mode. | Conditional: Fixed at 24 fps. |
Provider Option Reference
The following option definitions are generated from the provider catalog.
| Operation | Option | Description |
|---|---|---|
| video | mode | Inferred from the supplied image and references when omitted. |
| video | resolution | Output size class. Takes precedence over the standard resolution option. |
| video | promptEnhancement | Defaults to quality. Use disabled to preserve the original prompt. |
| video | image | First frame, including an existing HeyGen asset. Cannot be combined with a standard image. |
| video | referenceImages | Additional references, appended after standard inputReferences of the same kind. |
| video | referenceVideos | referenceVideos |
| video | referenceAudio | referenceAudio |