--- title: DeepInfra description: Learn how to use DeepInfra with AI Gateway and the direct provider. url: "https://ai-sdk.dev/providers/ai-sdk-providers/deepinfra" docs_index: /llms.txt --- > For an index of all documentation, see [/llms.txt](/llms.txt). The [DeepInfra](https://deepinfra.com) provider contains support for state-of-the-art models through the DeepInfra API, including Llama 3, Mixtral, Qwen, and many other popular open-source models. Start with [AI Gateway](/providers/ai-sdk-providers/ai-gateway) for models in its catalog (recommended), or use `@ai-sdk/deepinfra` to connect directly to DeepInfra. ```ts provider="deepinfra" model="meta/llama-4-scout" import { generateText } from 'ai'; const { text } = await generateText({ model: "meta/llama-4-scout", prompt: 'Write a vegetarian lasagna recipe for 4 people.', providerOptions: { gateway: { order: ['deepinfra'], }, }, }); console.log(text); ``` ## Using AI Gateway The AI SDK uses AI Gateway by default for model strings. Gateway provides access to models across providers through a single API key, with usage and cost tracking. Install `ai`: ```bash pnpm add ai ``` Set an [AI Gateway API key](https://vercel.com/docs/ai-gateway/getting-started) in your environment: ```bash export AI_GATEWAY_API_KEY=your_api_key_here ``` Vercel deployments can also use [OIDC authentication](/providers/ai-sdk-providers/ai-gateway#oidc-authentication-vercel-deployments). See the [AI Gateway provider documentation](/providers/ai-sdk-providers/ai-gateway) for configuration and the [model catalog](https://vercel.com/ai-gateway/models) for available models. The example sets `providerOptions.gateway.order` to `['deepinfra']`. This sends the request to DeepInfra first, then falls back to other available providers of the same model if DeepInfra is unavailable or the request fails. `order` sets a preference; it does not restrict fallback to DeepInfra. See [provider ordering](https://vercel.com/docs/ai-gateway/models-and-providers/provider-filtering-and-ordering). The dropdown lists models served by DeepInfra through Gateway with a corresponding direct-provider example. Provider availability depends on the model and your team's [Provider Allowlist](https://vercel.com/docs/ai-gateway/models-and-providers/provider-filtering-and-ordering). ## Using `@ai-sdk/deepinfra` For direct access to DeepInfra, install `ai` and `@ai-sdk/deepinfra`: ```bash pnpm add ai @ai-sdk/deepinfra ``` Set your DeepInfra API credential in the `DEEPINFRA_API_KEY` environment variable: ```bash export DEEPINFRA_API_KEY=your_api_key_here ``` AI Gateway does not expose every API in `@ai-sdk/deepinfra`. Use the direct provider for provider-specific APIs or custom connection settings. The following sections document the direct `@ai-sdk/deepinfra` provider, including its model factories and settings. Gateway model and feature availability can differ. ## Provider Instance You can import the default provider instance `deepInfra` from `@ai-sdk/deepinfra`: ```ts import { deepInfra } from '@ai-sdk/deepinfra'; ``` If you need a customized setup, you can import `createDeepInfra` from `@ai-sdk/deepinfra` and create a provider instance with your settings: ```ts import { createDeepInfra } from '@ai-sdk/deepinfra'; const deepInfra = createDeepInfra({ apiKey: process.env.DEEPINFRA_API_KEY ?? '', }); ``` You can use the following optional settings to customize the DeepInfra provider instance: - **baseURL** *string* Use a different URL prefix for API calls, e.g. to use proxy servers. The default prefix is `https://api.deepinfra.com/v1`. Note: Language models and embeddings use OpenAI-compatible endpoints at `{baseURL}/openai`, while image models use `{baseURL}/inference`. - **apiKey** *string | PromiseLike\ | (() => string | PromiseLike\)* API key that is being sent using the `Authorization` header. It defaults to the `DEEPINFRA_API_KEY` environment variable. A function is called before each request and may return a promise. Use it to supply refreshed credentials; a promise value resolves to the same key each time. - **headers** *Record\* Custom headers to include in the requests. - **fetch** *(input: RequestInfo, init?: RequestInit) => Promise\* Custom [fetch](https://developer.mozilla.org/en-US/docs/Web/API/fetch) implementation. Defaults to the global `fetch` function. You can use it as a middleware to intercept requests, or to provide a custom fetch implementation for e.g. testing. ## Language Models You can create language models using a provider instance. The first argument is the model ID, for example: ```ts import { deepInfra } from '@ai-sdk/deepinfra'; import { generateText } from 'ai'; const { text } = await generateText({ model: deepInfra('meta-llama/Meta-Llama-3.1-70B-Instruct'), prompt: 'Write a vegetarian lasagna recipe for 4 people.', }); ``` DeepInfra language models can also be used in the `streamText` function (see [AI SDK Core](/docs/ai-sdk-core)). ## Model Capabilities | Model | Image Input | Object Generation | Tool Usage | Tool Streaming | | --------------------------------------------------- | ----------- | ----------------- | ---------- | -------------- | | `meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8` | ✓ | ✗ | ✗ | ✗ | | `meta-llama/Llama-4-Scout-17B-16E-Instruct` | ✓ | ✗ | ✗ | ✗ | | `meta-llama/Llama-3.3-70B-Instruct-Turbo` | ✗ | ✓ | ✓ | ✓ | | `meta-llama/Llama-3.3-70B-Instruct` | ✗ | ✓ | ✓ | ✓ | | `meta-llama/Meta-Llama-3.1-405B-Instruct` | ✗ | ✓ | ✓ | ✓ | | `meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo` | ✗ | ✓ | ✓ | ✓ | | `meta-llama/Meta-Llama-3.1-70B-Instruct` | ✗ | ✓ | ✓ | ✓ | | `meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo` | ✗ | ✓ | ✓ | ✗ | | `meta-llama/Meta-Llama-3.1-8B-Instruct` | ✗ | ✓ | ✓ | ✓ | | `meta-llama/Llama-3.2-11B-Vision-Instruct` | ✓ | ✓ | ✗ | ✗ | | `meta-llama/Llama-3.2-90B-Vision-Instruct` | ✓ | ✓ | ✗ | ✗ | | `mistralai/Mixtral-8x7B-Instruct-v0.1` | ✗ | ✓ | ✓ | ✗ | | `deepseek-ai/DeepSeek-V3` | ✗ | ✓ | ✓ | ✓ | | `deepseek-ai/DeepSeek-R1` | ✗ | ✗ | ✗ | ✗ | | `deepseek-ai/DeepSeek-R1-Distill-Llama-70B` | ✗ | ✗ | ✗ | ✗ | | `deepseek-ai/DeepSeek-R1-Turbo` | ✗ | ✗ | ✗ | ✗ | | `nvidia/Llama-3.1-Nemotron-70B-Instruct` | ✗ | ✓ | ✓ | ✗ | | `Qwen/Qwen2-7B-Instruct` | ✗ | ✓ | ✗ | ✗ | | `Qwen/Qwen2.5-72B-Instruct` | ✗ | ✓ | ✓ | ✓ | | `Qwen/Qwen2.5-Coder-32B-Instruct` | ✗ | ✓ | ✗ | ✗ | | `Qwen/QwQ-32B-Preview` | ✗ | ✓ | ✗ | ✗ | | `google/codegemma-7b-it` | ✗ | ✗ | ✗ | ✗ | | `google/gemma-2-9b-it` | ✗ | ✗ | ✗ | ✗ | | `microsoft/WizardLM-2-8x22B` | ✗ | ✗ | ✗ | ✗ | The table above lists popular models. Please see the [DeepInfra docs](https://deepinfra.com) for a full list of available models. You can also pass any available provider model ID as a string if needed. ## Image Models You can create DeepInfra image models using the `.image()` factory method. For more on image generation with the AI SDK see [generateImage()](/docs/reference/ai-sdk-core/generate-image). ```ts import { deepInfra, type DeepInfraImageModelOptions } from '@ai-sdk/deepinfra'; import { generateImage } from 'ai'; const { image } = await generateImage({ model: deepInfra.image('stabilityai/sd3.5'), prompt: 'A futuristic cityscape at sunset', aspectRatio: '16:9', }); ``` Model support for `size` and `aspectRatio` parameters varies by model. Please check the individual model documentation on [DeepInfra's models page](https://deepinfra.com/models/text-to-image) for supported options and additional parameters. ### Model-specific options You can pass model-specific parameters using the `providerOptions.deepinfra` field: ```ts import { deepInfra } from '@ai-sdk/deepinfra'; import { generateImage } from 'ai'; const { image } = await generateImage({ model: deepInfra.image('stabilityai/sd3.5'), prompt: 'A futuristic cityscape at sunset', aspectRatio: '16:9', providerOptions: { deepinfra: { num_inference_steps: 30, } satisfies DeepInfraImageModelOptions, }, }); ``` The following options are available via `providerOptions.deepinfra`. You can type them with `DeepInfraImageModelOptions`. - **negative\_prompt** *string* Text description of what to avoid in the generated image. - **num\_inference\_steps** *number* Number of denoising steps for models that expose this option. - **guidance\_scale** *number* Guidance scale for models that expose this option. - **guidance** *number* Guidance value for image edit models that expose this option. - **response\_format** *'b64\_json'* Format for OpenAI-compatible image responses. DeepInfra currently supports `b64_json`. - **quality** *string* OpenAI-compatible image quality option. - **style** *string* OpenAI-compatible image style option. - **user** *string* Unique identifier for your end user. DeepInfra image generation models expose model-specific parameters. Additional fields can be passed through `providerOptions.deepinfra` and are validated by DeepInfra. ### Image Editing DeepInfra supports image editing through models like `Qwen/Qwen-Image-Edit`. Pass input images via `prompt.images` to transform or edit existing images. #### Basic Image Editing Transform an existing image using text prompts: ```ts const imageBuffer = readFileSync('./input-image.png'); const { images } = await generateImage({ model: deepInfra.image('Qwen/Qwen-Image-Edit'), prompt: { text: 'Turn the cat into a golden retriever dog', images: [imageBuffer], }, size: '1024x1024', }); ``` #### Inpainting with Mask Edit specific parts of an image using a mask. Transparent areas in the mask indicate where the image should be edited: ```ts const image = readFileSync('./input-image.png'); const mask = readFileSync('./mask.png'); const { images } = await generateImage({ model: deepInfra.image('Qwen/Qwen-Image-Edit'), prompt: { text: 'A sunlit indoor lounge area with a pool containing a flamingo', images: [image], mask: mask, }, }); ``` #### Multi-Image Combining Combine multiple reference images into a single output: ```ts const cat = readFileSync('./cat.png'); const dog = readFileSync('./dog.png'); const { images } = await generateImage({ model: deepInfra.image('Qwen/Qwen-Image-Edit'), prompt: { text: 'Create a scene with both animals together, playing as friends', images: [cat, dog], }, }); ``` Input images can be provided as `Buffer`, `ArrayBuffer`, `Uint8Array`, or base64-encoded strings. DeepInfra uses an OpenAI-compatible image editing API at `https://api.deepinfra.com/v1/openai/images/edits`. ### Model Capabilities For models supporting aspect ratios, the following ratios are typically supported: `1:1 (default), 16:9, 1:9, 3:2, 2:3, 4:5, 5:4, 9:16, 9:21` For models supporting size parameters, dimensions must typically be: - Multiples of 32 - Width and height between 256 and 1440 pixels - Default size is 1024x1024 | Model | Dimensions Specification | Notes | | -------------------------------------- | ------------------------ | -------------------------------------------------------- | | `stabilityai/sd3.5` | Aspect Ratio | Premium quality base model, 8B parameters | | `black-forest-labs/FLUX-1.1-pro` | Size | Latest state-of-art model with superior prompt following | | `black-forest-labs/FLUX-1-schnell` | Size | Fast generation in 1-4 steps | | `black-forest-labs/FLUX-1-dev` | Size | Optimized for anatomical accuracy | | `black-forest-labs/FLUX-pro` | Size | Flagship Flux model | | `black-forest-labs/FLUX.1-Kontext-dev` | Size | Image editing and transformation model | | `black-forest-labs/FLUX.1-Kontext-pro` | Size | Professional image editing and transformation | | `stabilityai/sd3.5-medium` | Aspect Ratio | Balanced 2.5B parameter model | | `stabilityai/sdxl-turbo` | Aspect Ratio | Optimized for fast generation | For more details and pricing information, see the [DeepInfra text-to-image models page](https://deepinfra.com/models/text-to-image). ## Embedding Models You can create DeepInfra embedding models using the `.embeddingModel()` factory method. For more on embedding models with the AI SDK see [embed()](/docs/reference/ai-sdk-core/embed). ```ts import { deepInfra } from '@ai-sdk/deepinfra'; import { embed } from 'ai'; const { embedding } = await embed({ model: deepInfra.embeddingModel('BAAI/bge-large-en-v1.5'), value: 'sunny day at the beach', }); ``` ### Model Capabilities | Model | Dimensions | Max Tokens | | ----------------------------------------------------- | ---------- | ---------- | | `BAAI/bge-base-en-v1.5` | 768 | 512 | | `BAAI/bge-large-en-v1.5` | 1024 | 512 | | `BAAI/bge-m3` | 1024 | 8192 | | `intfloat/e5-base-v2` | 768 | 512 | | `intfloat/e5-large-v2` | 1024 | 512 | | `intfloat/multilingual-e5-large` | 1024 | 512 | | `sentence-transformers/all-MiniLM-L12-v2` | 384 | 256 | | `sentence-transformers/all-MiniLM-L6-v2` | 384 | 256 | | `sentence-transformers/all-mpnet-base-v2` | 768 | 384 | | `sentence-transformers/clip-ViT-B-32` | 512 | 77 | | `sentence-transformers/clip-ViT-B-32-multilingual-v1` | 512 | 77 | | `sentence-transformers/multi-qa-mpnet-base-dot-v1` | 768 | 512 | | `sentence-transformers/paraphrase-MiniLM-L6-v2` | 384 | 128 | | `shibing624/text2vec-base-chinese` | 768 | 512 | | `thenlper/gte-base` | 768 | 512 | | `thenlper/gte-large` | 1024 | 512 | For a complete list of available embedding models, see the [DeepInfra embeddings page](https://deepinfra.com/models/embeddings). --- For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md) For an index of all available documentation, see [/llms.txt](/llms.txt) For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)