DeepInfra

The DeepInfra provider contains support for state-of-the-art models through the DeepInfra API, including Llama 3, Mixtral, Qwen, and many other popular open-source models.

Setup

The DeepInfra provider is available via the @ai-sdk/deepinfra module. You can install it with:

$ pnpm add @ai-sdk/deepinfra

$ npm install @ai-sdk/deepinfra

$ yarn add @ai-sdk/deepinfra

$ bun add @ai-sdk/deepinfra

Provider Instance

You can import the default provider instance deepinfra from @ai-sdk/deepinfra:

import { deepinfra } from '@ai-sdk/deepinfra';

If you need a customized setup, you can import createDeepInfra from @ai-sdk/deepinfra and create a provider instance with your settings:

import { createDeepInfra } from '@ai-sdk/deepinfra';

const deepinfra = createDeepInfra({
  apiKey: process.env.DEEPINFRA_API_KEY ?? '',
});

You can use the following optional settings to customize the DeepInfra provider instance:

• baseURL string

  Use a different URL prefix for API calls, e.g. to use proxy servers. The default prefix is https://api.deepinfra.com/v1.

  Note: Language models and embeddings use OpenAI-compatible endpoints at {baseURL}/openai, while image models use {baseURL}/inference.

• apiKey string

  API key that is being sent using the Authorization header. It defaults to the DEEPINFRA_API_KEY environment variable.

• headers Record<string,string>

  Custom headers to include in the requests.

• fetch (input: RequestInfo, init?: RequestInit) => Promise<Response>

  Custom fetch implementation. Defaults to the global fetch function. You can use it as a middleware to intercept requests, or to provide a custom fetch implementation for e.g. testing.

Language Models

You can create language models using a provider instance. The first argument is the model ID, for example:

import { deepinfra } from '@ai-sdk/deepinfra';
import { generateText } from 'ai';

const { text } = await generateText({
  model: deepinfra('meta-llama/Meta-Llama-3.1-70B-Instruct'),
  prompt: 'Write a vegetarian lasagna recipe for 4 people.',
});

DeepInfra language models can also be used in the streamText function (see AI SDK Core).

Model Capabilities

Image Models

You can create DeepInfra image models using the .image() factory method. For more on image generation with the AI SDK see generateImage().

import { deepinfra } from '@ai-sdk/deepinfra';
import { experimental_generateImage as generateImage } from 'ai';

const { image } = await generateImage({
  model: deepinfra.image('stabilityai/sd3.5'),
  prompt: 'A futuristic cityscape at sunset',
  aspectRatio: '16:9',
});

Model-specific options

You can pass model-specific parameters using the providerOptions.deepinfra field:

import { deepinfra } from '@ai-sdk/deepinfra';
import { experimental_generateImage as generateImage } from 'ai';

const { image } = await generateImage({
  model: deepinfra.image('stabilityai/sd3.5'),
  prompt: 'A futuristic cityscape at sunset',
  aspectRatio: '16:9',
  providerOptions: {
    deepinfra: {
      num_inference_steps: 30, // Control the number of denoising steps (1-50)
    },
  },
});

Model Capabilities

For models supporting aspect ratios, the following ratios are typically supported: 1:1 (default), 16:9, 1:9, 3:2, 2:3, 4:5, 5:4, 9:16, 9:21

For models supporting size parameters, dimensions must typically be:

• Multiples of 32
• Width and height between 256 and 1440 pixels
• Default size is 1024x1024

For more details and pricing information, see the DeepInfra text-to-image models page.

Embedding Models

You can create DeepInfra embedding models using the .textEmbedding() factory method. For more on embedding models with the AI SDK see embed().

import { deepinfra } from '@ai-sdk/deepinfra';
import { embed } from 'ai';

const { embedding } = await embed({
  model: deepinfra.textEmbedding('BAAI/bge-large-en-v1.5'),
  value: 'sunny day at the beach',
});

Model Capabilities