# Baseten Provider

[Baseten](https://baseten.co/) is an inference platform for serving frontier, enterprise-grade opensource AI models via their [API](https://docs.baseten.co/overview).

## Setup

The Baseten provider is available via the `@ai-sdk/baseten` module. You can install it with

<Tabs items={['pnpm', 'npm', 'yarn']}>
  <Tab>
    <Snippet text="pnpm add @ai-sdk/baseten" dark />
  </Tab>
  <Tab>
    <Snippet text="npm install @ai-sdk/baseten" dark />
  </Tab>
  <Tab>
    <Snippet text="yarn add @ai-sdk/baseten" dark />
  </Tab>
</Tabs>

## Provider Instance

You can import the default provider instance `baseten` from `@ai-sdk/baseten`:

```ts
import { baseten } from '@ai-sdk/baseten';
```

If you need a customized setup, you can import `createBaseten` from `@ai-sdk/baseten`
and create a provider instance with your settings:

```ts
import { createBaseten } from '@ai-sdk/baseten';

const baseten = createBaseten({
  apiKey: process.env.BASETEN_API_KEY ?? '',
});
```

You can use the following optional settings to customize the Baseten provider instance:

- **baseURL** _string_

  Use a different URL prefix for API calls, e.g. to use proxy servers.
  The default prefix is `https://inference.baseten.co/v1`.

- **apiKey** _string_

  API key that is being sent using the `Authorization` header. It defaults to
  the `BASETEN_API_KEY` environment variable. It is recommended you set the environment variable using `export` so you do not need to include the field everytime.
  You can grab your Baseten API Key [here](https://app.baseten.co/settings/api_keys)

- **modelURL** _string_

  Custom model URL for specific models (chat or embeddings). If not provided,
  the default Model APIs will be used.

- **headers** _Record&lt;string,string&gt;_

  Custom headers to include in the requests.

- **fetch** _(input: RequestInfo, init?: RequestInit) => Promise&lt;Response&gt;_

  Custom [fetch](https://developer.mozilla.org/en-US/docs/Web/API/fetch) implementation.

## Model APIs

You can select [Baseten models](https://www.baseten.co/products/model-apis/) using a provider instance.
The first argument is the model id, e.g. `'moonshotai/Kimi-K2-Instruct-0905'`: The complete supported models under Model APIs can be found [here](https://docs.baseten.co/development/model-apis/overview#supported-models).

```ts
const model = baseten('moonshotai/Kimi-K2-Instruct-0905');
```

### Example

You can use Baseten language models to generate text with the `generateText` function:

```ts
import { baseten } from '@ai-sdk/baseten';
import { generateText } from 'ai';

const { text } = await generateText({
  model: baseten('moonshotai/Kimi-K2-Instruct-0905'),
  prompt: 'What is the meaning of life? Answer in one sentence.',
});
```

Baseten language models can also be used in the `streamText` function
(see [AI SDK Core](/docs/ai-sdk-core)).

## Dedicated Models

Baseten supports dedicated model URLs for both chat and embedding models. You have to specify a `modelURL` when creating the provider:

### OpenAI-Compatible Endpoints (`/sync/v1`)

For models deployed with Baseten's OpenAI-compatible endpoints:

```ts
import { createBaseten } from '@ai-sdk/baseten';

const baseten = createBaseten({
  modelURL: 'https://model-{MODEL_ID}.api.baseten.co/sync/v1',
});
// No modelId is needed because we specified modelURL
const model = baseten();
const { text } = await generateText({
  model: model,
  prompt: 'Say hello from a Baseten chat model!',
});
```

### `/predict` Endpoints

`/predict` endpoints are currently NOT supported for chat models. You must use `/sync/v1` endpoints for chat functionality.

## Embedding Models

You can create models that call the Baseten embeddings API using the `.textEmbeddingModel()` factory method. The Baseten provider uses the high-performance `@basetenlabs/performance-client` for optimal embedding performance.

<Note>
  **Important:** Embedding models require a dedicated deployment with a custom
  `modelURL`. Unlike chat models, embeddings cannot use Baseten's default Model
  APIs and must specify a dedicated model endpoint.
</Note>

```ts
import { createBaseten } from '@ai-sdk/baseten';
import { embed, embedMany } from 'ai';

const baseten = createBaseten({
  modelURL: 'https://model-{MODEL_ID}.api.baseten.co/sync',
});

const embeddingModel = baseten.textEmbeddingModel();

// Single embedding
const { embedding } = await embed({
  model: embeddingModel,
  value: 'sunny day at the beach',
});

// Batch embeddings
const { embeddings } = await embedMany({
  model: embeddingModel,
  values: [
    'sunny day at the beach',
    'rainy afternoon in the city',
    'snowy mountain peak',
  ],
});
```

### Endpoint Support for Embeddings

**Supported:**

- `/sync` endpoints (Performance Client automatically adds `/v1/embeddings`)
- `/sync/v1` endpoints (automatically strips `/v1` before passing to Performance Client)

**Not Supported:**

- `/predict` endpoints (not compatible with Performance Client)

### Performance Features

The embedding implementation includes:

- **High-performance client**: Uses `@basetenlabs/performance-client` for optimal performance
- **Automatic batching**: Efficiently handles multiple texts in a single request
- **Connection reuse**: Performance Client is created once and reused for all requests
- **Built-in retries**: Automatic retry logic for failed requests

## Error Handling

The Baseten provider includes built-in error handling for common API errors:

```ts
import { baseten } from '@ai-sdk/baseten';
import { generateText } from 'ai';

try {
  const { text } = await generateText({
    model: baseten('moonshotai/Kimi-K2-Instruct-0905'),
    prompt: 'Hello, world!',
  });
} catch (error) {
  console.error('Baseten API error:', error.message);
}
```

### Common Error Scenarios

```ts
// Embeddings require a modelURL
try {
  baseten.textEmbeddingModel();
} catch (error) {
  // Error: "No model URL provided for embeddings. Please set modelURL option for embeddings."
}

// /predict endpoints are not supported for chat models
try {
  const baseten = createBaseten({
    modelURL:
      'https://model-{MODEL_ID}.api.baseten.co/environments/production/predict',
  });
  baseten(); // This will throw an error
} catch (error) {
  // Error: "Not supported. You must use a /sync/v1 endpoint for chat models."
}

// /sync/v1 endpoints are now supported for embeddings
const baseten = createBaseten({
  modelURL:
    'https://model-{MODEL_ID}.api.baseten.co/environments/production/sync/v1',
});
const embeddingModel = baseten.textEmbeddingModel(); // This works fine!

// /predict endpoints are not supported for embeddings
try {
  const baseten = createBaseten({
    modelURL:
      'https://model-{MODEL_ID}.api.baseten.co/environments/production/predict',
  });
  baseten.textEmbeddingModel(); // This will throw an error
} catch (error) {
  // Error: "Not supported. You must use a /sync or /sync/v1 endpoint for embeddings."
}

// Image models are not supported
try {
  baseten.imageModel('test-model');
} catch (error) {
  // Error: NoSuchModelError for imageModel
}
```

<Note>
  For more information about Baseten models and deployment options, see the
  [Baseten documentation](https://docs.baseten.co/).
</Note>


## Navigation

- [AI Gateway](/v5/providers/ai-sdk-providers/ai-gateway)
- [xAI Grok](/v5/providers/ai-sdk-providers/xai)
- [Vercel](/v5/providers/ai-sdk-providers/vercel)
- [OpenAI](/v5/providers/ai-sdk-providers/openai)
- [Azure OpenAI](/v5/providers/ai-sdk-providers/azure)
- [Anthropic](/v5/providers/ai-sdk-providers/anthropic)
- [Amazon Bedrock](/v5/providers/ai-sdk-providers/amazon-bedrock)
- [Groq](/v5/providers/ai-sdk-providers/groq)
- [Fal](/v5/providers/ai-sdk-providers/fal)
- [AssemblyAI](/v5/providers/ai-sdk-providers/assemblyai)
- [DeepInfra](/v5/providers/ai-sdk-providers/deepinfra)
- [Deepgram](/v5/providers/ai-sdk-providers/deepgram)
- [Black Forest Labs](/v5/providers/ai-sdk-providers/black-forest-labs)
- [Gladia](/v5/providers/ai-sdk-providers/gladia)
- [LMNT](/v5/providers/ai-sdk-providers/lmnt)
- [Google Generative AI](/v5/providers/ai-sdk-providers/google-generative-ai)
- [Hume](/v5/providers/ai-sdk-providers/hume)
- [Google Vertex AI](/v5/providers/ai-sdk-providers/google-vertex)
- [Rev.ai](/v5/providers/ai-sdk-providers/revai)
- [Baseten](/v5/providers/ai-sdk-providers/baseten)
- [Hugging Face](/v5/providers/ai-sdk-providers/huggingface)
- [Mistral AI](/v5/providers/ai-sdk-providers/mistral)
- [Together.ai](/v5/providers/ai-sdk-providers/togetherai)
- [Cohere](/v5/providers/ai-sdk-providers/cohere)
- [Fireworks](/v5/providers/ai-sdk-providers/fireworks)
- [DeepSeek](/v5/providers/ai-sdk-providers/deepseek)
- [Moonshot AI](/v5/providers/ai-sdk-providers/moonshotai)
- [Alibaba](/v5/providers/ai-sdk-providers/alibaba)
- [Cerebras](/v5/providers/ai-sdk-providers/cerebras)
- [Replicate](/v5/providers/ai-sdk-providers/replicate)
- [Perplexity](/v5/providers/ai-sdk-providers/perplexity)
- [Luma](/v5/providers/ai-sdk-providers/luma)
- [ElevenLabs](/v5/providers/ai-sdk-providers/elevenlabs)


[Full Sitemap](/sitemap.md)