---
title: Baseten
description: Learn how to use Baseten with AI Gateway and the direct provider.
url: "https://ai-sdk.dev/providers/ai-sdk-providers/baseten"
docs_index: /llms.txt
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

[Baseten](https://baseten.co/) is an inference platform for serving frontier, enterprise-grade opensource AI models via their [API](https://docs.baseten.co/overview).
Start with [AI Gateway](/providers/ai-sdk-providers/ai-gateway) for models in its catalog (recommended), or use `@ai-sdk/baseten` to connect directly to Baseten.

```ts provider="baseten" model="openai/gpt-oss-120b"
import { generateText } from 'ai';

const { text } = await generateText({
  model: "openai/gpt-oss-120b",
  prompt: 'Write a vegetarian lasagna recipe for 4 people.',
  providerOptions: {
    gateway: {
      order: ['baseten'],
    },
  },
});

console.log(text);
```

## Using AI Gateway

The AI SDK uses AI Gateway by default for model strings. Gateway provides access to models across providers through a single API key, with usage and cost tracking. Install `ai`:

```bash
pnpm add ai
```

Set an [AI Gateway API key](https://vercel.com/docs/ai-gateway/getting-started) in your environment:

```bash
export AI_GATEWAY_API_KEY=your_api_key_here
```

Vercel deployments can also use [OIDC authentication](/providers/ai-sdk-providers/ai-gateway#oidc-authentication-vercel-deployments).
See the [AI Gateway provider documentation](/providers/ai-sdk-providers/ai-gateway) for configuration and the [model catalog](https://vercel.com/ai-gateway/models) for available models.

The example sets `providerOptions.gateway.order` to `['baseten']`. This sends the request to Baseten first, then falls back to other available providers of the same model if Baseten is unavailable or the request fails. `order` sets a preference; it does not restrict fallback to Baseten. See [provider ordering](https://vercel.com/docs/ai-gateway/models-and-providers/provider-filtering-and-ordering).

The dropdown lists models served by Baseten through Gateway with a corresponding direct-provider example. Provider availability depends on the model and your team's [Provider Allowlist](https://vercel.com/docs/ai-gateway/models-and-providers/provider-filtering-and-ordering).

## Using `@ai-sdk/baseten`

For direct access to Baseten, install `ai` and `@ai-sdk/baseten`:

```bash
pnpm add ai @ai-sdk/baseten
```

Set your Baseten API credential in the `BASETEN_API_KEY` environment variable:

```bash
export BASETEN_API_KEY=your_api_key_here
```

AI Gateway does not expose every API in `@ai-sdk/baseten`. Use the direct provider for provider-specific APIs or custom connection settings.

The following sections document the direct `@ai-sdk/baseten` provider,
including its model factories and settings. Gateway model and feature
availability can differ.

## Provider Instance

You can import the default provider instance `baseten` from `@ai-sdk/baseten`:

```ts
import { baseten } from '@ai-sdk/baseten';
```

If you need a customized setup, you can import `createBaseten` from `@ai-sdk/baseten`
and create a provider instance with your settings:

```ts
import { createBaseten } from '@ai-sdk/baseten';

const baseten = createBaseten({
  apiKey: process.env.BASETEN_API_KEY ?? '',
});
```

You can use the following optional settings to customize the Baseten provider instance:

- **baseURL** *string*

  Use a different URL prefix for API calls, e.g. to use proxy servers.
  The default prefix is `https://inference.baseten.co/v1`.

- **apiKey** *string*

  API key that is being sent using the `Authorization` header. It defaults to
  the `BASETEN_API_KEY` environment variable. It is recommended you set the environment variable using `export` so you do not need to include the field every time.
  You can grab your Baseten API Key [here](https://app.baseten.co/settings/api_keys)

- **modelURL** *string*

  Custom model URL for specific models (chat or embeddings). If not provided,
  the default Model APIs will be used.

- **headers** *Record\<string,string>*

  Custom headers to include in the requests.

- **performanceClient** *PerformanceClient constructor*

  Opt in to Baseten's native performance client for embeddings, for client-side
  batching and request hedging. Pass the `PerformanceClient` constructor from
  `@basetenlabs/performance-client`, which you install yourself. When omitted,
  embeddings use plain HTTP. See
  [Native performance client](#native-performance-client-optional).

- **fetch** *(input: RequestInfo, init?: RequestInit) => Promise\<Response>*

  Custom [fetch](https://developer.mozilla.org/en-US/docs/Web/API/fetch) implementation.

## Model APIs

You can select [Baseten models](https://www.baseten.co/products/model-apis/) using a provider instance.
The first argument is the model id, e.g. `'moonshotai/Kimi-K2-Instruct-0905'`: The complete supported models under Model APIs can be found [here](https://docs.baseten.co/development/model-apis/overview#supported-models).

```ts
const model = baseten('moonshotai/Kimi-K2-Instruct-0905');
```

### Example

You can use Baseten language models to generate text with the `generateText` function:

```ts
import { baseten } from '@ai-sdk/baseten';
import { generateText } from 'ai';

const { text } = await generateText({
  model: baseten('moonshotai/Kimi-K2-Instruct-0905'),
  prompt: 'What is the meaning of life? Answer in one sentence.',
});
```

Baseten language models can also be used in the `streamText` function
(see [AI SDK Core](/docs/ai-sdk-core)).

## Dedicated Models

Baseten supports dedicated model URLs for both chat and embedding models. You have to specify a `modelURL` when creating the provider:

### OpenAI-Compatible Endpoints (`/sync/v1`)

For models deployed with Baseten's OpenAI-compatible endpoints:

```ts
import { createBaseten } from '@ai-sdk/baseten';

const baseten = createBaseten({
  modelURL: 'https://model-{MODEL_ID}.api.baseten.co/sync/v1',
});
// No modelId is needed because we specified modelURL
const model = baseten();
const { text } = await generateText({
  model: model,
  prompt: 'Say hello from a Baseten chat model!',
});
```

### `/predict` Endpoints

`/predict` endpoints are currently NOT supported for chat models. You must use `/sync/v1` endpoints for chat functionality.

## Embedding Models

You can create models that call the Baseten embeddings API using the `.embeddingModel()` factory method. Baseten Embeddings Inference deployments are OpenAI-compatible, so embeddings use plain HTTP by default with no extra dependencies.

**Important:** Embedding models require a dedicated deployment with a custom
`modelURL`. Unlike chat models, embeddings cannot use Baseten's default Model
APIs and must specify a dedicated model endpoint.

```ts
import { createBaseten } from '@ai-sdk/baseten';
import { embed, embedMany } from 'ai';

const baseten = createBaseten({
  modelURL: 'https://model-{MODEL_ID}.api.baseten.co/sync',
});

const embeddingModel = baseten.embeddingModel();

// Single embedding
const { embedding } = await embed({
  model: embeddingModel,
  value: 'sunny day at the beach',
});

// Batch embeddings
const { embeddings } = await embedMany({
  model: embeddingModel,
  values: [
    'sunny day at the beach',
    'rainy afternoon in the city',
    'snowy mountain peak',
  ],
});
```

Each request sends at most 128 values. `embedMany` splits larger inputs into
chunks of that size and runs them in parallel, so you can pass as many values as
you like.

### Endpoint Support for Embeddings

**Supported:**

- `/sync` endpoints (`/v1/embeddings` is appended for you)
- `/sync/v1` endpoints

**Not Supported:**

- `/predict` endpoints

### Native performance client (optional)

Baseten also publishes `@basetenlabs/performance-client`, a native client that
adds client-side batching and request hedging on top of the server-side dynamic
batching your deployment already does. It is **not** installed by default: it is
a native addon, so it cannot load in edge runtimes and bundlers cannot resolve
its platform binaries.

To use it, install it yourself and pass the constructor:

```bash
npm i @basetenlabs/performance-client
```

```ts
import { createBaseten } from '@ai-sdk/baseten';
import { PerformanceClient } from '@basetenlabs/performance-client';

const baseten = createBaseten({
  modelURL:
    'https://model-{MODEL_ID}.api.baseten.co/environments/production/sync',
  performanceClient: PerformanceClient,
});
```

When you opt in, the client handles batching itself, so values are sent in a
single call rather than being split at 128.

## Error Handling

The Baseten provider includes built-in error handling for common API errors:

```ts
import { baseten } from '@ai-sdk/baseten';
import { generateText } from 'ai';

try {
  const { text } = await generateText({
    model: baseten('moonshotai/Kimi-K2-Instruct-0905'),
    prompt: 'Hello, world!',
  });
} catch (error) {
  console.error('Baseten API error:', error.message);
}
```

### Common Error Scenarios

```ts
// Embeddings require a modelURL
try {
  baseten.embeddingModel();
} catch (error) {
  // Error: "No model URL provided for embeddings. Please set modelURL option for embeddings."
}

// /predict endpoints are not supported for chat models
try {
  const baseten = createBaseten({
    modelURL:
      'https://model-{MODEL_ID}.api.baseten.co/environments/production/predict',
  });
  baseten(); // This will throw an error
} catch (error) {
  // Error: "Not supported. You must use a /sync/v1 endpoint for chat models."
}

// /sync/v1 endpoints are now supported for embeddings
const baseten = createBaseten({
  modelURL:
    'https://model-{MODEL_ID}.api.baseten.co/environments/production/sync/v1',
});
const embeddingModel = baseten.embeddingModel(); // This works fine!

// /predict endpoints are not supported for embeddings
try {
  const baseten = createBaseten({
    modelURL:
      'https://model-{MODEL_ID}.api.baseten.co/environments/production/predict',
  });
  baseten.embeddingModel(); // This will throw an error
} catch (error) {
  // Error: "Not supported. You must use a /sync or /sync/v1 endpoint for embeddings."
}

// Image models are not supported
try {
  baseten.imageModel('test-model');
} catch (error) {
  // Error: NoSuchModelError for imageModel
}
```

For more information about Baseten models and deployment options, see the
[Baseten documentation](https://docs.baseten.co/).

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)