# Baseten Provider

[Baseten](https://baseten.co/) is a platform for running and testing LLMs.
It allows you to deploy models that are OpenAI API compatible that you can use with the AI SDK.

## Setup

The Baseten provider is available via the `@ai-sdk/openai-compatible` module as it is compatible with the OpenAI API.
You can install it with

<Tabs items={['pnpm', 'npm', 'yarn']}>
  <Tab>
    <Snippet text="pnpm add @ai-sdk/openai-compatible" dark />
  </Tab>
  <Tab>
    <Snippet text="npm install @ai-sdk/openai-compatible" dark />
  </Tab>
  <Tab>
    <Snippet text="yarn add @ai-sdk/openai-compatible" dark />
  </Tab>
</Tabs>

## Provider Instance

To use Baseten, you can create a custom provider instance with the `createOpenAICompatible` function from `@ai-sdk/openai-compatible`:

```ts
import { createOpenAICompatible } from '@ai-sdk/openai-compatible';

const BASETEN_MODEL_ID = '<model-id>'; // e.g. 5q3z8xcw
const BASETEN_MODEL_URL = `https://model-${BASETEN_MODEL_ID}.api.baseten.co/environments/production/sync/v1`;

const baseten = createOpenAICompatible({
  name: 'baseten',
  baseURL: BASETEN_MODEL_URL,
  headers: {
    Authorization: `Bearer ${process.env.BASETEN_API_KEY ?? ''}`,
  },
});
```

Be sure to have your `BASETEN_API_KEY` set in your environment and the model `<model-id>` ready. The `<model-id>` will be given after you have deployed the model on Baseten.

## Language Models

You can create [Baseten models](https://www.baseten.co/library/) using a provider instance.
The first argument is the served model name, e.g. `llama`.

```ts
const model = baseten('llama');
```

### Example

You can use Baseten language models to generate text with the `generateText` function:

```ts
import { createOpenAICompatible } from '@ai-sdk/openai-compatible';
import { generateText } from 'ai';

const BASETEN_MODEL_ID = '<model-id>'; // e.g. 5q3z8xcw
const BASETEN_MODEL_URL = `https://model-${BASETEN_MODEL_ID}.api.baseten.co/environments/production/sync/v1`;

const baseten = createOpenAICompatible({
  name: 'baseten',
  baseURL: BASETEN_MODEL_URL,
  headers: {
    Authorization: `Bearer ${process.env.BASETEN_API_KEY ?? ''}`,
  },
});

const { text } = await generateText({
  model: baseten('llama'),
  prompt: 'Tell me about yourself in one sentence',
});

console.log(text);
```

Baseten language models are also able to generate text in a streaming fashion with the `streamText` function:

```ts
import { createOpenAICompatible } from '@ai-sdk/openai-compatible';
import { streamText } from 'ai';

const BASETEN_MODEL_ID = '<model-id>'; // e.g. 5q3z8xcw
const BASETEN_MODEL_URL = `https://model-${BASETEN_MODEL_ID}.api.baseten.co/environments/production/sync/v1`;

const baseten = createOpenAICompatible({
  name: 'baseten',
  baseURL: BASETEN_MODEL_URL,
  headers: {
    Authorization: `Bearer ${process.env.BASETEN_API_KEY ?? ''}`,
  },
});

const result = streamText({
  model: baseten('llama'),
  prompt: 'Tell me about yourself in one sentence',
});

for await (const message of result.textStream) {
  console.log(message);
}
```

Baseten language models can also be used in the `generateObject`, and `streamObject` functions.


## Navigation

- [Writing a Custom Provider](/v4/providers/openai-compatible-providers/custom-providers)
- [LM Studio](/v4/providers/openai-compatible-providers/lmstudio)
- [NVIDIA NIM](/v4/providers/openai-compatible-providers/nim)
- [Baseten](/v4/providers/openai-compatible-providers/baseten)


[Full Sitemap](/sitemap.md)