# Writing a Custom Provider

The AI SDK provides a [Language Model Specification](https://github.com/vercel/ai/tree/main/packages/provider/src/language-model/v1).
You can write your own provider that adheres to the specification and it will be compatible with the AI SDK.

You can find the Language Model Specification in the [AI SDK repository](https://github.com/vercel/ai/tree/main/packages/provider/src/language-model/v1).
It can be imported from `'@ai-sdk/provider'`. We also provide utilities that make it easier to implement a custom provider. You can find them in the `@ai-sdk/provider-utils` package ([source code](https://github.com/vercel/ai/tree/main/packages/provider-utils)).

<Note>
  If you open-source a provider, we'd love to promote it here. Please send us a
  PR to add it to the [Community Providers](/providers/community-providers)
  section.
</Note>

## Provider Implementation Guide

Implementing a custom language model provider involves several steps:

- Creating an entry point
- Adding a language model implementation
- Mapping the input (prompt, tools, settings)
- Processing the results (generate, streaming, tool calls)
- Supporting object generation

The best way to get started is to copy a reference implementation and modify it to fit your needs.
Check out the [Mistral reference implementation](https://github.com/vercel/ai/tree/main/packages/mistral)
to see how the project is structured, and feel free to copy the setup.

### Creating an Entry Point

Each AI SDK provider should follow the pattern of using a factory function that returns a provider instance
and provide a default instance.

```ts filename="custom-provider.ts"
import {
  generateId,
  loadApiKey,
  withoutTrailingSlash,
} from '@ai-sdk/provider-utils';
import { CustomChatLanguageModel } from './custom-chat-language-model';
import { CustomChatModelId, CustomChatSettings } from './custom-chat-settings';

// model factory function with additional methods and properties
export interface CustomProvider {
  (
    modelId: CustomChatModelId,
    settings?: CustomChatSettings,
  ): CustomChatLanguageModel;

  // explicit method for targeting a specific API in case there are several
  chat(
    modelId: CustomChatModelId,
    settings?: CustomChatSettings,
  ): CustomChatLanguageModel;
}

// optional settings for the provider
export interface CustomProviderSettings {
  /**
Use a different URL prefix for API calls, e.g. to use proxy servers.
   */
  baseURL?: string;

  /**
API key.
   */
  apiKey?: string;

  /**
Custom headers to include in the requests.
     */
  headers?: Record<string, string>;
}

// provider factory function
export function createCustomProvider(
  options: CustomProviderSettings = {},
): CustomProvider {
  const createModel = (
    modelId: CustomChatModelId,
    settings: CustomChatSettings = {},
  ) =>
    new CustomChatLanguageModel(modelId, settings, {
      provider: 'custom.chat',
      baseURL:
        withoutTrailingSlash(options.baseURL) ?? 'https://custom.ai/api/v1',
      headers: () => ({
        Authorization: `Bearer ${loadApiKey({
          apiKey: options.apiKey,
          environmentVariableName: 'CUSTOM_API_KEY',
          description: 'Custom Provider',
        })}`,
        ...options.headers,
      }),
      generateId: options.generateId ?? generateId,
    });

  const provider = function (
    modelId: CustomChatModelId,
    settings?: CustomChatSettings,
  ) {
    if (new.target) {
      throw new Error(
        'The model factory function cannot be called with the new keyword.',
      );
    }

    return createModel(modelId, settings);
  };

  provider.chat = createModel;

  return provider;
}

/**
 * Default custom provider instance.
 */
export const customProvider = createCustomProvider();
```

### Implementing the Language Model

A [language model](https://github.com/vercel/ai/blob/main/packages/provider/src/language-model/v1/language-model-v1.ts) needs to implement:

- metadata fields
  - `specificationVersion: 'v1'` - always `'v1'`
  - `provider: string` - name of the provider
  - `modelId: string` - unique identifier of the model
  - `defaultObjectGenerationMode` - default object generation mode, e.g. "json"
- `doGenerate` method
- `doStream` method

Check out the [Mistral language model](https://github.com/vercel/ai/blob/main/packages/mistral/src/mistral-chat-language-model.ts) as an example.

At a high level, both `doGenerate` and `doStream` methods should:

1. **Map the prompt and the settings to the format required by the provider API.** This can be extracted, e.g. the Mistral provider contains a `getArgs` method.
2. **Call the provider API.** You could e.g. use fetch calls or a library offered by the provider.
3. **Process the results.** You need to convert the response to the format required by the AI SDK.

### Errors

The AI SDK provides [standardized errors](https://github.com/vercel/ai/tree/main/packages/provider/src/errors) that should be used by providers where possible. This will make it easy for user to debug them.

### Retries, timeouts, and abort signals

The AI SDK will handle retries, timeouts, and aborting requests in a unified way. The model classes should not implement retries or timeouts themselves. Instead, they should use the `abortSignal` parameter to determine when the call should be aborted, and they should throw `ApiCallErrors` (or similar) with a correct `isRetryable` flag when errors such as network errors occur.


## Navigation

- [Writing a Custom Provider](/v4/providers/community-providers/custom-providers)
- [Qwen](/v4/providers/community-providers/qwen)
- [Ollama](/v4/providers/community-providers/ollama)
- [Chrome AI](/v4/providers/community-providers/chrome-ai)
- [FriendliAI](/v4/providers/community-providers/friendliai)
- [Portkey](/v4/providers/community-providers/portkey)
- [Cloudflare Workers AI](/v4/providers/community-providers/cloudflare-workers-ai)
- [Cloudflare AI Gateway](/v4/providers/community-providers/cloudflare-ai-gateway)
- [OpenRouter](/v4/providers/community-providers/openrouter)
- [Azure AI](/v4/providers/community-providers/azure-ai)
- [Crosshatch](/v4/providers/community-providers/crosshatch)
- [Requesty](/v4/providers/community-providers/requesty)
- [Mixedbread](/v4/providers/community-providers/mixedbread)
- [Voyage AI](/v4/providers/community-providers/voyage-ai)
- [Hyperbolic](/v4/providers/community-providers/hyperbolic)
- [Mem0](/v4/providers/community-providers/mem0)
- [Letta](/v4/providers/community-providers/letta)
- [LLamaCpp](/v4/providers/community-providers/llama-cpp)
- [Anthropic Vertex](/v4/providers/community-providers/anthropic-vertex-ai)
- [Spark](/v4/providers/community-providers/spark)
- [Inflection AI](/v4/providers/community-providers/inflection-ai)
- [LangDB](/v4/providers/community-providers/langdb)
- [Zhipu AI](/v4/providers/community-providers/zhipu)
- [SambaNova](/v4/providers/community-providers/sambanova)
- [Dify](/v4/providers/community-providers/dify)
- [Sarvam](/v4/providers/community-providers/sarvam)


[Full Sitemap](/sitemap.md)