# Cloudflare AI Gateway

The Cloudflare AI Gateway Provider is a library that integrates Cloudflare's AI Gateway with the Vercel AI SDK. It enables seamless access to multiple AI models from various providers through a unified interface, with automatic fallback for high availability.

## Features

- **Runtime Agnostic**: Compatible with Node.js, Edge Runtime, and other JavaScript runtimes supported by the Vercel AI SDK.
- **Automatic Fallback**: Automatically switches to the next available model if one fails, ensuring resilience.
- **Multi-Provider Support**: Supports models from OpenAI, Anthropic, DeepSeek, Google AI Studio, Grok, Mistral, Perplexity AI, Replicate, and Groq.
- **Cloudflare AI Gateway Integration**: Leverages Cloudflare's AI Gateway for request management, caching, and rate limiting.
- **Simplified Configuration**: Easy setup with support for API key authentication or Cloudflare AI bindings.

## Setup

The Cloudflare AI Gateway Provider is available in the `ai-gateway-provider` module. Install it with:

<Tabs items={['pnpm', 'npm', 'yarn', 'bun']}>
  <Tab>
    <Snippet text="pnpm add ai-gateway-provider" />
  </Tab>
  <Tab>
    <Snippet text="npm install ai-gateway-provider" />
  </Tab>
  <Tab>
    <Snippet text="yarn add ai-gateway-provider" />
  </Tab>

  <Tab>
    <Snippet text="bun add ai-gateway-provider" dark />
  </Tab>
</Tabs>

## Provider Instance

Create an `aigateway` provider instance using the `createAiGateway` function. You can authenticate using an API key or a Cloudflare AI binding.

### API Key Authentication

```typescript
import { createAiGateway } from 'ai-gateway-provider';

const aigateway = createAiGateway({
  accountId: 'your-cloudflare-account-id',
  gateway: 'your-gateway-name',
  apiKey: 'your-cloudflare-api-key', // Only required if your gateway has authentication enabled
  options: {
    skipCache: true, // Optional request-level settings
  },
});
```

### Cloudflare AI Binding

This method is only available inside Cloudflare Workers.

Configure an AI binding in your `wrangler.toml`:

```bash
[AI]
binding = "AI"
```

In your worker, create a new instance using the binding:

```typescript
import { createAiGateway } from 'ai-gateway-provider';

const aigateway = createAiGateway({
  binding: env.AI.gateway('my-gateway'),
  options: {
    skipCache: true, // Optional request-level settings
  },
});
```

## Language Models

Create a model instance by passing an array of models to the `aigateway` provider. The provider will attempt to use the models in order, falling back to the next if one fails.

```typescript
import { createAiGateway } from 'ai-gateway-provider';
import { createOpenAI } from '@ai-sdk/openai';
import { createAnthropic } from '@ai-sdk/anthropic';

const aigateway = createAiGateway({
  accountId: 'your-cloudflare-account-id',
  gateway: 'your-gateway-name',
  apiKey: 'your-cloudflare-api-key',
});

const openai = createOpenAI({ apiKey: 'openai-api-key' });
const anthropic = createAnthropic({ apiKey: 'anthropic-api-key' });

const model = aigateway([
  anthropic('claude-3-5-haiku-20241022'), // Primary model
  openai('gpt-4o-mini'), // Fallback model
]);
```

### Request Options

Customize Cloudflare AI Gateway settings per request:

- `cacheKey`: Custom cache key for the request.
- `cacheTtl`: Cache time-to-live in seconds.
- `skipCache`: Bypass caching.
- `metadata`: Custom metadata for the request.
- `collectLog`: Enable/disable log collection.
- `eventId`: Custom event identifier.
- `requestTimeoutMs`: Request timeout in milliseconds.
- `retries`:
  - `maxAttempts`: Number of retry attempts (1-5).
  - `retryDelayMs`: Delay between retries.
  - `backoff`: Retry strategy (`constant`, `linear`, `exponential`).

Example:

```typescript
const aigateway = createAiGateway({
  accountId: 'your-cloudflare-account-id',
  gateway: 'your-gateway-name',
  apiKey: 'your-cloudflare-api-key',
  options: {
    cacheTtl: 3600, // Cache for 1 hour
    metadata: { userId: 'user123' },
    retries: {
      maxAttempts: 3,
      retryDelayMs: 1000,
      backoff: 'exponential',
    },
  },
});
```

## Examples

### `generateText`

Generate non-streaming text using the Cloudflare AI Gateway Provider:

```typescript
import { createAiGateway } from 'ai-gateway-provider';
import { createOpenAI } from '@ai-sdk/openai';
import { generateText } from 'ai';

const aigateway = createAiGateway({
  accountId: 'your-cloudflare-account-id',
  gateway: 'your-gateway-name',
  apiKey: 'your-cloudflare-api-key',
});

const openai = createOpenAI({ apiKey: 'openai-api-key' });

const { text } = await generateText({
  model: aigateway([openai('gpt-4o-mini')]),
  prompt: 'Write a greeting.',
});

console.log(text); // Output: "Hello"
```

### `streamText`

Stream text responses using the Cloudflare AI Gateway Provider:

```typescript
import { createAiGateway } from 'ai-gateway-provider';
import { createOpenAI } from '@ai-sdk/openai';
import { streamText } from 'ai';

const aigateway = createAiGateway({
  accountId: 'your-cloudflare-account-id',
  gateway: 'your-gateway-name',
  apiKey: 'your-cloudflare-api-key',
});

const openai = createOpenAI({ apiKey: 'openai-api-key' });

const result = await streamText({
  model: aigateway([openai('gpt-4o-mini')]),
  prompt: 'Write a multi-part greeting.',
});

let accumulatedText = '';
for await (const chunk of result.textStream) {
  accumulatedText += chunk;
}

console.log(accumulatedText); // Output: "Hello world!"
```

## Supported Providers

- OpenAI
- Anthropic
- DeepSeek
- Google AI Studio
- Grok
- Mistral
- Perplexity AI
- Replicate
- Groq

## Error Handling

The provider throws the following custom errors:

- `AiGatewayUnauthorizedError`: Invalid or missing API key when authentication is enabled.
- `AiGatewayDoesNotExist`: Specified Cloudflare AI Gateway does not exist.


## Navigation

- [Writing a Custom Provider](/v5/providers/community-providers/custom-providers)
- [Qwen](/v5/providers/community-providers/qwen)
- [Ollama](/v5/providers/community-providers/ollama)
- [A2A](/v5/providers/community-providers/a2a)
- [ACP (Agent Client Protocol)](/v5/providers/community-providers/acp)
- [Helicone](/v5/providers/community-providers/helicone)
- [FriendliAI](/v5/providers/community-providers/friendliai)
- [Portkey](/v5/providers/community-providers/portkey)
- [Built-in AI](/v5/providers/community-providers/built-in-ai)
- [Gemini CLI](/v5/providers/community-providers/gemini-cli)
- [MCP Sampling AI Provider](/v5/providers/community-providers/mcp-sampling)
- [Automatic1111](/v5/providers/community-providers/automatic1111)
- [Cloudflare Workers AI](/v5/providers/community-providers/cloudflare-workers-ai)
- [Cloudflare AI Gateway](/v5/providers/community-providers/cloudflare-ai-gateway)
- [OpenRouter](/v5/providers/community-providers/openrouter)
- [Azure AI](/v5/providers/community-providers/azure-ai)
- [Aihubmix](/v5/providers/community-providers/aihubmix)
- [SAP AI Core](/v5/providers/community-providers/sap-ai)
- [Crosshatch](/v5/providers/community-providers/crosshatch)
- [Requesty](/v5/providers/community-providers/requesty)
- [MiniMax](/v5/providers/community-providers/minimax)
- [Mixedbread](/v5/providers/community-providers/mixedbread)
- [Voyage AI](/v5/providers/community-providers/voyage-ai)
- [Jina AI](/v5/providers/community-providers/jina-ai)
- [Mem0](/v5/providers/community-providers/mem0)
- [Letta](/v5/providers/community-providers/letta)
- [Supermemory](/v5/providers/community-providers/supermemory)
- [React Native Apple](/v5/providers/community-providers/react-native-apple)
- [Anthropic Vertex](/v5/providers/community-providers/anthropic-vertex-ai)
- [Spark](/v5/providers/community-providers/spark)
- [Inflection AI](/v5/providers/community-providers/inflection-ai)
- [LangDB](/v5/providers/community-providers/langdb)
- [Zhipu AI](/v5/providers/community-providers/zhipu)
- [SambaNova](/v5/providers/community-providers/sambanova)
- [Dify](/v5/providers/community-providers/dify)
- [Sarvam](/v5/providers/community-providers/sarvam)
- [AI/ML API](/v5/providers/community-providers/aimlapi)
- [Claude Code](/v5/providers/community-providers/claude-code)


[Full Sitemap](/sitemap.md)