--- title: Together.ai description: Learn how to use Together.ai with AI Gateway and the direct provider. url: "https://ai-sdk.dev/providers/ai-sdk-providers/togetherai" docs_index: /llms.txt --- > For an index of all documentation, see [/llms.txt](/llms.txt). The [Together.ai](https://together.ai) provider contains support for 200+ open-source models through the [Together.ai API](https://docs.together.ai/reference). Start with [AI Gateway](/providers/ai-sdk-providers/ai-gateway) for models in its catalog (recommended), or use `@ai-sdk/togetherai` to connect directly to Together.ai. ```ts provider="togetherai" model="deepseek/deepseek-v4-flash-0731" import { generateText } from 'ai'; const { text } = await generateText({ model: "deepseek/deepseek-v4-flash-0731", prompt: 'Write a vegetarian lasagna recipe for 4 people.', providerOptions: { gateway: { order: ['togetherai'], }, }, }); console.log(text); ``` ## Using AI Gateway The AI SDK uses AI Gateway by default for model strings. Gateway provides access to models across providers through a single API key, with usage and cost tracking. Install `ai`: ```bash pnpm add ai ``` Set an [AI Gateway API key](https://vercel.com/docs/ai-gateway/getting-started) in your environment: ```bash export AI_GATEWAY_API_KEY=your_api_key_here ``` Vercel deployments can also use [OIDC authentication](/providers/ai-sdk-providers/ai-gateway#oidc-authentication-vercel-deployments). See the [AI Gateway provider documentation](/providers/ai-sdk-providers/ai-gateway) for configuration and the [model catalog](https://vercel.com/ai-gateway/models) for available models. The example sets `providerOptions.gateway.order` to `['togetherai']`. This sends the request to Together.ai first, then falls back to other available providers of the same model if Together.ai is unavailable or the request fails. `order` sets a preference; it does not restrict fallback to Together.ai. See [provider ordering](https://vercel.com/docs/ai-gateway/models-and-providers/provider-filtering-and-ordering). The dropdown lists models served by Together.ai through Gateway with a corresponding direct-provider example. Provider availability depends on the model and your team's [Provider Allowlist](https://vercel.com/docs/ai-gateway/models-and-providers/provider-filtering-and-ordering). ## Using `@ai-sdk/togetherai` For direct access to Together.ai, install `ai` and `@ai-sdk/togetherai`: ```bash pnpm add ai @ai-sdk/togetherai ``` Set your Together.ai API credential in the `TOGETHER_API_KEY` environment variable: ```bash export TOGETHER_API_KEY=your_api_key_here ``` AI Gateway does not expose every API in `@ai-sdk/togetherai`. Use the direct provider for provider-specific APIs or custom connection settings. The following sections document the direct `@ai-sdk/togetherai` provider, including its model factories and settings. Gateway model and feature availability can differ. ## Provider Instance You can import the default provider instance `togetherai` from `@ai-sdk/togetherai`: ```ts import { togetherai } from '@ai-sdk/togetherai'; ``` If you need a customized setup, you can import `createTogetherAI` from `@ai-sdk/togetherai` and create a provider instance with your settings: ```ts import { createTogetherAI } from '@ai-sdk/togetherai'; const togetherai = createTogetherAI({ apiKey: process.env.TOGETHER_API_KEY ?? '', }); ``` You can use the following optional settings to customize the Together.ai provider instance: - **baseURL** *string* Use a different URL prefix for API calls, e.g. to use proxy servers. The default prefix is `https://api.together.xyz/v1`. - **apiKey** *string* API key that is being sent using the `Authorization` header. It defaults to the `TOGETHER_API_KEY` environment variable. - **headers** *Record\* Custom headers to include in the requests. - **fetch** *(input: RequestInfo, init?: RequestInit) => Promise\* Custom [fetch](https://developer.mozilla.org/en-US/docs/Web/API/fetch) implementation. Defaults to the global `fetch` function. You can use it as a middleware to intercept requests, or to provide a custom fetch implementation for e.g. testing. ## Language Models You can create [Together.ai models](https://docs.together.ai/docs/serverless-models) using a provider instance. The first argument is the model id, e.g. `google/gemma-2-9b-it`. ```ts const model = togetherai('google/gemma-2-9b-it'); ``` ### Reasoning Models Together.ai exposes the thinking of `deepseek-ai/DeepSeek-R1` in the generated text using the `` tag. You can use the `extractReasoningMiddleware` to extract this reasoning and expose it as a `reasoning` property on the result: ```ts import { togetherai } from '@ai-sdk/togetherai'; import { wrapLanguageModel, extractReasoningMiddleware } from 'ai'; const enhancedModel = wrapLanguageModel({ model: togetherai('deepseek-ai/DeepSeek-R1'), middleware: extractReasoningMiddleware({ tagName: 'think' }), }); ``` You can then use that enhanced model in functions like `generateText` and `streamText`. ### Example You can use Together.ai language models to generate text with the `generateText` function: ```ts import { togetherai } from '@ai-sdk/togetherai'; import { generateText } from 'ai'; const { text } = await generateText({ model: togetherai('meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo'), prompt: 'Write a vegetarian lasagna recipe for 4 people.', }); ``` Together.ai language models can also be used in the `streamText` function (see [AI SDK Core](/docs/ai-sdk-core)). The Together.ai provider also supports [completion models](https://docs.together.ai/docs/serverless-models#language-models) via (following the above example code) `togetherai.completionModel()` and [embedding models](https://docs.together.ai/docs/serverless-models#embedding-models) via `togetherai.embeddingModel()`. ## Model Capabilities | Model | Image Input | Object Generation | Tool Usage | Tool Streaming | | --------------------------------------------------- | ----------- | ----------------- | ---------- | -------------- | | `moonshotai/Kimi-K2.5` | ✓ | ✓ | ✓ | ✓ | | `Qwen/Qwen3.5-397B-A17B` | ✓ | ✓ | ✓ | ✓ | | `MiniMaxAI/MiniMax-M2.5` | ✗ | ✓ | ✓ | ✓ | | `zai-org/GLM-5` | ✗ | ✓ | ✓ | ✓ | | `deepseek-ai/DeepSeek-V3.1` | ✗ | ✓ | ✓ | ✓ | | `openai/gpt-oss-120b` | ✗ | ✓ | ✓ | ✓ | | `openai/gpt-oss-20b` | ✗ | ✓ | ✓ | ✓ | | `meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8` | ✓ | ✓ | ✓ | ✓ | The table above lists popular models. Please see the [Together.ai docs](https://docs.together.ai/docs/serverless-models) for a full list of available models. You can also pass any available provider model ID as a string if needed. ## Image Models You can create Together.ai image models using the `.image()` factory method. For more on image generation with the AI SDK see [generateImage()](/docs/reference/ai-sdk-core/generate-image). ```ts import { togetherai } from '@ai-sdk/togetherai'; import { generateImage } from 'ai'; const { images } = await generateImage({ model: togetherai.image('black-forest-labs/FLUX.1-dev'), prompt: 'A delighted resplendent quetzal mid flight amidst raindrops', }); ``` You can pass optional provider-specific request parameters using the `providerOptions` argument. ```ts import { togetherai, type TogetherAIImageModelOptions, } from '@ai-sdk/togetherai'; import { generateImage } from 'ai'; const { images } = await generateImage({ model: togetherai.image('black-forest-labs/FLUX.1-dev'), prompt: 'A delighted resplendent quetzal mid flight amidst raindrops', size: '512x512', // Optional additional provider-specific request parameters providerOptions: { togetherai: { steps: 40, } satisfies TogetherAIImageModelOptions, }, }); ``` The following provider options are available: - **steps** *number* Number of generation steps. Higher values can improve quality. - **guidance** *number* Guidance scale for image generation. - **negative\_prompt** *string* Negative prompt to guide what to avoid. - **disable\_safety\_checker** *boolean* Disable the safety checker for image generation. When true, the API will not reject images flagged as potentially NSFW. Not available for Flux Schnell Free and Flux Pro models. ### Image Editing Together AI supports image editing through FLUX Kontext models. Pass input images via `prompt.images` to transform or edit existing images. Together AI does not support mask-based inpainting. Instead, use descriptive prompts to specify what you want to change in the image. #### Basic Image Editing Transform an existing image using text prompts: ```ts const imageBuffer = readFileSync('./input-image.png'); const { images } = await generateImage({ model: togetherai.image('black-forest-labs/FLUX.1-kontext-pro'), prompt: { text: 'Turn the cat into a golden retriever dog', images: [imageBuffer], }, size: '1024x1024', providerOptions: { togetherai: { steps: 28, } satisfies TogetherAIImageModelOptions, }, }); ``` #### Editing with URL Reference You can also pass image URLs directly: ```ts const { images } = await generateImage({ model: togetherai.image('black-forest-labs/FLUX.1-kontext-pro'), prompt: { text: 'Make the background a lush rainforest', images: ['https://example.com/photo.png'], }, size: '1024x1024', providerOptions: { togetherai: { steps: 28, } satisfies TogetherAIImageModelOptions, }, }); ``` Input images can be provided as `Buffer`, `ArrayBuffer`, `Uint8Array`, base64-encoded strings, or URLs. Together AI only supports a single input image per request. #### Supported Image Editing Models | Model | Description | | -------------------------------------- | ------------------------------------------------------- | | `black-forest-labs/FLUX.1-kontext-pro` | Production quality, balanced speed | | `black-forest-labs/FLUX.1-kontext-max` | Maximum image fidelity | | `black-forest-labs/FLUX.1-kontext-dev` | Development and experimentation | | `black-forest-labs/FLUX.2-pro` | High fidelity single-image editing | | `black-forest-labs/FLUX.2-flex` | Single-image editing with adjustable steps and guidance | `supportsFileInputs` reports `true` for these models and `supportsMaskInputs` reports `false`. The provider sends the first standardized image input as `image_url`. FLUX.2 Dev and `google/gemini-3-pro-image` require `reference_images`, which this mapping does not send, so their standardized file-input capability is `false`. Unrecognized model IDs report `undefined` (unknown), including IDs under a known provider namespace. ### Model Capabilities Together.ai image models support various image dimensions that vary by model. Common sizes include 512x512, 768x768, and 1024x1024, with some models supporting up to 1792x1792. The default size is 1024x1024. | Available Models | | ------------------------------------------ | | `stabilityai/stable-diffusion-xl-base-1.0` | | `black-forest-labs/FLUX.1-dev` | | `black-forest-labs/FLUX.1-dev-lora` | | `black-forest-labs/FLUX.1-schnell` | | `black-forest-labs/FLUX.1-canny` | | `black-forest-labs/FLUX.1-depth` | | `black-forest-labs/FLUX.1-redux` | | `black-forest-labs/FLUX.1.1-pro` | | `black-forest-labs/FLUX.1-pro` | | `black-forest-labs/FLUX.1-schnell-Free` | | `black-forest-labs/FLUX.1-kontext-pro` | | `black-forest-labs/FLUX.1-kontext-max` | | `black-forest-labs/FLUX.1-kontext-dev` | Please see the [Together.ai models page](https://docs.together.ai/docs/serverless-models#image-models) for a full list of available image models and their capabilities. ## Embedding Models You can create Together.ai embedding models using the `.embeddingModel()` factory method. For more on embedding models with the AI SDK see [embed()](/docs/reference/ai-sdk-core/embed). ```ts import { togetherai } from '@ai-sdk/togetherai'; import { embed } from 'ai'; const { embedding } = await embed({ model: togetherai.embeddingModel('togethercomputer/m2-bert-80M-2k-retrieval'), value: 'sunny day at the beach', }); ``` ### Model Capabilities | Model | Dimensions | Max Tokens | | ----------------------------------------- | ---------- | ---------- | | `BAAI/bge-large-en-v1.5` | 1024 | 512 | | `Alibaba-NLP/gte-modernbert-base` | 768 | 8192 | | `intfloat/multilingual-e5-large-instruct` | 1024 | 514 | For a complete list of available embedding models, see the [Together.ai models page](https://docs.together.ai/docs/serverless-models#embedding-models). ## Reranking Models You can create Together.ai reranking models using the `.reranking()` factory method. For more on reranking with the AI SDK see [rerank()](/docs/reference/ai-sdk-core/rerank). ```ts import { togetherai } from '@ai-sdk/togetherai'; import { rerank } from 'ai'; const documents = [ 'sunny day at the beach', 'rainy afternoon in the city', 'snowy night in the mountains', ]; const { ranking } = await rerank({ model: togetherai.reranking('mixedbread-ai/Mxbai-Rerank-Large-V2'), documents, query: 'talk about rain', topN: 2, }); console.log(ranking); // [ // { originalIndex: 1, score: 0.9, document: 'rainy afternoon in the city' }, // { originalIndex: 0, score: 0.3, document: 'sunny day at the beach' } // ] ``` Together.ai reranking models support additional provider options for object documents. You can specify which fields to use for ranking: ```ts import { togetherai, type TogetherAIRerankingModelOptions, } from '@ai-sdk/togetherai'; import { rerank } from 'ai'; const documents = [ { from: 'Paul Doe', subject: 'Follow-up', text: 'We are happy to give you a discount of 20%.', }, { from: 'John McGill', subject: 'Missing Info', text: 'Here is the pricing from Oracle: $5000/month', }, ]; const { ranking } = await rerank({ model: togetherai.reranking('mixedbread-ai/Mxbai-Rerank-Large-V2'), documents, query: 'Which pricing did we get from Oracle?', providerOptions: { togetherai: { rankFields: ['from', 'subject', 'text'], // Specify which fields to rank by } satisfies TogetherAIRerankingModelOptions, }, }); ``` The following provider options are available: - **rankFields** *string\[]* Array of field names to use for ranking when documents are JSON objects. If not specified, all fields are used. ### Model Capabilities | Model | | ------------------------------------- | | `mixedbread-ai/Mxbai-Rerank-Large-V2` | --- For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md) For an index of all available documentation, see [/llms.txt](/llms.txt) For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)