--- title: Cartesia description: Learn how to use the Cartesia provider for the AI SDK. url: "https://ai-sdk.dev/providers/ai-sdk-providers/cartesia" docs_index: /llms.txt --- > For an index of all documentation, see [/llms.txt](/llms.txt). The [Cartesia](https://cartesia.ai/) provider contains Sonic speech generation, Ink-Whisper batch transcription, and Ink 2 realtime and streaming transcription support. ## Setup The Cartesia provider is available in the `@ai-sdk/cartesia` module. You can install it with #### pnpm ``` pnpm add @ai-sdk/cartesia ``` #### npm ``` npm install @ai-sdk/cartesia ``` #### yarn ``` yarn add @ai-sdk/cartesia ``` #### bun ``` bun add @ai-sdk/cartesia ``` ## Provider Instance You can import the default provider instance `cartesia` from `@ai-sdk/cartesia`: ```ts import { cartesia } from '@ai-sdk/cartesia'; ``` If you need a customized setup, you can import `createCartesia` from `@ai-sdk/cartesia` and create a provider instance with your settings: ```ts import { createCartesia } from '@ai-sdk/cartesia'; const cartesia = createCartesia({ // custom settings, e.g. fetch: customFetch, }); ``` You can use the following optional settings to customize the Cartesia provider instance: - **apiKey** *string* API key that is being sent using the `Authorization` header. It defaults to the `CARTESIA_API_KEY` environment variable. - **version** *string* The Cartesia API version to use (sent via the `Cartesia-Version` header). - **headers** *Record\* Custom headers to include in the requests. - **fetch** *(input: RequestInfo, init?: RequestInit) => Promise\* Custom [fetch](https://developer.mozilla.org/en-US/docs/Web/API/fetch) implementation. Defaults to the global `fetch` function. You can use it as a middleware to intercept requests, or to provide a custom fetch implementation for e.g. testing. - **webSocket** *WebSocketConstructor* Custom WebSocket implementation for Ink 2 streaming transcription. Defaults to the global `WebSocket` constructor. ## Speech Models You can create models that call the [Cartesia text-to-speech API](https://docs.cartesia.ai/api-reference/tts/bytes) using the `.speech()` factory method. The first argument is the model id, e.g. `sonic-3.5`. ```ts const model = cartesia.speech('sonic-3.5'); ``` You can use the model with the `generateSpeech` function. Cartesia requires a `voice` id: ```ts import { generateSpeech } from 'ai'; import { cartesia } from '@ai-sdk/cartesia'; const result = await generateSpeech({ model: cartesia.speech('sonic-3.5'), text: 'Hello, world!', voice: '694f9389-aac1-45b6-b726-9d9369183238', }); ``` You can also pass additional provider-specific options using the `providerOptions` argument: ```ts {7-12} import { generateSpeech } from 'ai'; import { cartesia, type CartesiaSpeechModelOptions } from '@ai-sdk/cartesia'; const result = await generateSpeech({ model: cartesia.speech('sonic-3.5'), text: 'Hello, world!', voice: '694f9389-aac1-45b6-b726-9d9369183238', providerOptions: { cartesia: { container: 'wav', encoding: 'pcm_s16le', sampleRate: 24000, } satisfies CartesiaSpeechModelOptions, }, }); ``` The following provider options are available: - **container** *string* Container format for the output audio. Supported values: `'raw'`, `'wav'`, `'mp3'`. Optional. - **encoding** *string* Encoding type for the audio output. Supported values: `'pcm_f32le'`, `'pcm_s16le'`, `'pcm_mulaw'`, `'pcm_alaw'`. Optional. - **sampleRate** *number* Sample rate for the output audio in Hz (e.g. `8000`, `16000`, `22050`, `24000`, `44100`, `48000`). Optional. - **bitRate** *number* Bitrate for `mp3` output in bits per second (e.g. `32000`, `64000`, `128000`, `192000`). Optional. - **speed** *number* Controls the speed of the generated speech between `0.6` and `1.5`. Optional. - **language** *string* The language to generate speech in (ISO 639-1 code). Optional. ### Model Capabilities | Model | | -------------- | | `sonic-3.5` | | `sonic-3` | | `sonic-2` | | `sonic-turbo` | | `sonic-latest` | ## Realtime Transcription Models Realtime is an experimental feature. You can create a model for Cartesia's realtime [Ink 2](https://docs.cartesia.ai/build-with-cartesia/stt/latest) API using the `.experimental_realtime()` factory method: ```ts import { cartesia } from '@ai-sdk/cartesia'; const model = cartesia.experimental_realtime('ink-2'); ``` Realtime sessions run in the browser and require a short-lived access token created on your server with `cartesia.experimental_realtime.getToken()`: ```ts const token = await cartesia.experimental_realtime.getToken({ model: 'ink-2', sessionConfig: { inputAudioFormat: { type: 'audio/pcm', rate: 24000 }, inputAudioTranscription: { language: 'en' }, turnDetection: { type: 'server-vad' }, }, }); ``` Ink 2 produces input transcription events and supports English audio. Turn detection is enabled by default. To use manual finalization instead, set `turnDetection` to `null` in the session config and call `commitAudio()` when the current input is complete. See [Realtime](/docs/ai-sdk-core/realtime) for the complete browser session setup. ### Model Capabilities | Model | Streaming Transcription | Turn Detection | | ------- | ----------------------- | -------------- | | `ink-2` | ✓ | ✓ | ## Streaming Transcription Models Streaming transcription is an experimental feature. Ink 2 is also available through the transcription streaming API. Create the model with `.transcription()` and pass it to `experimental_streamTranscribe`: ```ts import { cartesia } from '@ai-sdk/cartesia'; import { experimental_streamTranscribe as streamTranscribe } from 'ai'; const result = streamTranscribe({ model: cartesia.transcription('ink-2'), audio, // ReadableStream containing raw audio chunks inputAudioFormat: { type: 'audio/pcm', rate: 24000 }, providerOptions: { cartesia: { language: 'en', }, }, }); for await (const part of result.fullStream) { if (part.type === 'transcript-partial') { console.log('partial:', part.text); } if (part.type === 'transcript-final') { console.log('final:', part.text); } } ``` The provider creates a short-lived, STT-scoped Cartesia access token before opening the WebSocket, so the API key is not included in the WebSocket URL. Send audio chunks at approximately their playback rate; Cartesia's realtime endpoint is intended for live audio rather than bulk file upload. Ink 2 supports English audio and uses Cartesia's native turn detection by default. To finalize only when the input audio stream ends, pass `providerOptions.cartesia.streaming.turnDetection: false`. By default, the provider maps `audio/pcm`, `audio/pcmu`, and `audio/pcma` to 16-bit PCM, G.711 μ-law, and G.711 A-law respectively. To stream another raw PCM representation, set `providerOptions.cartesia.streaming.encoding` to `pcm_s32le`, `pcm_f16le`, or `pcm_f32le`. For example: ```ts const result = streamTranscribe({ model: cartesia.transcription('ink-2'), audio, inputAudioFormat: { type: 'audio/pcm', rate: 48000 }, providerOptions: { cartesia: { streaming: { encoding: 'pcm_f32le', }, }, }, }); ``` See [Transcription](/docs/ai-sdk-core/transcription) for more information about streaming transcription. ### Model Capabilities | Model | Streaming Transcription | Turn Detection | | ------- | ----------------------- | -------------- | | `ink-2` | ✓ | ✓ | ## Batch Transcription Models You can create models that call the [Cartesia transcription API](https://docs.cartesia.ai/api-reference/stt/transcribe) using the `.transcription()` factory method. The first argument is the model id e.g. `ink-whisper`. ```ts const model = cartesia.transcription('ink-whisper'); ``` You can use the model with the `transcribe` function: ```ts import { transcribe } from 'ai'; import { cartesia } from '@ai-sdk/cartesia'; import { readFile } from 'fs/promises'; const result = await transcribe({ model: cartesia.transcription('ink-whisper'), audio: await readFile('audio.mp3'), }); ``` You can also pass additional provider-specific options using the `providerOptions` argument: ```ts {11-15} import { transcribe } from 'ai'; import { cartesia, type CartesiaTranscriptionModelOptions, } from '@ai-sdk/cartesia'; import { readFile } from 'fs/promises'; const result = await transcribe({ model: cartesia.transcription('ink-whisper'), audio: await readFile('audio.mp3'), providerOptions: { cartesia: { language: 'en', } satisfies CartesiaTranscriptionModelOptions, }, }); ``` The following provider options are available: - **language** *string* The language of the audio (ISO 639-1 code). If not specified, it defaults to English. Optional. - **timestampGranularities** *array of strings* The timestamp granularities to populate. Currently only `'word'` is supported. Optional. ### Model Capabilities | Model | Transcription | Duration | Segments | Language | | ------------- | ------------- | -------- | -------- | -------- | | `ink-whisper` | ✓ | ✓ | ✓ | ✓ | --- For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md) For an index of all available documentation, see [/llms.txt](/llms.txt) For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)