`experimental_streamTranslate()`

experimental_streamTranslate is an experimental feature.

Streams a speech-to-speech translation from live raw audio. Models translate live source audio into target-language audio and text.

experimental_streamTranslate is built on the speech translation model specification (Experimental_SpeechTranslationModelV4). Provider implementations of the specification ship separately — see your provider's documentation for available translation models.

import { experimental_streamTranslate as streamTranslate } from 'ai';

const result = streamTranslate({
  // any provider model instance that implements the experimental
  // speech translation model specification (Experimental_SpeechTranslationModelV4):
  model: translationModel,
  audio: audioStream, // ReadableStream<Uint8Array | string>
  inputAudioFormat: { type: 'audio/pcm', rate: 24000 },
  targetLanguage: 'es',
});

for await (const part of result.fullStream) {
  if (part.type === 'output-text-delta') {
    process.stdout.write(part.delta);
  }
}

console.log(await result.translationText);

Import

import { experimental_streamTranslate as streamTranslate } from "ai"

API Signature

Parameters

model:

Experimental_SpeechTranslationModelV4

audio:

ReadableStream<Uint8Array | string>

inputAudioFormat:

{ type: string; rate?: number }

targetLanguage:

string

sourceLanguage?:

string

outputAudioFormat?:

{ type: string; rate?: number }

providerOptions?:

Record<string, JSONObject>

abortSignal?:

AbortSignal

headers?:

Record<string, string>

includeRawChunks?:

boolean

Returns

fullStream:

AsyncIterableStream<TranslationStreamPart>

sourceText:

Promise<string>

translationText:

Promise<string>

durationInSeconds:

Promise<number | undefined>

usage:

Promise<Experimental_SpeechTranslationModelV4Usage | undefined>

warnings:

Promise<Warning[]>

response:

Promise<SpeechTranslationModelResponseMetadata>

providerMetadata:

Promise<Record<string, JSONObject>>

The result promises settle as the stream is consumed. If you stop consuming fullStream early (e.g. break out of the loop), the underlying provider connection is closed and pending result promises reject.