--- title: Groq description: Learn how to use Groq with AI Gateway and the direct provider. url: "https://ai-sdk.dev/providers/ai-sdk-providers/groq" docs_index: /llms.txt --- > For an index of all documentation, see [/llms.txt](/llms.txt). The [Groq](https://groq.com/) provider contains language model support for the Groq API. Start with [AI Gateway](/providers/ai-sdk-providers/ai-gateway) for models in its catalog (recommended), or use `@ai-sdk/groq` to connect directly to Groq. ```ts provider="groq" model="openai/gpt-oss-120b" import { generateText } from 'ai'; const { text } = await generateText({ model: "openai/gpt-oss-120b", prompt: 'Write a vegetarian lasagna recipe for 4 people.', providerOptions: { gateway: { order: ['groq'], }, }, }); console.log(text); ``` ## Using AI Gateway The AI SDK uses AI Gateway by default for model strings. Gateway provides access to models across providers through a single API key, with usage and cost tracking. Install `ai`: ```bash pnpm add ai ``` Set an [AI Gateway API key](https://vercel.com/docs/ai-gateway/getting-started) in your environment: ```bash export AI_GATEWAY_API_KEY=your_api_key_here ``` Vercel deployments can also use [OIDC authentication](/providers/ai-sdk-providers/ai-gateway#oidc-authentication-vercel-deployments). See the [AI Gateway provider documentation](/providers/ai-sdk-providers/ai-gateway) for configuration and the [model catalog](https://vercel.com/ai-gateway/models) for available models. The example sets `providerOptions.gateway.order` to `['groq']`. This sends the request to Groq first, then falls back to other available providers of the same model if Groq is unavailable or the request fails. `order` sets a preference; it does not restrict fallback to Groq. See [provider ordering](https://vercel.com/docs/ai-gateway/models-and-providers/provider-filtering-and-ordering). The dropdown lists models served by Groq through Gateway with a corresponding direct-provider example. Provider availability depends on the model and your team's [Provider Allowlist](https://vercel.com/docs/ai-gateway/models-and-providers/provider-filtering-and-ordering). ## Using `@ai-sdk/groq` For direct access to Groq, install `ai` and `@ai-sdk/groq`: ```bash pnpm add ai @ai-sdk/groq ``` Set your Groq API credential in the `GROQ_API_KEY` environment variable: ```bash export GROQ_API_KEY=your_api_key_here ``` AI Gateway does not expose every API in `@ai-sdk/groq`. Use the direct provider for provider-specific APIs or custom connection settings. The following sections document the direct `@ai-sdk/groq` provider, including its model factories and settings. Gateway model and feature availability can differ. ## Provider Instance You can import the default provider instance `groq` from `@ai-sdk/groq`: ```ts import { groq } from '@ai-sdk/groq'; ``` If you need a customized setup, you can import `createGroq` from `@ai-sdk/groq` and create a provider instance with your settings: ```ts import { createGroq } from '@ai-sdk/groq'; const groq = createGroq({ // custom settings }); ``` You can use the following optional settings to customize the Groq provider instance: - **baseURL** *string* Use a different URL prefix for API calls, e.g. to use proxy servers. The default prefix is `https://api.groq.com/openai/v1`. - **apiKey** *string | PromiseLike\ | (() => string | PromiseLike\)* API key that is being sent using the `Authorization` header. It defaults to the `GROQ_API_KEY` environment variable. A function is called before each request and may return a promise. Use it to supply refreshed credentials; a promise value resolves to the same key each time. - **headers** *Record\* Custom headers to include in the requests. - **fetch** *(input: RequestInfo, init?: RequestInit) => Promise\* Custom [fetch](https://developer.mozilla.org/en-US/docs/Web/API/fetch) implementation. Defaults to the global `fetch` function. You can use it as a middleware to intercept requests, or to provide a custom fetch implementation for e.g. testing. ## Language Models You can create [Groq models](https://console.groq.com/docs/models) using a provider instance. The first argument is the model id, e.g. `llama-3.1-8b-instant`. ```ts const model = groq('llama-3.1-8b-instant'); ``` ### Reasoning Models Groq offers reasoning models such as `qwen/qwen3.6-27b` and `openai/gpt-oss-120b`. You can configure how the reasoning is exposed in the generated text by using the `reasoningFormat` option. It supports the options `parsed`, `hidden`, and `raw`. ```ts import { groq, type GroqLanguageModelChatOptions } from '@ai-sdk/groq'; import { generateText } from 'ai'; const result = await generateText({ model: groq('qwen/qwen3.6-27b'), providerOptions: { groq: { reasoningFormat: 'parsed', reasoningEffort: 'default', parallelToolCalls: true, // Enable parallel function calling (default: true) user: 'user-123', // Unique identifier for end-user (optional) serviceTier: 'flex', // Use flex tier for higher throughput (optional) } satisfies GroqLanguageModelChatOptions, }, prompt: 'How many "r"s are in the word "strawberry"?', }); ``` The following optional provider options are available for Groq language models: - **reasoningFormat** *'parsed' | 'raw' | 'hidden'* Controls how reasoning is exposed in the generated text. Supported by reasoning models such as `qwen/qwen3.6-27b`; available formats depend on the model. For a complete list of reasoning models and their capabilities, see [Groq's reasoning models documentation](https://console.groq.com/docs/reasoning). - **reasoningEffort** *'low' | 'medium' | 'high' | 'none' | 'default'* Controls the level of effort the model will put into reasoning. - `qwen/qwen3.6-27b` - Supported values: - `none`: Disable reasoning. The model will not use any reasoning tokens. - `default`: Enable reasoning. - `openai/gpt-oss-20b` and `openai/gpt-oss-120b` - Supported values: - `low`: Use a low level of reasoning effort. - `medium`: Use a medium level of reasoning effort. - `high`: Use a high level of reasoning effort. Defaults to `default` for `qwen/qwen3.6-27b`. - **structuredOutputs** *boolean* Whether to use structured outputs. Defaults to `true`. When enabled, object generation will use the `json_schema` format instead of `json_object` format, providing more reliable structured outputs. - **strictJsonSchema** *boolean* Whether to use strict JSON schema validation. When `true`, the model uses constrained decoding to guarantee schema compliance. Defaults to `true`. Only used when `structuredOutputs` is enabled and a schema is provided. See [Groq's Structured Outputs documentation](https://console.groq.com/docs/structured-outputs) for details on strict mode limitations. - **parallelToolCalls** *boolean* Whether to enable parallel function calling during tool use. Defaults to `true`. - **user** *string* A unique identifier representing your end-user, which can help with monitoring and abuse detection. - **serviceTier** *'on\_demand' | 'performance' | 'flex' | 'auto'* Service tier for the request. Defaults to `'on_demand'`. - `'on_demand'`: Default tier with consistent performance and fairness - `'performance'`: Prioritized tier for latency-sensitive workloads - `'flex'`: Higher throughput tier (10x rate limits) optimized for workloads that can handle occasional request failures - `'auto'`: Uses on\_demand rate limits first, then falls back to flex tier if exceeded For more details about service tiers and their benefits, see [Groq's service tiers documentation](https://console.groq.com/docs/service-tiers). Only Groq reasoning models support the `reasoningFormat` option. #### Structured Outputs Structured outputs are enabled by default for Groq models. You can disable them by setting the `structuredOutputs` option to `false`. ```ts import { groq } from '@ai-sdk/groq'; import { generateText, Output } from 'ai'; import { z } from 'zod'; const result = await generateText({ model: groq('moonshotai/kimi-k2-instruct-0905'), output: Output.object({ schema: z.object({ recipe: z.object({ name: z.string(), ingredients: z.array(z.string()), instructions: z.array(z.string()), }), }), }), prompt: 'Generate a simple pasta recipe.', }); console.log(JSON.stringify(result.output, null, 2)); ``` You can disable structured outputs for models that don't support them: ```ts {9} import { groq, type GroqLanguageModelChatOptions } from '@ai-sdk/groq'; import { generateText, Output } from 'ai'; import { z } from 'zod'; const result = await generateText({ model: groq('llama-3.1-8b-instant'), providerOptions: { groq: { structuredOutputs: false, } satisfies GroqLanguageModelChatOptions, }, output: Output.object({ schema: z.object({ recipe: z.object({ name: z.string(), ingredients: z.array(z.string()), instructions: z.array(z.string()), }), }), }), prompt: 'Generate a simple pasta recipe in JSON format.', }); console.log(JSON.stringify(result.output, null, 2)); ``` Structured outputs are only supported by newer Groq models like `moonshotai/kimi-k2-instruct-0905`. For unsupported models, you can disable structured outputs by setting `structuredOutputs: false`. When disabled, Groq uses the `json_object` format which requires the word "JSON" to be included in your messages. ### Example You can use Groq language models to generate text with the `generateText` function: ```ts import { groq } from '@ai-sdk/groq'; import { generateText } from 'ai'; const { text } = await generateText({ model: groq('llama-3.1-8b-instant'), prompt: 'Write a vegetarian lasagna recipe for 4 people.', }); ``` ### Image Input Groq's multi-modal models like `meta-llama/llama-4-scout-17b-16e-instruct` support image inputs. You can include images in your messages using either URLs or base64-encoded data: ```ts import { groq } from '@ai-sdk/groq'; import { generateText } from 'ai'; const { text } = await generateText({ model: groq('meta-llama/llama-4-scout-17b-16e-instruct'), messages: [ { role: 'user', content: [ { type: 'text', text: 'What do you see in this image?' }, { type: 'file', mediaType: 'image', data: 'https://example.com/image.jpg', }, ], }, ], }); ``` You can also use base64-encoded images: ```ts import { groq } from '@ai-sdk/groq'; import { generateText } from 'ai'; import { readFileSync } from 'fs'; const imageData = readFileSync('path/to/image.jpg', 'base64'); const { text } = await generateText({ model: groq('meta-llama/llama-4-scout-17b-16e-instruct'), messages: [ { role: 'user', content: [ { type: 'text', text: 'Describe this image in detail.' }, { type: 'file', mediaType: 'image', data: `data:image/jpeg;base64,${imageData}`, }, ], }, ], }); ``` ## Model Capabilities | Model | Image Input | Object Generation | Tool Usage | Tool Streaming | Browser Search | | ----------------------------------------------- | ----------- | ----------------- | ---------- | -------------- | -------------- | | `gemma2-9b-it` | ✗ | ✓ | ✓ | ✓ | ✗ | | `llama-3.1-8b-instant` | ✗ | ✓ | ✓ | ✓ | ✗ | | `llama-3.3-70b-versatile` | ✗ | ✓ | ✓ | ✓ | ✗ | | `meta-llama/llama-guard-4-12b` | ✓ | ✓ | ✗ | ✗ | ✗ | | `deepseek-r1-distill-llama-70b` | ✗ | ✓ | ✓ | ✓ | ✗ | | `meta-llama/llama-4-maverick-17b-128e-instruct` | ✓ | ✓ | ✓ | ✓ | ✗ | | `meta-llama/llama-4-scout-17b-16e-instruct` | ✓ | ✓ | ✓ | ✓ | ✗ | | `meta-llama/llama-prompt-guard-2-22m` | ✗ | ✓ | ✗ | ✗ | ✗ | | `meta-llama/llama-prompt-guard-2-86m` | ✗ | ✓ | ✗ | ✗ | ✗ | | `moonshotai/kimi-k2-instruct-0905` | ✗ | ✓ | ✓ | ✓ | ✗ | | `qwen/qwen3.6-27b` | ✗ | ✓ | ✓ | ✓ | ✗ | | `llama-guard-3-8b` | ✗ | ✓ | ✓ | ✓ | ✗ | | `llama3-70b-8192` | ✗ | ✓ | ✓ | ✓ | ✗ | | `llama3-8b-8192` | ✗ | ✓ | ✓ | ✓ | ✗ | | `mixtral-8x7b-32768` | ✗ | ✓ | ✓ | ✓ | ✗ | | `qwen-qwq-32b` | ✗ | ✓ | ✓ | ✓ | ✗ | | `qwen-2.5-32b` | ✗ | ✓ | ✓ | ✓ | ✗ | | `deepseek-r1-distill-qwen-32b` | ✗ | ✓ | ✓ | ✓ | ✗ | | `openai/gpt-oss-20b` | ✗ | ✓ | ✓ | ✓ | ✓ | | `openai/gpt-oss-120b` | ✗ | ✓ | ✓ | ✓ | ✓ | The tables above list the most commonly used models. Please see the [Groq docs](https://console.groq.com/docs/models) for a complete list of available models. You can also pass any available provider model ID as a string if needed. ## Browser Search Tool Groq provides a browser search tool that offers interactive web browsing capabilities. Unlike traditional web search, browser search navigates websites interactively, providing more detailed and comprehensive results. ### Supported Models Browser search is available on Groq's GPT-OSS models, for example: - `openai/gpt-oss-20b` - `openai/gpt-oss-120b` Known model IDs without browser search support (such as the Llama, Qwen, and Kimi models) will generate a warning and the tool will be ignored. Unknown model IDs are forwarded with the tool, and the Groq API decides whether it is supported. ### Basic Usage ```ts import { groq } from '@ai-sdk/groq'; import { generateText } from 'ai'; const result = await generateText({ model: groq('openai/gpt-oss-120b'), // Must use supported model prompt: 'What are the latest developments in AI? Please search for recent news.', tools: { browser_search: groq.tools.browserSearch({}), }, toolChoice: 'required', // Ensure the tool is used }); console.log(result.text); ``` ### Streaming Example ```ts import { groq } from '@ai-sdk/groq'; import { streamText } from 'ai'; const result = streamText({ model: groq('openai/gpt-oss-120b'), prompt: 'Search for the latest tech news and summarize it.', tools: { browser_search: groq.tools.browserSearch({}), }, toolChoice: 'required', }); for await (const delta of result.stream) { if (delta.type === 'text-delta') { process.stdout.write(delta.text); } } ``` ### Key Features - **Interactive Browsing**: Navigates websites like a human user - **Comprehensive Results**: More detailed than traditional search snippets - **Server-side Execution**: Runs on Groq's infrastructure, no setup required - **Powered by Exa**: Uses Exa search engine for optimal results - **Currently Free**: Available at no additional charge during beta ### Best Practices - Use `toolChoice: 'required'` to ensure the browser search is activated - Use a model that supports browser search, such as `openai/gpt-oss-20b` or `openai/gpt-oss-120b` - The tool works automatically - no configuration parameters needed - Server-side execution means no additional API keys or setup required ### Model Validation The provider ignores the tool for known model IDs without browser search support: ```ts // ✅ Supported - will work const result = await generateText({ model: groq('openai/gpt-oss-120b'), tools: { browser_search: groq.tools.browserSearch({}) }, }); // ❌ Unsupported - will show warning and ignore tool const result = await generateText({ model: groq('llama-3.1-8b-instant'), tools: { browser_search: groq.tools.browserSearch({}) }, }); // Warning: "Browser search is not supported on model llama-3.1-8b-instant." ``` For more details about browser search capabilities and limitations, see the [Groq Browser Search Documentation](https://console.groq.com/docs/browser-search). ## Transcription Models You can create models that call the [Groq transcription API](https://console.groq.com/docs/speech-to-text) using the `.transcription()` factory method. The first argument is the model id e.g. `whisper-large-v3`. ```ts const model = groq.transcription('whisper-large-v3'); ``` You can also pass additional provider-specific options using the `providerOptions` argument. For example, supplying the input language in ISO-639-1 (e.g. `en`) format will improve accuracy and latency. ```ts {6} import { transcribe } from 'ai'; import { groq, type GroqTranscriptionModelOptions } from '@ai-sdk/groq'; import { readFile } from 'fs/promises'; const result = await transcribe({ model: groq.transcription('whisper-large-v3'), audio: await readFile('audio.mp3'), providerOptions: { groq: { language: 'en' } satisfies GroqTranscriptionModelOptions, }, }); ``` For audio hosted at an HTTP(S) URL, pass a `URL` to let Groq fetch the audio directly: ```ts import { transcribe } from 'ai'; import { groq } from '@ai-sdk/groq'; const result = await transcribe({ model: groq.transcription('whisper-large-v3-turbo'), audio: new URL('https://example.com/large-audio.mp3'), }); ``` The SDK sends Groq's `url` parameter without downloading or uploading the audio locally. This avoids the attachment size limit; Groq's overall audio size and duration limits still apply. The URL must be accessible to Groq. If Groq rejects a redirect, the SDK falls back to its secure local download and file upload; the attachment limit then applies. Request headers configure the Groq API request and are not sent to the audio host. If you supply a custom `download` function, the SDK downloads the audio with that function and uploads the resulting bytes instead. The following provider options are available: - **timestampGranularities** *string\[]* The granularity of the timestamps in the transcription. Defaults to `['segment']`. Possible values are `['word']`, `['segment']`, and `['word', 'segment']`. Note: There is no additional latency for segment timestamps, but generating word timestamps incurs additional latency. **Important:** Requires `responseFormat` to be set to `'verbose_json'`. - **responseFormat** *string* The format of the response. Set to `'verbose_json'` to receive timestamps for audio segments and enable `timestampGranularities`. Set to `'text'` to return only the transcribed text. Optional. - **language** *string* The language of the input audio. Supplying the input language in ISO-639-1 format (e.g. 'en') will improve accuracy and latency. Optional. - **prompt** *string* An optional text to guide the model's style or continue a previous audio segment. The prompt should match the audio language. Optional. - **temperature** *number* The sampling temperature, between 0 and 1. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic. If set to 0, the model will use log probability to automatically increase the temperature until certain thresholds are hit. Defaults to 0. Optional. ### Model Capabilities | Model | Transcription | Streaming | Duration | Segments | Language | | ------------------------ | ------------- | --------- | ------------------------------------------------------------- | ------------------------------------------------------------- | ------------------------------------------------------------- | | `whisper-large-v3` | ✓ | ✗ | Conditional: Returned when responseFormat is 'verbose\_json'. | Conditional: Returned when responseFormat is 'verbose\_json'. | Conditional: Returned when responseFormat is 'verbose\_json'. | | `whisper-large-v3-turbo` | ✓ | ✗ | Conditional: Returned when responseFormat is 'verbose\_json'. | Conditional: Returned when responseFormat is 'verbose\_json'. | Conditional: Returned when responseFormat is 'verbose\_json'. | --- For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md) For an index of all available documentation, see [/llms.txt](/llms.txt) For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)