Web Search Agent
Learn how to build an agent that has access to web with the AI SDK and Node
There are two approaches you can take to building a web search agent with the AI SDK:
- Use a model that has native web-searching capabilities
- Use a tool to access the web and return search results.
Both approaches have their advantages and disadvantages. Models with native search capabilities tend to be faster and there is no additional cost to make the search. The disadvantage is that you have less control over what is being searched, and the functionality is limited to models that support it.
instead, by using a tool, you can achieve more flexibility and greater control over your search queries. It allows you to customize your search strategy, specify search parameters, and you can use it with any LLM that supports tool calling. This approach will incur additional costs for the search API you use, but gives you complete control over the search experience.
Using native web-search
There are several models that offer native web-searching capabilities (Perplexity, OpenAI, Gemini). Let's look at how you could build a Web Search Agent across providers.
OpenAI Responses API
OpenAI's Responses API has a built-in web search tool that can be used to search the web and return search results. This tool is called web_search and is accessed via the openai provider.
Perplexity
Perplexity's Sonar models combines real-time web search with natural language processing. Each response is grounded in current web data and includes detailed citations.
Gemini
With compatible Gemini models, you can enable search grounding to give the model access to the latest information using Google search.
Using tools
When using tools for web search, you have two options: use ready-made tools that integrate directly with the AI SDK, or build custom tools tailored to your specific needs.
Unlike the native web search examples where searching is built into the model, using web search tools requires multiple steps. The language model will make two generations - the first to call the relevant web search tool (extracting search queries from the context), and the second to process the results and generate a response. This multi-step process is handled automatically when you set stopWhen: stepCountIs(n) to a value greater than 1.
By using stopWhen, you can automatically send tool results back to the
language model alongside the original question, enabling the model to respond
with information relevant to the user's query based on the search results.
This creates a seamless experience where the agent can search the web and
incorporate those findings into its response.
Use ready-made tools
If you prefer a ready-to-use web search tool without building one from scratch, there are several options that integrate directly with the AI SDK.
Exa
Get your API key from the Exa Dashboard.
First, install the Exa webSearch tool:
Then, you can import and pass it into generateText, streamText, or your agent:
For more configuration options and customization, see the Exa AI SDK documentation.
Parallel Web
First, install the Parallel Web AI SDK tools:
Then, you can import and pass the tools into generateText, streamText, or your agent. Parallel Web provides two tools: searchTool for web search and extractTool for extracting web page content:
Perplexity Search
Get your API key from the Perplexity API Keys page.
First, install the Perplexity Search tool:
Then, you can import and pass it into generateText, streamText, or your agent. Perplexity Search provides real-time web search with advanced filtering options including domain, language, date range, and recency filters:
For more configuration options and customization, see the Perplexity Search API documentation.
Tavily
Get your API key from the Tavily Dashboard.
First, install the tavilySearch tool:
Then, you can import and pass it into generateText, streamText, or your agent:
For more customization options over your agent's web-access functionality, visit the Tavily AI SDK Documentation.
Build and use custom tools
For more control over your web search functionality, you can build custom tools using web scraping and crawling APIs. This approach allows you to customize search parameters, handle specific data formats, and integrate with specialized search services.
Exa
Let's look at how you could implement a search tool using Exa:
Firecrawl
Firecrawl provides an API for web scraping and crawling. Let's look at how you can build a custom scraping tool using Firecrawl: