← All posts

2026-10-22 · Telem.ai Team

Research

Even the Best Search Provider Isn't Enough

At Telem.ai, we want every search your agent makes to find the best answer on the web, whichever provider holds it. You shouldn't have to track which provider is good at what, because we take that part off your hands. Your agent shouldn't miss an answer because it asked the wrong web search provider.

Let's start with a simple question: is one good provider enough? Better yet, is the best provider enough? We tested a dozen providers on the same agent searches and counted what each one found. The web is wider than any one provider, and even the best one sees only part of it.

How we tested

We ran a series of experiments on real searches from an AI agent and checked which providers came back with the answer. Every extra provider costs money and gives your agent more to read, so we wanted to know how much each one adds. Methods at the end has the full setup.

Most of the gain comes from the second provider

On its own, the best provider found the answer page 68% of the time. About a third of the searches came back without the known answer in their top five. No single provider stood out: the top three finished within a point of each other, and which one comes first depends on which searches you test.

You don't need the perfect pair, either. The dashed line in Fig. 1 averages over every possible mix. One provider picked at random found the page 55% of the time. Two picked at random found it 68% of the time, about as often as the best provider alone. That's handy if you'd rather not test 12 providers yourself.

The answers only a few providers had

Fig. 2 sorts the searches by how many providers had the answer page. It runs from all 12 on the left down to none on the right. Start at the right end: for 16% of searches, none of the 12 had it. Adding providers won't fix those. The agent has to search a different way, or open pages and read them.

Just left of that is where your choice of provider decides things. For 14% of searches, only one, two or three providers had the page. If your agent's one provider isn't among them, it never sees the answer. At the far left, only 5.5% of searches were easy enough for all 12 providers.

What's next

This is the gap we built Telem to close. On its own, the best provider missed the known answer for about a third of these searches. A second provider found 9 more in 100, and some answers sat with only one to three providers.

This test only checks whether the answer page shows up in the results. The agent still has to read them and get the answer right. Every extra provider also gives it more to read, which we'll come back to in a later post. One agent wrote all these searches, one per question, so your agent's numbers will look a bit different.

So where do those extra answers come from? Our next post looks at how little the providers' results overlap. After that, we look at what happens when an agent asks another provider after a miss.

Wanna try it out? Explore a demo in our playground.

---

Methods and setup

Searches

We used 363 test questions with known answers. Each search is the first one our test agent wrote for that question in September, sent again word for word. The agent is opencode, an open-source coding agent, running a language model, with Telem as its web search tool. The questions come from public sets (the RealTime QA news quizzes, WIT, Exa's company benchmark, MIRACL, FinSearchComp, LitSearch, FinanceBench and Needle in the Web) and from our own questions about posts on data silo sites (platforms whose content is hard to reach through search). We don't publish the second group, because those questions can name private people.

News quizzes are 41% of the questions. Without them the gain from more providers is larger: 56% for the best provider, 66% for two, 72% for three and 78% for all 12.

Measure

Known answer in the top five. The question's known answer page or value is in a provider's top five results. A simple rule checks it, with no judge. A mix of providers counts a hit when any provider in it has one.

The two lines

Best possible pick: for each number of providers, the set that found the most, chosen after seeing the results. Picked at random: the average over every possible set of that size. When we chose the mix on four fifths of the searches and scored it on the rest, one to four providers reached 64.5%, 75.5%, 79.9% and 82.6%. The one-provider figure drops because three providers tie near the top, so each split picks a different one.

Configuration

Telem sends agent searches to all of these providers, and we are a customer of each. For this test each provider ran in its best mode from our earlier tests.