← All posts

2026-08-23 · TelemAI Research

TelemAI Research

The Web-Search Latency Your Agent Actually Pays

We measured cold-start latency across 9 web-search APIs; the number your agent pays isn't the one benchmarks report.

measurement setup & methodology at the end

Ask an agent a genuinely hard question then it doesn't run a search: it runs a deep search campaign. The question gets decomposed, sub-agents fan out, and every sub-task searches on its own. One deep research can run on our own infrastructure issued 148 web-search calls before writing a word of its answer. OpenAI tells Deep Research users up front that a single answer may take 5–30 minutes^[1]; Anthropic's research agent parallelizes its sub-agents precisely because doing so cuts research time by up to 90%^[2]. Search is the inner loop of all of it.

Web-search latency is also one of the worst-understood numbers: the latency a benchmark measures, and the latency an agent pays, are usually two different things. Here are two measurements of the same search API, same request shape, same region, hours apart:

Both numbers are real. Both are median values over 150 requests, not cherry-picked outliers. The only difference between them is whether the provider had seen that query string in roughly the last half hour.

That's a 37× envelope for a single provider. As a result, published search-API latency numbers are so hard to reconcile with what you see in production. Almost every published comparison we could find reports a single latency figure without saying whether its queries were fresh. Within a 37× envelope, a number without that disclosure is not interpretable.

This post is about the right-hand number: As we'll show, if you're building agents, the cache essentially never helps you.

The cold price list

To compare nine providers fairly, we forced them onto one output contract (10 results, each with a text snippet). We then found the cheapest request that achieves it on each API, and issued 150 brand-new, never-before-seen queries^[5] per provider, strictly serially, with per-phase timing (the numbers below are time-to-first-byte with connection setup subtracted; one network round-trip still rides inside TTFB, which is why providers closer together than the geography envelope are reported as ties rather than ranked). Same contract, all cold:

Three things:

• The spread is 12×: 320ms to 3.9s for the same deliverable. This is not a "premium vs budget" spread either; the fastest provider here is not the most expensive.
• The tail is a second ranking hiding inside the first: serpapi's p95 is 13.6 seconds (3.5× its own median). ceramic's p95 is only 1.05× its median, which is slow but metronomic. If your agent fans out to several providers and waits for all of them, the tail is your latency. With a 9-way fan-out, the p50 you experience is the maximum of nine draws. We measured it at 4.3s, with p95 at 13.6s.
• Some rankings are not decidable from one vantage point. We measured our network distance to each provider (a TLS 1.3 handshake costs one network round-trip^[6]; spread 3.7–89ms). Pairs whose medians differ by less than that envelope, such as you.com vs brave (16ms apart), or ceramic vs tavily (73ms), are tied as far as a single measurement location can say, which is why we have not ranked them.

Q: Do these rankings survive real agent queries?

Our 150 test queries were synthetic^[5]. The Information Retrieval literature is actually encouraging here: LLM-built test collections have been shown to reproduce system rankings faithfully^[7] even though synthetic queries differ measurably from human ones in absolute terms^[8]. We also replayed the 77 queries the real agent had generated (after a three-hour cool-down) through the same instrument, paired across all nine providers, with 693 calls and zero failures. We also included a batch of agent and synthetic queries fired within the same minutes, so time-of-day can't tip the scale. Rank agreement with the synthetic set: Kendall τ = 0.71^[9] by our preregistered rule, 0.90 after also excluding the provider whose latency appears server-state-driven rather than query-driven. And side by side, the two kinds of query are simply indistinguishable:

Every interval contains, or in brave's case grazes, 1.00. The widest belong to the two highest-variance providers (serpapi's point estimate of 0.79 comes with a 0.60–1.33 interval: noise, not a discount).

Your agent always pays the cold-start latency, and here are some more interesting findings:

Web-search caches are startlingly short-lived. We warmed each provider's cache with fresh queries, then revisited each query once after a controlled delay. We tested each delay with its own independent query set, because revisiting resets the clock. The full cycle, cold → warm → cold again:

One caveat before the table: each point above plots the median of six revisits. That representation is fair for providers whose cache answers consistently, and misleading for one that doesn't, as tavily's row explains.

Every cache we could observe dies within an hour. Most die within 15 minutes.

Some of the providers include interesting Cache: We took 150 queries from MS MARCO^[3], which consists of real Bing user queries that have been public, studied, and re-issued by researchers for years. These long-studied queries seemed to be the most likely to be pre-cached of any queries on the internet. First-touch cache hit rates:

serpapi: 0.0%. exa: 1.3%. you.com: 4.7%. Brave showed meaningful residual warmth (29%). It could be that some labs were experimenting or benchmarking MSMARCO there.

What the wait buys you

Latency is only half the invoice. The same "10 results with snippets" contract delivers very different amounts of material. Fast is not automatically good value, and slow is not automatically bad:

tavily's 1.2s buys ~8KB; seltz's 0.32s buys 56KB; brave's 0.64s buys 52KB. Per kilobyte of material, the spread across providers is wider than the latency spread itself. Three providers (linkup, parallel, tavily) also ignore Accept-Encoding: gzip entirely and ship uncompressed responses. The others compress 2.6–4.5×.

When "snippet" secretly means "run an LLM"

Exa is the only API where snippets are opt-in, and it offers several kinds. We observed significant variance across them:

Extractive highlights arrive in 1.3s at $0.007 per call, which is statistically indistinguishable from requesting no text at all. Asking for the LLM-written summary field costs 6.8s and $0.017: a 5× latency, 2.4× price cliff for running a language model inside your search call, which is material your downstream model would happily read raw. Check what your "snippet" actually is.

The cost–latency plane

Latency and price turn out to be independent axes, so here they are as one picture — every provider under the same contract, cold median TTFB against price per 1,000 calls. Five of the nine sell at the same $5/1k list price, and at that identical price they differ 7.5× in speed. The efficient frontier is two providers: ceramic (two orders of magnitude cheaper than the field, at mid-pack speed) and seltz (fastest, at the standard price) — everything else is dominated, and the most expensive API is also the slowest. The LLM-summary route from the chart above lands exactly where you would fear: the expensive-and-slow corner. How each price was established — published price pages, live reads of the vendors' billing APIs, or billing the API reports in-band — is itemized under the chart.

Cache anatomy: it's not a yes/no question

"Does this API cache?" turns out to be four separate questions, and providers differ on every axis independently:

1. Admission: how many hits before it stores? Five providers served a warm response from the second sighting onward. Four providers — ceramic, linkup, parallel, seltz — showed no cache benefit even after ten consecutive hits on the same query.
2. Hit consistency: tavily admitted every test query by the second or third hit (like the others), but its warm hits were intermittent: after a single prior hit, only 1–2 of 6 revisits came back fast, at every delay we tested.
3. Depth (how much faster is a hit?): From 2.4× (brave) to 37× (serpapi, whose cached responses arrive in ~12–105ms. The fastest search responses we measured all experiment long came from the slowest cold provider).
4. Lifetime: 5–60 minutes, as above.

Below, we see the same experiment side by side. Six fresh queries per provider, each hammered ten times; first request vs the settled repeat requests:

Concurrency: the second thing agents do that benchmarks don't

For an agent, concurrency is latency: a deep-research step fires its sub-queries at once and waits for the slowest, times however many rounds it runs (our harvested runs sent 2–4 per tool call). A provider that rejects concurrent requests stretches every loop. So — how many simultaneous requests does each one admit before it starts saying no?

And does the ceiling we measured match what each provider documents? Where a number is published and a burst can reach it, it holds almost exactly:

Per-minute and hourly limits can't be reached by a burst — those rows are marked consistent, not pinned. "cleared N" means every one of N concurrent requests succeeded — the highest burst we fired, not a wall; the true ceiling is at least that high.

Where it's allowed, concurrency is nearly free

The second half of the story is what parallelism does to the latency of the requests that succeed. Answer: for most providers, almost nothing:

Six of nine providers sit inside the noise band. Sixteen parallel searches cost roughly the same per-request latency as one. exa (~1.4×) and parallel (~1.5×) show consistent mild elevation. serpapi's ratio is not interpretable either way (its cold-latency variance dwarfs the per-level samples). For an agent, this is the practically important half: where the gate allows it, fanning out is free speed.

The agent scorecard

No composite score here, as these axes don't collapse into one number honestly. Pick the column your workload cares about:

A few readings of this table:

• Building a latency-critical agent with fan-out?: The predictable providers (low tail ratio, no cache surprises) beat the nominally fast ones. seltz is both fastest and most deterministic (no observed cache — every request costs the same ~320ms).
• Re-running the same queries all day: (monitoring, dashboards)? serpapi's 37× cache depth turns it from the slowest into by far the fastest, but only if your repeat interval beats its 30–60 min lifetime.
• Fan-out breadth matters more than single-call speed?: Watch the concurrency ceiling and rejection-signal columns, not the latency column.

Method: how hard we tried not to fool ourselves

1 · One contract, one clock. Every number in this post comes from the same instrument: the cheapest request achieving "10 results + snippets" on each API, issued strictly serially, one fresh connection per request, timed in four phases so that network effects never contaminate provider time. Three properties we verified rather than assumed, and then we applied the same decomposition to all nine providers:

2 · Fresh queries, provably. Queries came from a deterministic generator + seed^[5] (compositional strings that have never existed) cross-checked against a ledger of every query ever consumed in the study (936 entries). A parallel set of real user queries (MS MARCO^[3]) ran alongside it. Every query hit a given provider at most once, ever, except where measuring repetition was the experiment.

3 · Validated on real agent traffic. Ten research tasks run twice through a real agent harness produced 77 queries; replaying them cold reproduced the synthetic ranking (τ = 0.71–0.90) and turned the "agents never repeat" assumption into a measurement (0/77).

What we can and can't promise. The discipline above proved that early small-sample probes suggesting a tidy conclusion simply dissolved when we re-measured at 48 and 150 queries (the cache section keeps one example). Nothing reached this page without surviving that treatment. Disclaimer: This report was run using nine providers, using one vantage point, over one week of measurement, and may still have blind spots. If your numbers disagree with ours, please let us know.