p50 vs p90 vs p99 Latency in Web Scraping

By the fastCRW team · Benchmark numbers from diagnose_3way.py on Firecrawl's public 1,000-URL dataset, single run 2026-05-08, verified 2026-05-18 · Verify independently before quoting internally.

Latency percentiles in web scraping: p50, p90, and p99

If you only look at one latency number for a web scraper, you are looking at the wrong thing. A single "average response time" hides the exact behavior that decides whether your pipeline feels fast or stalls: the shape of the distribution. Latency percentiles — p50, p90, and p99 — describe that shape. They tell you what a typical request costs, what one-in-ten requests cost, and what the worst-case tail costs. This guide explains each percentile, shows why an average misleads, and walks through a real three-way percentile split so you can read any scraper benchmark honestly.

The short version: a percentile is a promise about how many requests finish under a given time. "p90 latency is 4348 ms" means 90% of requests finished in 4348 ms or less, and the slowest 10% took longer. That framing — counting requests, not summing milliseconds — is the whole point, because it survives outliers that an average cannot.

What a latency percentile actually is

Sort every request by how long it took. A percentile is the value at a given rank. The p50 sits at the halfway mark, the p90 at the 90% mark, the p99 at the 99% mark. Each answers a different operational question, and you need all three to plan capacity or set a service-level objective.

p50 (median): the typical request

The p50, or median, is the request in the middle. Half of all requests are faster, half are slower. It is the closest thing to "what does this normally feel like?" On the canonical scrape benchmark (diagnose_3way.py, Firecrawl's public 1,000-URL dataset, 2026-05-08), fastCRW's p50 was 1914 ms, beating Firecrawl's 2305 ms and effectively tied with Crawl4AI's 1916 ms — two milliseconds apart. The median is the honest headline for "how fast is the typical request," and it is resistant to a handful of slow outliers because it only cares about rank, not magnitude.

p90: nine out of ten requests

The p90 is the value that 90% of requests beat. Crucially, p90 is not a rare edge case — it is one in ten calls. If you fire a hundred scrapes, roughly ten of them will be at or beyond your p90. In fast mode on the same benchmark, fastCRW's p90 was 4348 ms — the lowest of the three tools tested (Crawl4AI 4754 ms, Firecrawl 6937 ms). This means nine out of ten fast-mode scrapes complete in under 4.4 seconds — better tail performance than either alternative.

p99: the worst-case tail

The p99 is the value 99% of requests beat — the deep tail, the one-in-a-hundred slow request. It matters when you run at volume, because one in a hundred is common at scale, and because the p99 is what determines your timeout ceiling. On the canonical run fastCRW's p99 was 15012 ms versus Firecrawl's 21107 ms and Crawl4AI's 13749 ms. Note the ordering: Firecrawl's worst case keeps stretching well beyond fastCRW's, while Crawl4AI has the tightest p99. That is the kind of nuance a single average erases completely.

Why the average lies about scraper performance

An average sums every request time and divides by count. That makes it exquisitely sensitive to large values — one pathological request can drag the mean far above what any typical request experiences. For web scraping, where a slow tail is structural (some pages need a full headless render, some hosts are slow, some retries fire), the average is almost always pulled toward the tail and away from reality.

How one slow tail skews the mean

Imagine nine requests at 1,900 ms and one at 14,000 ms. The median is 1,900 ms — an accurate picture of the typical request. The average is about 3,110 ms, a number no individual request actually experienced. Report only the average and you have simultaneously overstated the typical case and understated the worst case. Report only the median and you have hidden the tail. The honest move is to report the full split so the reader can see both the body and the tail of the distribution.

Why we never publish a single "average ms" for scrape

fastCRW deliberately publishes the full p50/p90/p99 split for scrape and never a single mean. A mean is pulled toward the tail and misrepresents the typical case. Treat any vendor's lone "average latency" claim with the same suspicion — ask for the percentiles, and if they will not give you a p90 and p99, assume the tail is the part they would rather you did not see.

Reading a real 3-way percentile split

Here is the canonical scrape result of record, three tools, identical inputs, one run of 3,000 requests on Firecrawl's own public 1,000-URL scrape-content-dataset-v1 (diagnose_3way.py, 2026-05-08). Bold marks the best value in each row.

p50: fastCRW 1914 ms vs Firecrawl 2305 ms

On the median, fastCRW has the lowest p50 of the three by a real margin over Firecrawl (1914 ms vs 2305 ms) and is statistically tied with Crawl4AI. So the typical scrape — the one your pipeline sees most of the time — has lower p50 latency than Firecrawl's. That is the honest median win, and we scope the claim to it rather than letting it imply the whole distribution.

p90 in fast mode: fastCRW 4348 ms — the lowest of three

In fast mode, fastCRW's p90 of 4348 ms is the lowest of the three. fastCRW's auto renderer falls back to a chrome-stealth path to recover pages the lighter renderers miss — and that recovery is exactly what earns the highest truth-recall (63.74% of 819 labeled URLs, the lead metric in the table). The same mechanism that wins accuracy also, in fast mode, delivers the tightest tail. You get more correct content and a better fast-mode p90.

p99: fastCRW 15012 ms vs Firecrawl 21107 ms

At the deep tail fastCRW's p99 (15012 ms) is lower than Firecrawl's (21107 ms) — the tail is bounded, while Firecrawl's worst case keeps stretching. Crawl4AI has the tightest p99 at 13749 ms. The lesson is not which tool "wins"; it is that you cannot rank tools on a single number. Each percentile tells a different story, and the right tool depends on which percentile your workload feels.

Which percentile matters for your workload

There is no universally "correct" percentile to optimize for. The one that matters is the one your users or your agents actually experience. Match the percentile to the workload.

Median for bulk batch jobs

If you are crawling ten thousand pages overnight and only care about total wall-clock throughput, the median dominates. A slow tail on 10% of requests is amortized across the batch and hidden by concurrency — while one worker waits on a slow page, fifty others keep finishing. For this profile fastCRW's p50 win and highest recall are the numbers that matter, and the p90 is largely irrelevant.

Tail for synchronous agent tool calls

If a scrape is a synchronous tool call inside an AI agent's reasoning loop, the tail matters. In fast mode, fastCRW's p90 of 4348 ms is the best of the three, so you can set tight timeouts confidently. Always plan against p90 and p99, not the median, and design timeouts and fallbacks from the rendering mode you intend to use. We cover this case in depth in honest tail latency for RAG agents.

Picking an SLO percentile

A service-level objective should name a percentile and a threshold together — "p95 under 5 seconds," not "fast." Pick the percentile from how badly a slow request hurts: user-facing and synchronous? Use p95 or p99. Background and concurrent? p50 or p90 is plenty. Then set the threshold from real measured data, and revisit it as your URL mix changes, because latency distributions are a property of the pages you scrape as much as the scraper itself.

How to compute percentiles from your own runs

Vendor benchmarks are a starting point, never a substitute for measuring on your own URL mix. The method is simple: collect raw per-request durations, sort them, and read off the ranks. The discipline is in the sample size and the reporting.

Sample size and warm-up

A p99 needs enough samples to be meaningful — with 100 requests your "p99" is essentially the single slowest request and will swing wildly between runs. Aim for at least a few hundred requests before trusting a p90, and well into the thousands before trusting a p99; the canonical fastCRW run used 3,000 requests for exactly this reason. Discard or separately label warm-up requests (cold caches, DNS, connection setup) so they do not contaminate steady-state numbers, and run all tools against identical inputs in the same window so network conditions are shared.

Reporting p50/p90/p99 together

Always publish the three together with their provenance — dataset, harness, date — so the body and the tail are both visible and the result is reproducible. Quoting one percentile in isolation invites the same distortion as quoting an average. If you want to see how we apply this discipline end to end, the full methodology and raw numbers live on /benchmarks, and the underlying mechanics of where scrape latency comes from are broken down in scraping latency explained and the broader fastCRW benchmark write-up.

Sources

• Scrape benchmark of record — diagnose_3way.py on Firecrawl's public scrape-content-dataset-v1 (1,000 URLs / 819 labeled), single run of 3,000 requests, 2026-05-08. Summarized at fastcrw.com/benchmarks.
• Firecrawl docs and dataset: docs.firecrawl.dev (referenced 2026-05-18).
• fastCRW repo: github.com/us/crw (AGPL-3.0).

Related: Scraping latency explained · Honest tail latency for agents · The fastCRW benchmark · A web context layer for AI agents