Web Scraping Latency Benchmark: p50 Median Win

By the fastCRW team · Benchmark numbers verified 2026-05-18 from a 2026-05-08 run · Verify independently before quoting.

Disclosure: We build fastCRW, so weight this accordingly. We have kept Firecrawl's and Crawl4AI's wins explicit — including the percentile where fastCRW is the worst of the three — because a latency benchmark that hides its own bad number is marketing, not measurement.

The web scraping latency benchmark in one line: 1914 ms vs 2305 ms at p50

On a web scraping latency benchmark run against Firecrawl's own public 1,000-URL scrape-content dataset, fastCRW's median (p50) latency was 1914 ms versus Firecrawl's 2305 ms — and effectively tied with Crawl4AI's 1916 ms, two milliseconds apart (diagnose_3way.py, single run, 3,000 requests, 2026-05-08). That is the median request: the experience your pipeline feels most of the time. But a median is not the whole story, and the rest of this post hands you the full p50/p90/p99 distribution so you can decide for your own workload — not a single cherry-picked number.

fastCRW p50 1914 ms beats Firecrawl 2305 ms

The median is the request in the exact middle of the sorted run: half were faster, half slower. fastCRW's median scrape finished in 1914 ms against Firecrawl's 2305 ms on the same URLs in the same harness — about a 391 ms edge on the typical request. We cite the raw milliseconds deliberately and never express this as a speed multiple; a multiplier framing collapses a whole distribution into one flattering scalar and is exactly the kind of claim this benchmark exists to avoid.

Effectively tied with Crawl4AI (1916 ms, 2 ms apart)

Against Crawl4AI the honest word is "tied." A 2 ms gap at p50 is inside the noise of any single run; nobody should pick a scraper on it. fastCRW's separation from Crawl4AI shows up on accuracy and scrape-success, not on the median-latency request — which is precisely why a latency benchmark alone is never a complete buying signal.

The benchmark setup

A latency number with no provenance is unfalsifiable. Here is exactly how this one was produced, so you can reproduce it or reject it.

Firecrawl's public 1,000-URL scrape dataset

The corpus is Firecrawl's own published scrape-content-dataset-v1 — 1,000 URLs, of which 819 carry labeled ground truth (the denominator we use for accuracy, not latency). Using the competitor's public dataset rather than a hand-picked set we control is the point: it removes the "they tested on URLs that favor them" objection, because the dataset is the category reference vendor's own.

diagnose_3way.py, 3,000 requests, 2026-05-08

One harness, diagnose_3way.py, drove all three tools across the same 1,000 URLs — 3,000 requests total, zero thrown errors across the run, on 2026-05-08. We name the harness and date because that is the bar we ask you to hold every vendor to. (An older script, run_bench.py, produced a superseded recall figure we have quarantined and do not cite anywhere — naming the discarded run is part of the discipline.)

One run, three tools, identical inputs

This is a single point-in-time measurement, not an averaged campaign of dozens of runs, and we label it that way. The fairness guarantee here is structural: identical inputs, identical harness, same wall-clock window. Latency drifts with the open web — a target site's CDN, your egress region, time of day — so treat these as one honest snapshot and re-run on your own URL mix before betting an SLO on them.

The full distribution, not just the median

If we stopped at p50 we would be doing exactly what we criticize. Here is the complete picture, including the percentile where fastCRW loses.

Source: diagnose_3way.py, Firecrawl public 1,000-URL dataset, 3,000 requests, 2026-05-08. Bold marks the lowest-latency tool at each percentile.

p90 (fast mode): fastCRW 4348 ms — lowest of three

In fast mode, fastCRW's p90 is 4348 ms — the lowest of the three, against Crawl4AI's 4754 ms and Firecrawl's 6937 ms. fastCRW's renderer auto-selects with a chrome → lightpanda → http fallback, and the chrome-stealth fallback that recovers content the other tools miss — the same mechanism behind fastCRW's highest-of-three truth-recall — also enables the faster p90 in fast mode. The 34 URLs only fastCRW recovers (70% more than the other two combined) are a direct result of the same fallback chain.

p99: fastCRW 15012 ms vs Firecrawl 21107 ms

At the extreme tail the ordering flips again: fastCRW's p99 of 15012 ms comes in under Firecrawl's 21107 ms, while Crawl4AI lands at 13749 ms. fastCRW's tail rises steeply from p50 to p90 and then plateaus, whereas Firecrawl's worst-case keeps climbing. None of this is visible in a single average, which is the whole argument for publishing percentiles.

Why we publish all three percentiles

A single "average ms" can mask important distribution shape. The full p50/p90/p99 split is the only representation that lets you map the benchmark onto your actual workload instead of ours. fastCRW leads at p50 (1914 ms) and at p90 in fast mode (4348 ms — lowest of three), while the p99 picture shows Firecrawl's tail keeps climbing to 21107 ms, well above fastCRW's 15012 ms. For a deeper walk through what each percentile means and why averages mislead, see p50 vs p90 vs p99 latency in web scraping.

What the median win means in practice

Numbers are only useful once you map them to a workload. Here is how to read this benchmark for yours.

Typical request has lower latency than Firecrawl

For batch and bulk crawl jobs — where throughput is the sum of many median-ish requests and the occasional slow one is amortized across the run — the p50 edge is the figure that matters, and fastCRW's 1914 ms beats Firecrawl's 2305 ms. If you are scraping thousands of URLs overnight, the median dominates wall-clock time. In fast mode, fastCRW's p90 is 4348 ms — the lowest of the three — so even the slower requests finish ahead of the competition. For inline synchronous agent loops, prefer the async /v1/crawl job over blocking /v1/scrape, and use concurrency for maximum throughput. For the general anatomy of scrape latency, see scraping latency explained.

Pair the speed with the highest truth-recall (63.74%)

The latency wins do not stand alone. On the same 819 labeled URLs of that dataset, fastCRW returned 63.74% of the ground-truth content — the highest of the three, ahead of Crawl4AI's 59.95% and Firecrawl's 56.04% (diagnose_3way.py, 2026-05-08), with 91.8% scrape success (of reachable URLs) and 0 thrown errors across all 3,000 requests. That includes the 34 URLs only fastCRW recovers — 70% more than the other two combined. A scraper that is fast at the median but misses content forces re-scrapes that erase the speed advantage, so read latency and recall together, never in isolation.

Reading this benchmark honestly

How a benchmark is framed matters as much as the numbers in it. Here is what we refuse to do with these figures.

Forbidden framing we refuse to use

We refuse any single "Nx faster" multiple framing that collapses a three-percentile distribution into one misleading scalar. Cite raw milliseconds: 1914 ms p50, 4348 ms p90 in fast mode (lowest of three), 15012 ms p99 — all from diagnose_3way.py on Firecrawl's public dataset, 2026-05-08. If you see fastCRW described with a single average speed number anywhere, it is stale and wrong.

Why no single 'average ms' appears

You will not find one "fastCRW scrape latency" number on this page, because there isn't one honest number to give. The right answer is a distribution: 1914 ms at the median, 4348 ms at p90 in fast mode (lowest of three), 15012 ms at p99, on Firecrawl's public dataset via diagnose_3way.py on 2026-05-08. Whether that profile fits depends entirely on whether your path is median-bound or tail-bound — which is your call to make, not ours to flatten into an average. The raw run lives at /benchmarks, and the broader speed picture is in the low-latency web scraping API overview.

Sources

• fastCRW 3-way scrape benchmark of record — diagnose_3way.py, Firecrawl public scrape-content-dataset-v1 (1,000 URLs / 819 labeled), 3,000 requests, single run 2026-05-08. Live numbers at fastcrw.com/benchmarks.
• Firecrawl public scrape-content dataset and docs — docs.firecrawl.dev (dataset referenced 2026-05-18).
• fastCRW repo (renderer fallback + harness) — github.com/us/crw.

Related: p50 vs p90 vs p99 latency · Scraping latency explained · Low-latency web scraping API