By the fastCRW team · Pricing/features verified 2026-05-18 · fastCRW launch pricing expires 2026-06-01 · Verify independently before buying.
Disclosure: We build fastCRW. This is a vendor-authored comparison, so weight it accordingly — the only pricing we present as frozen fact is fastCRW's own credit table; every competitor figure here is dated, sourced, and flagged "verify current pricing."
Website crawl pricing at scale: why the unit matters
When you plan a whole-site crawl, the question that decides your budget is not "what does one page cost?" but "does that per-page cost stay flat across ten thousand pages, or does it climb once rendering and extraction multipliers kick in?" Website crawl pricing splits into two broad models: a fixed per-page credit, and a multiplier model where the effective cost per page depends on how the page was fetched. At a single-page scale the difference is invisible. At crawl scale — thousands of pages in one job — it is the entire bill.
fastCRW meters crawling at a flat 1 credit per page for every renderer — auto, lightpanda, http, and chrome all cost the same 1 credit, per the credit table in. That means a crawl bill is forecastable from page count alone, before you start the job. This post walks the two pricing models, shows the per-10,000-page math on fastCRW's frozen credits, and is honest about where a multiplier-heavy vendor still earns its keep.
How crawl pricing is metered
Every crawl API starts from a base per-page unit, then either holds that unit constant or layers multipliers on top.
Per-page credits as the base unit
The base unit is one fetched page. A site map discovers the URLs; the crawler walks them breadth-first; each retrieved page consumes the base credit. fastCRW's base unit is 1 credit per crawled page. Firecrawl's locked base rate is also 1 credit = 1 page (source: marketing/competitor-prices.lock.md, verified 2026-05-18) — the one competitor anchor in this post that is an audited, locked figure rather than an illustrative one.
Where render and extract multipliers stack on top
The base unit only holds if nothing multiplies it. Two common multipliers change the effective per-page price:
- JS rendering multipliers. Some APIs charge a multiple of the base credit whenever a page needs a headless browser to render JavaScript. On a modern site where most pages need rendering, that multiple becomes the effective rate for nearly every page.
- Extraction surcharges. If you want structured JSON out of each crawled page, some vendors bill that through a separate token plan stacked on top of the crawl credit — two line items, not one.
Why a big crawl is the worst case for a multiplier model
A multiplier you barely notice on a 50-page test compounds linearly across a 10,000-page crawl. If a render multiplier turns a 1-credit page into 5 effective credits, a 10,000-page crawl is not 10,000 credits — it is 50,000. The multiplier model's cost is a function of how your pages are fetched, which you often cannot predict before the crawl runs. That uncertainty is the real tax, separate from the dollar total.
fastCRW crawl pricing
1 credit per page, every renderer
fastCRW's crawl is a flat 1 credit per crawled page regardless of which renderer runs — auto, lightpanda, http, and chrome all cost the same single credit. There is no separate render-multiplier tier and no separate extraction subscription. The cost for any crawled page is always 1 credit, full stop. That flat ceiling is what makes the bill forecastable: you do not need to know the render mix in advance to bound the cost.
maxDepth and maxPages controls
A crawl is bounded by two parameters. maxDepth caps how many link-hops from the seed URL the crawler will follow (cap 10), and maxPages caps the total page count (cap 1000); limit and max_pages are accepted serde aliases of maxPages. Because maxPages is a hard ceiling on the page count and the per-page credit is always 1 regardless of renderer, you can compute the absolute worst-case credit cost of a single crawl job before you submit it: maxPages × 1 credit — no renderer surcharge can raise that figure.
Forecasting a crawl bill before you start the job
The forecasting recipe is simple. Estimate the site's page count (a /v1/map call costs 1 credit and returns the URL list), pick a maxPages ceiling, and assume a render mix. A 4,000-page documentation site that is mostly static HTML lands at exactly 4,000 credits; the same crawl on a JS-heavy app where chrome runs on every page is still exactly 4,000 credits — because every renderer costs the same 1 credit. No multiplier surprises exist at any point in that range.
Crawl cost at scale, compared
Per-10,000-page math: fixed credit vs illustrative render multiplier
Here is a like-for-like crawl of 10,000 pages. The fastCRW columns are frozen credits. The "multiplier model" column is illustrative only: it shows what a hypothetical 5× JS-render multiplier does to the same page count. It is not a quote for any named vendor, and any real multiplier figure should be verified against that vendor's live pricing page.
| Render mix on 10,000 pages | fastCRW credits (1/page, any renderer) | Illustrative 5× multiplier model |
|---|---|---|
| All static HTML | 10,000 | 10,000 |
| Half need a browser | 10,000 | 30,000 |
| All need a browser | 10,000 | 50,000 |
fastCRW's cost is always 10,000 credits regardless of render mix — chrome, lightpanda, and http all cost the same 1 credit per page, so best case equals worst case. Under an illustrative 5× multiplier, an all-rendered crawl of the same 10,000 pages reaches 50,000 credits — five times the baseline. The gap is not just the headline price; it is the slope. fastCRW's cost is flat across any render mix; a 5× multiplier model's cost rises 5× over the static baseline. Translate credits to dollars by your plan rate at /pricing rather than hard-coding a tier here, because launch pricing reverts on 2026-06-01.
How the auto-renderer keeps most pages at 1 credit
fastCRW's renderer defaults to auto, which selects chrome → lightpanda → http and escalates as needed to produce usable content. Because every renderer costs the same 1 credit, the auto-selection path carries no credit penalty — a page handled by chrome costs exactly the same as one served by the lightweight http fetcher. The render mix has no effect on your bill; every page is 1 credit regardless of which engine ran.
The self-host path for whole-site passes
$0 per 1,000 scrapes self-hosting AGPL-3.0
The fastCRW engine is a single static Rust binary under AGPL-3.0. Self-hosted, it costs $0 per 1,000 scrapes — you pay only for your own server. For a crawl, that converts the per-page credit into a fixed infrastructure cost: a recurring 50,000-page crawl that would consume managed credits every month instead runs against a binary you already pay to host, and the marginal cost of one more page is effectively zero.
Why large recurring crawls favor self-host
One-off crawls rarely justify the operational overhead of self-hosting. Large recurring crawls do. If you re-crawl a 20,000-page catalog nightly for freshness, a per-page credit model meters every page every night, while a self-hosted binary on a small VPS caps the cost at your server bill regardless of crawl volume. The crossover point is volume and cadence: the more pages and the more often, the more the $0-per-1,000 floor matters relative to any per-credit rate. See cost of web scraping at scale for the broader self-host-versus-managed math.
Choosing a crawl pricing model
One-off audit vs recurring scheduled crawl
For a one-off site audit — crawl once, extract structure, move on — managed credits on a flat per-page model are the least-friction choice: no infrastructure, a forecastable bill, done. For a recurring scheduled crawl, the calculus tilts toward self-host, because the fixed-floor advantage compounds with every repeat pass. Map the URL set first with crawl an entire website from its sitemap, decide cadence, then pick the model that matches.
When predictable per-page beats pay-per-feature
The honest summary: a fixed per-page rate wins when you value forecastability and a bounded worst case, and a free self-host escape hatch raises that ceiling further. A pay-per-feature multiplier model can still be the right call when you depend on capabilities a flat engine does not offer — which is the next, honest section.
Honest gaps: where a multiplier vendor genuinely wins
A fixed per-page model is not free of trade-offs, and pretending otherwise would not help you decide. State the gaps plainly:
- No Fire-engine anti-bot. fastCRW has no heavy anti-bot fire-engine path and no residential proxy pool. A crawl that must defeat aggressive bot defenses on every page is a job where a proxy-and-anti-bot-heavy vendor's multiplier earns its cost — you are paying for capability the flat engine does not provide.
- No multi-URL batched extract. There is no
/v1/batch/scrapeand no multi-URL batched/v1/extract. For structured extraction across a crawl you iterate/v1/scrapewithformats: ["json"]or crawl then extract per page. - Stateless, single-URL extraction. fastCRW is stateless per request, and LLM extraction supports OpenAI and Anthropic providers only. A vendor offering deep agentic crawl-and-extract orchestration covers ground a primitive engine deliberately does not.
If your crawl depends on those capabilities, the multiplier you pay is buying something real. If it does not, a fixed per-page rate plus a $0 self-host floor is the cheaper and more predictable shape.
Sources
marketing/competitor-prices.lock.md— Firecrawl base crawl rate 1 credit = 1 page (verified 2026-05-18); all render-multiplier framing for other vendors is illustrative, not a locked quote.- fastCRW repo and live pricing: github.com/us/crw · fastcrw.com/pricing
- Firecrawl crawl docs: docs.firecrawl.dev (verify current pricing before relying on any multiplier figure)
Related: Crawl an entire website from its sitemap · Firecrawl crawl endpoint deep dive · Cost of web scraping at scale · fastCRW pricing