CRW vs Firecrawl: A Self-Hostable, Single-Binary Firecrawl Alternative

The problem, in one sentence

You want a scraping and search API you can drop into existing Firecrawl code, but you don't want to run a five-container stack — or pay per request for content you could pull on your own hardware.

That is the whole reason CRW exists. Firecrawl is a genuinely good product and the engineering behind it is solid — lovely engineering, honestly — but it is built to be run by Firecrawl, and the hosted meter and the self-hosted footprint both reflect that. CRW takes the opposite bet: one small Rust binary you own, an AGPL-3.0 engine you can self-host for the cost of your own server, and a request/response shape close enough to Firecrawl's that most callers change a base URL and nothing else.

This post is a head-to-head. We lead with where CRW wins — local/open footprint, and accuracy plus median latency on Firecrawl's own benchmark — and then we spend a full section on where CRW loses, because the honesty is the point, not a footnote.

What you're actually deploying

Start with the thing you have to operate, because that is where the difference is most physical. These are structural facts, not benchmark claims (source: the README §"Structural footprint").

• CRW ships as a single ~8 MB binary. Firecrawl's stack is roughly 2–3 GB total across its images.
• CRW runs in one container (plus an optional sidecar). Firecrawl runs five.
• The default Compose file ships LightPanda as the browser; a heavier Chrome variant is opt-in (~500 MB image, ~1 GB resident when you enable it).

For idle resource use, the engineering write-up cites CRW holding ~50 MB RAM idle on a $5 VPS (source: the how-we-built-fastCRW post's title/description), against a Crawl4AI/Playwright-style baseline it describes as 300 MB+ idle RAM and ~200–300 MB per worker. We're deliberately not quoting other RAM figures here — those are the ones we've verified.

The practical read: CRW fits on the cheapest box you have lying around, and there is exactly one process to babysit. If your reason for looking at a Firecrawl alternative is "I don't want to run a small Kubernetes cluster to scrape some pages," this is the headline.

The API is close to a drop-in

CRW speaks a Firecrawl-compatible API. In most codebases the migration is the base URL and your key:

curl -X POST https://your-crw-host/v1/scrape \
  -H "Authorization: Bearer $CRW_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com/article",
    "formats": ["markdown", "html"]
  }'

// TypeScript — the only change from a Firecrawl call is usually the base URL
const res = await fetch("https://your-crw-host/v1/scrape", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.CRW_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({ url, formats: ["markdown"] }),
});
const { data } = await res.json();

crawl, search, and map follow the same shape. Structured extraction is available by asking for formats: ["json"].

Two honest caveats so you're not surprised in code review:

• There is minor divergence in some response fields and the error envelope versus Firecrawl. Close, not byte-identical — check anything that parses error bodies.
• Some Firecrawl surface area simply does not exist in CRW: no multi-URL batched extract endpoint (you iterate or crawl), no agent endpoint, no deep-research endpoint. Requests are stateless — there's no persistent session and no WebSocket/SSE crawl-progress stream, so long crawls are polled, not streamed.

If your integration leans on those specific endpoints, CRW isn't a drop-in for you yet. If you're doing scrape / crawl / search / map, it usually is.

Accuracy: the benchmark we lead with

Here's the part that matters more than footprint for most people: does it actually get the content?

On Firecrawl's public scrape-content-dataset-v1 (1,000 URLs, 819 labeled), diagnose_3way.py, single run 3,000 requests, 2026-05-08, CRW came out highest on truth-recall:

• Truth-recall (of 819 labeled URLs): fastCRW 63.74% (522) · Crawl4AI 59.95% (491) · Firecrawl 56.04% (459).

That's +3.79 pp over Crawl4AI and +7.70 pp over Firecrawl on the labeled set — running against a dataset Firecrawl published. Truth-recall is the share of labeled URLs where the run returned the correct content, so this is the closest thing to "did the scraper actually do its job."

Now the honest counter-number, because a single stat lies. On raw scrape-success across all 1,000 URLs, Firecrawl is actually ahead:

• Scrape-success (of 1,000): fastCRW 87.7% (877) · Crawl4AI 83.5% (835) · Firecrawl 89.7% (897, highest).
• Thrown errors (of 3,000 requests): 0 · 0 · 0.

So all three threw zero errors across 3,000 requests — but "zero errors" only means nothing crashed, not that everything came back rich, which is why we pair it with the 87.7% success figure. Firecrawl returns a response slightly more often; CRW returns the right content more often on the labeled set. Those are different wins, and which one you care about depends on whether you're counting HTTP 200s or counting usable extractions.

Latency: the median win and the tail we won't hide

Same benchmark run. We publish the full percentile split every time, never a single average — averaging hides exactly the thing you need to see here.

Read that honestly. On the median request, CRW is the fastest of the three — 1914 ms, effectively tied with Crawl4AI two milliseconds away, and comfortably ahead of Firecrawl. On the p90 tail, CRW is the slowest of the three by a wide margin (14157 ms), and it stays behind Crawl4AI at p99.

That tail is not a mystery bug — it's causal. CRW's slow requests are the ones where the cheap renderer came back thin and CRW fell back to a Chrome-stealth render to recover a URL the others gave up on. That fallback is a big part of how it wins truth-recall, and it's exactly why the p90 blows out. You're paying tail latency to buy back content. If your workload is latency-critical and you'd rather drop a hard page than wait, that trade may not be for you — and you should know it going in.

There is a lighter path: in fast mode, CRW's p90 is 4348 ms — the lowest of the three — at 91.8% scrape success of reachable URLs (source: the how-we-built-fastCRW post's FAQ). Fast mode trades some of that recovered recall back for a tight tail. Pick the mode that matches whether you're optimizing for coverage or for p90.

Search is a separate story (and a strong one)

CRW also does web search, and it's worth being precise: this benchmark measures search only, not scrape — different harness, different thing.

Over a 100-query benchmark across 10 categories, benchmarks/triple-bench.ts, single point-in-time run:

• Average latency: fastCRW 880 ms · Firecrawl 954 ms · Tavily 2,000 ms.
• Median latency: fastCRW 785 ms · Firecrawl 932 ms · Tavily 1,724 ms.
• P95 latency: fastCRW 1,433 ms · Firecrawl 1,343 ms · Tavily 3,534 ms.
• Latency wins: fastCRW 73/100 · Firecrawl 25/100 · Tavily 2/100.
• Success rate: 100% · 100% · 100%.

The frozen way to cite it: fastCRW search averaged 880 ms over a 100-query benchmark; 73 of 100 latency wins. Note the one place CRW doesn't lead — Firecrawl edges it at P95 (1,343 ms vs 1,433 ms) — so on the worst-case search request they're close and Firecrawl is nominally ahead. Everywhere else in this run, CRW is faster, and all three succeeded on every query.

Cost, and the self-host escape hatch

This is the other reason people go looking for a Firecrawl alternative: the meter.

CRW's engine is AGPL-3.0. Self-host it and scraping costs $0 per 1,000 scrapes — you pay for your server, nothing else. For comparison, Firecrawl's hosted pricing works out to $0.83–5.33 per 1,000 scrapes across its tiers (source: marketing/competitor-prices.lock.md, verified 2026-05-18). On a single ~8 MB binary that idles around 50 MB of RAM, "run it yourself" is a real option, not a theoretical one.

If you'd rather use the hosted CRW plans, credits are deliberately simple:

• scrape — 1 credit, on any renderer (auto / http / lightpanda / chrome). Chrome no longer costs extra; it's a flat 1 as of 2026-06-24.
• crawl — 1 per page · search — 1 per query · map — 1 · browse (an MCP session) — 1 per session.
• Structured extract / any request with formats: ["json"] — 5 credits.

For the tier prices and included credits, see the live /pricing page — self-hosting the AGPL-3.0 engine remains free regardless of which hosted tier you'd otherwise be on.

On the LLM-backed features (search answer/summarize and JSON extraction): managed LLM usage is metered in credits based on usage. Those features are available on paid plans only — the free tier returns a 402 rather than calling a model. There's no bring-your-own-key path; it's the managed path or self-host the engine and wire your own.

The honest gaps

We keep a whole post titled "Where CRW Still Falls Short," and a comparison wouldn't be honest without the same treatment. Here are the real ones (source: CANONICAL-FACTS §9 and the limitations post):

No PDF / DOCX / document parsing. CRW handles HTML pages. Point /v1/scrape at a PDF and you get empty or an error — not text. Firecrawl parses documents; CRW does not, today.

• Who it affects: anyone scraping docs, filings, or paper repositories.
• Workaround: route document URLs to a dedicated parser.

# ponytail-style fallback: let CRW do HTML, hand PDFs to pdfplumber
import requests, pdfplumber, io

def fetch_text(url: str) -> str:
    if url.lower().endswith(".pdf"):
        raw = requests.get(url, timeout=30).content
        with pdfplumber.open(io.BytesIO(raw)) as pdf:
            return "\n".join(p.extract_text() or "" for p in pdf.pages)
    r = requests.post(
        "https://your-crw-host/v1/scrape",
        headers={"Authorization": f"Bearer {CRW_API_KEY}"},
        json={"url": url, "formats": ["markdown"]},
        timeout=60,
    )
    return r.json()["data"]["markdown"]

PDF text extraction is on the roadmap (not this cycle); DOCX is further out.

Anti-bot is not best-in-class. CRW does the basics well — realistic user agents, header mimicry, cookies, sensible delays — which covers roughly the 80% of public content that isn't actively defended. Against aggressively-protected sites (Cloudflare Enterprise, PerimeterX, DataDome, Akamai Bot Manager) it gets blocked more often than a specialized service would. There's no equivalent of Firecrawl's heavier anti-bot layer.

• Workaround: bring an external residential proxy via the proxy field.
• CAPTCHA solving isn't planned; built-in proxy rotation is being evaluated.

SPA coverage via LightPanda is inconsistent. The default browser, LightPanda, is an experimental Zig engine implementing a subset of the W3C APIs — it is not Chromium. Server-hydrated pages render well; heavy client-side React/Vue/Angular apps with lazy-loading and client routing may come back thin.

• Detection: under ~200 words on a page you know is rich.
• Workaround: force the chrome renderer, or fall back to Playwright/Firecrawl for that page.
• This is the highest-priority core gap and is under active development.

Statelessness and streaming. No persistent sessions, no WebSocket/SSE crawl-progress stream — you poll crawl status. And the LLM features are managed-only on paid plans, with the free tier returning 402. robots.txt is respected by default.

None of these are secrets we're burying — they're the reasons you might legitimately stay on Firecrawl, and we'd rather you find out here than in production.

Who should pick which

• Pick Firecrawl if you need document parsing, best-in-class anti-bot against hard targets, streamed crawl progress, or the batch/agent/deep-research endpoints — and you're happy to run the five-container stack or pay the hosted meter.
• Pick CRW if you want to own the thing: one ~8 MB binary, ~50 MB idle RAM, a self-hostable AGPL-3.0 engine at $0 per 1,000 scrapes, a Firecrawl-compatible API for scrape/crawl/search/map, top-of-three truth-recall on Firecrawl's own dataset, and the fastest median request in that run — as long as you can live with the slow p90 tail that recall costs, or run fast mode instead.

Most teams evaluating a Firecrawl alternative aren't choosing on a single axis. They're weighing "how much do I want to operate" against "how good is the content" against "what does it cost at my volume." CRW's answer is: very little to operate, highest recall on the shared benchmark, free if you self-host — with a genuinely worse latency tail and a shorter feature list. That's the trade. Now you can make it with the actual numbers.