How to Migrate from Firecrawl Without Rewriting Your Code (2026)

By the fastCRW team · Last reviewed 2026-05-18

Disclosure: We build fastCRW, a Firecrawl-compatible scraper. This guide is written to be genuinely useful even if you migrate to something else — the validation checklist applies to any target.

The core idea: it's a base-URL swap, not a rewrite

The single most important fact about leaving Firecrawl is this: the official Firecrawl SDK works with fastCRW by changing the api_url. fastCRW exposes a Firecrawl-compatible API surface (/v1/scrape, /v1/crawl, /v1/map, /v1/search), so your existing client code, method calls, and response handling stay the same. You point the SDK at a different host and keep your key. That is the migration for the common case.

This is why "switching cost" — normally a vendor's strongest retention moat — barely applies here. You are not porting an integration; you are redirecting one.

Step 1: the one-line change

Python, using the official Firecrawl SDK:

# Before — Firecrawl Cloud
from firecrawl import FirecrawlApp

app = FirecrawlApp(
    api_key="fc-...",
    api_url="https://api.firecrawl.dev",
)

# After — fastCRW (self-hosted or Cloud)
app = FirecrawlApp(
    api_key="your-fastcrw-key",
    api_url="https://your-fastcrw-host",  # or https://api.fastcrw.com
)

doc = app.scrape_url("https://example.com", params={"formats": ["markdown"]})

Node / TypeScript:

import FirecrawlApp from "@mendable/firecrawl-js";

const app = new FirecrawlApp({
  apiKey: "your-fastcrw-key",
  apiUrl: "https://your-fastcrw-host",
});

const doc = await app.scrapeUrl("https://example.com", { formats: ["markdown"] });

If you use LangChain's FirecrawlLoader or LlamaIndex's Firecrawl reader, those accept a base URL / API URL parameter too — same principle, same one-line change.

Step 2: know where the surfaces overlap and diverge

The compatibility is on the overlap surface — scrape, crawl, map, search. Most production pipelines live entirely inside that surface, which is why most migrations are clean. Be deliberate about these areas:

• Field names and error envelopes: a small number of fields and the error response shape can differ slightly. Wrap your client and assert on the fields you actually consume rather than trusting the whole payload blindly.
• Crawl job semantics: both expose submit-then-poll crawl jobs. Validate that your status-polling loop reads the right job-state and result fields against the new host.
• Structured extraction: on fastCRW, JSON extraction is part of the scrape call under the same credit model — there is no separate extract subscription. If your code calls a dedicated Firecrawl extract endpoint, re-express it as a scrape with a JSON format/schema. This is usually simpler, not harder.
• Cloud-only Firecrawl features: some Firecrawl features (heavy anti-bot fire-engine paths, certain agentic endpoints) are cloud-only specialties. If you depend on one of those specifically, test that path explicitly before cutover rather than assuming parity.

Treat the overlap as "the 90% that just works" and the divergences as "the short, known list to test." Don't assume; verify the fields you read.

Step 3: the validation checklist

Before cutting traffic over, run this against a sample of your real URLs (not just example.com):

1. Output parity: scrape the same 50–100 representative URLs against both hosts. Diff the markdown. Minor whitespace differences are fine; missing main content is not.
2. Crawl coverage: crawl a known site on both. Compare the set of discovered URLs and page counts. Investigate any large gap.
3. Schema extraction: if you extract structured JSON, run your schema against 20+ pages and validate the JSON shape and field fill rate.
4. Error handling: deliberately request a 404, a timeout-prone page, and a blocked page. Confirm your error-handling code still classifies them correctly under the new error envelope.
5. Latency: measure p50 and p95 for your traffic mix on both. fastCRW's engine is a single Rust binary built for speed (open-core, ~6MB), so latency is typically a win, but measure on your URLs, not ours.
6. Throughput / concurrency: run your peak concurrency for 10 minutes and watch for rate-limit or queue-depth differences.

Step 4: pilot, then cut over

Don't flip 100% of traffic at once. Recommended rollout:

1. 5% shadow: send 5% of production scrapes to the new host in parallel, log both, compare offline. No user impact.
2. 25% live: route a quarter of real traffic to the new host. Watch error rate and latency for 24–48 hours.
3. 100%: cut over fully. Keep the old client config behind a feature flag for one week so rollback is a config change, not a deploy.
4. Decommission: after a clean week, remove the old credentials and cancel the old plan.

Why teams migrate off Firecrawl

The technical migration is easy; the reasons people do it are worth naming so you can confirm the move actually solves your problem:

• Cost. The per-page credit model is fine until extraction's separate subscription or a tier-ceiling jump roughly doubles the bill. Migrating to an engine you can self-host caps the worst case at a VPS price.
• Lock-in. Usage-based pricing scales super-linearly with agent volume and there's no self-host escape valve. An open-core engine removes the hostage dynamic.
• Privacy / compliance. On a hosted-only service, every scraped page flows through the vendor's cloud. Self-hosting an open-core engine keeps scraped data and target URLs on your own infrastructure.
• Footprint and speed. A single ~6MB Rust binary with tiny idle RAM is a different operational story than a multi-service hosted stack.

The two destinations: self-host or managed

fastCRW gives you both with the same Firecrawl-compatible API:

• Self-host (AGPL-3.0): docker run -p 3000:3000 ghcr.io/us/crw:latest. Unlimited requests, no license fee, data never leaves your infra. Point the Firecrawl SDK at http://your-host:3000.
• Managed Cloud: if you don't want to run infrastructure, use the hosted version — same API, proxy network, higher concurrency. Free tier is a one-time lifetime 500 credits (not monthly), then paid tiers that undercut Firecrawl tier-for-tier.

The strategic advantage of choosing an open-core target: if you ever dislike the managed pricing, you self-host the exact same engine with zero further migration. The escape hatch is permanent.

Rollback plan

Because the migration is a config change, rollback is too. Keep both client configs available behind an environment variable for the first week. If parity checks regress in production, flip the variable back to Firecrawl, redeploy config (no code change), and investigate offline. A migration you can reverse in 60 seconds is a migration you can do on a Tuesday.

A realistic migration timeline

Teams overestimate this because they pattern-match it to a typical vendor migration. It is not. Here is a realistic timeline for a mid-sized production pipeline:

• Day 1 (1–2 hours): Stand up the target (one docker run for self-host, or sign up for the managed cloud). Point a scratch script's api_url at it. Run your ten most common scrape calls. Confirm they return.
• Day 1–2 (half a day): Run the parity harness on 50–100 real URLs. Triage divergences against the known list (fields, error envelope, any cloud-only Firecrawl path you use). Most teams find the divergence list is short and boring.
• Day 2–3: Wrap the SDK in the thin adapter if you have not already, move the base URL into config, add the backend label to telemetry. This is the only actual code change, and it is the kind you should have made anyway.
• Day 3–7: 5% shadow, then 25% live, watching error rate and latency. No user impact at shadow; reversible at 25%.
• Day 7–10: 100% cutover, old config behind a flag for one week, then decommission.

The dominant cost is validation time, not engineering time. That is the inverse of a normal migration and it is entirely due to API compatibility doing the heavy lifting.

What to measure to know the migration succeeded

"It still works" is not a success criterion. Define these before you start so the decision is data, not vibes:

• Content parity rate: percentage of your validation URLs where extracted main content matches within tolerance. Set a bar (e.g. ≥95%) and hold to it.
• Latency delta: p50 and p95 on your traffic mix, new vs old. A migration that improves cost but regresses p95 into user-visible territory is not a clean win — know the trade.
• Error-rate delta: normalized by error class (4xx vs 5xx vs timeout vs empty-content). A shift in the mix of errors is as informative as the total.
• Cost delta: projected monthly bill, including the extract line if you extract. This is usually the reason you migrated, so quantify it explicitly rather than assuming.

Capture these as a one-page before/after. It is the artifact that justifies the migration to whoever owns the budget, and the baseline you will want if you ever need to migrate again.

Common migration mistakes to avoid

1. Testing only example.com. Toy URLs hide every interesting divergence. Validate on the messy real sites your pipeline actually hits.
2. Snapshot-asserting whole payloads. Brittle and guaranteed to "fail" on cosmetic metadata differences. Assert the fields you consume.
3. Migrating during a traffic peak. Do it on a quiet window with rollback staged, not the day before a launch.
4. Skipping the extract re-expression. If you used a dedicated extract endpoint, re-express it as a scrape-with-JSON early; do not discover at cutover that you never tested it.
5. Not keeping the escape hatch. The whole point of migrating to a Firecrawl-compatible target is reversibility. Keep your client backend-neutral so the next move is also one line.

Sources

• fastCRW repo and Firecrawl compatibility notes: github.com/us/crw
• Firecrawl SDK docs: docs.firecrawl.dev

Related: Self-host a Firecrawl-like API · Firecrawl vs Crawl4AI vs CRW