Firecrawl API Compatibility: What 'Drop-in Compatible' Actually Means (2026)

By the fastCRW team · Last reviewed 2026-05-18

Disclosure: fastCRW is a Firecrawl-compatible scraper built by the author. This is an engineering post about API surface compatibility — the kind of detail you need before you trust "drop-in" claims in production.

What "compatible" should mean (and what it shouldn't)

"Drop-in compatible" is an overused phrase. For a scraping API it should mean something precise: a client written against Firecrawl's REST surface keeps working when you change only the base URL, for the endpoints both products implement. It should not mean "every Firecrawl feature, byte-for-byte." Anyone claiming total parity with a closed, fast-moving product is overselling. The honest claim is: the overlap surface — scrape, crawl, map, search — is compatible enough that migration is a base-URL change plus a short, known validation list.

The overlap surface

fastCRW implements the Firecrawl-shaped endpoints that the vast majority of production pipelines actually use:

Because this surface is where RAG ingestion, agent web access, crawling, and enrichment all live, "compatible on the overlap surface" covers most real workloads end to end.

The request shape

The scrape request is the one you'll touch most. The Firecrawl-shaped contract fastCRW honors:

POST /v1/scrape
Content-Type: application/json
Authorization: Bearer YOUR_KEY

{
  "url": "https://example.com/article",
  "formats": ["markdown", "html"]
}

formats controls output: markdown for LLM-ready content, html for the cleaned DOM, and a JSON format for structured extraction against a schema. The key compatibility point: the field that selects structured extraction lives inside the scrape call on fastCRW — there is no separate extract product or separate subscription. If your Firecrawl code calls a dedicated extract endpoint, you re-express it as a scrape requesting the JSON format. That is the single most common rewrite, and it makes the integration simpler, not harder.

The response shape

A scrape response carries the content under the requested formats plus metadata:

{
  "success": true,
  "data": {
    "markdown": "# Article Title\n\nBody text...",
    "metadata": {
      "title": "Article Title",
      "sourceURL": "https://example.com/article",
      "statusCode": 200
    }
  }
}

Defensive client design matters here. Read the specific fields you consume (data.markdown, data.metadata.statusCode) rather than asserting deep equality on the whole envelope. A small number of metadata fields and the exact error-envelope shape can differ between products; code that reads what it needs survives those differences, code that snapshots the entire payload does not.

The crawl job model

Crawl is asynchronous on both: you POST a crawl, get a job id, then poll. The compatible pattern:

# 1. start
POST /v1/crawl  -> { "id": "job_abc" }

# 2. poll until done
GET /v1/crawl/job_abc
-> { "status": "scraping", "completed": 12, "total": 80, "data": [ ... ] }
-> { "status": "completed", "completed": 80, "total": 80, "data": [ ... ] }

When validating a migration, assert on status transitions and that data accumulates page documents in the same shape as scrape. Don't hardcode assumptions about pagination chunk sizes or the exact intermediate status strings beyond the start/working/done lifecycle.

Where surfaces diverge (be honest)

• Cloud-only Firecrawl specialties: heavy anti-bot fire-engine paths and certain agentic/research endpoints are Firecrawl-cloud features. fastCRW's open-core engine handles most sites including JS rendering, but if you specifically depend on a cloud-only Firecrawl path, test that exact path — don't assume parity.
• Error envelopes: the JSON shape of error responses can differ in field naming. Classify errors by HTTP status first, payload fields second.
• Metadata extras: some optional metadata keys are product-specific. Treat anything beyond core fields (title, sourceURL, statusCode) as best-effort.
• Credit accounting: the concept of credits is shared but the per-feature accounting differs (notably, fastCRW folds JSON extraction into the scrape credit rather than billing a separate subscription). Read response credit metadata; don't port cost assumptions.

Writing migration-resilient client code

Whether or not you ever migrate, these patterns make your scraping client portable across compatible backends:

1. Inject the base URL. Never hardcode https://api.firecrawl.dev. Read it from config/env so swapping backends is zero code change.
2. Wrap the SDK in a thin adapter. Your app calls scrape(url, opts); the adapter calls the SDK. Divergences get handled in one file.
3. Validate fields, not payloads. Assert the data you read. Ignore fields you don't.
4. Classify errors by status code. 4xx vs 5xx vs timeout is portable; exact error JSON is not.
5. Pin output expectations loosely. Markdown will differ in whitespace between engines; assert on presence of key content, not exact strings.

Code written this way isn't just easier to migrate — it's easier to test, mock, and reason about regardless of backend.

Why compatibility is the strategic point

For a closed, hosted product, the entrenched client code is the retention moat. API compatibility deliberately dissolves that moat: if leaving is a base-URL change, the vendor has to keep earning your bill on price, speed, and trust every month rather than on switching friction. fastCRW leans into this on purpose — the open-core engine (single ~6MB Rust binary, AGPL-3.0, self-hostable with unlimited requests) plus the Firecrawl-compatible surface means your escape hatch is always one config line away, to self-host or to the managed cloud.

A 10-minute compatibility test you can run today

1. Stand up the engine locally: docker run -p 3000:3000 ghcr.io/us/crw:latest.
2. Point your existing Firecrawl SDK at http://localhost:3000.
3. Run your real scrape + crawl + map calls against it.
4. Diff outputs vs Firecrawl on 20 representative URLs.
5. Note any divergence against the known list above. That delta is your actual migration scope — usually small.

A taxonomy of compatibility, so the word stops being marketing

"Compatible" collapses several distinct guarantees. Pull them apart and the claim becomes testable:

• Wire compatibility: same HTTP method, path, and request body shape. This is what lets the SDK work unchanged after a base-URL swap. fastCRW targets this on the overlap surface.
• Semantic compatibility: the same request produces the same meaning of result — a scrape returns the page's main content as markdown, a crawl returns the site's pages. High on the overlap surface; the engines differ in cleaning heuristics, so byte-equality is never the bar.
• Operational compatibility: the job lifecycle behaves the same (submit → poll → terminal for crawl; synchronous for scrape). This is what makes your control flow portable.
• Economic compatibility: deliberately not claimed. The credit accounting differs on purpose — most importantly, fastCRW folds JSON extraction into the per-page scrape credit instead of a separate subscription. Do not port cost assumptions; re-derive them.

When a vendor says "drop-in compatible," ask which of these four they mean. The honest answer for a closed, fast-moving reference product is "wire + semantic + operational on the overlap surface; economics intentionally different." Anything claiming all four byte-for-byte across a moving target is overselling.

The compatibility contract you should hold a backend to

If you are evaluating any Firecrawl-compatible engine, here is the minimum contract worth testing rather than trusting:

1. SDK-unchanged scrape: the official Firecrawl SDK, only api_url changed, returns usable markdown for your real URLs.
2. Crawl lifecycle: submit returns an id; polling transitions through a working state to a terminal state; results accumulate page documents in the scrape shape.
3. Map fidelity: discovered link sets substantially overlap for the same seed.
4. Structured JSON: a schema-driven scrape returns a typed object whose required fields are populated at a comparable rate.
5. Error classifiability: failures map cleanly onto HTTP status classes so your existing handling keeps working.

If a backend passes all five on your traffic, "compatible" is earned, not asserted. If it fails one, you now know your exact migration scope — which is the entire value of testing the contract instead of believing the adjective.

Why the maintainer of a compatible engine is incentivized to keep it compatible

There is a structural reason to trust ongoing compatibility from an open-core challenger specifically. Its entire acquisition strategy is "frustrated reference-product user changes one line." That only works if the one-line change keeps working. Divergence on the overlap surface would directly destroy the challenger's primary growth mechanism, so compatibility is not a courtesy — it is load-bearing for the business. Combined with the engine being open source (auditable single ~6MB Rust binary, AGPL-3.0), you can verify the compatibility claim yourself rather than take it on faith, which is the only kind of compatibility claim worth relying on in production.

Sources

• fastCRW repo: github.com/us/crw
• Firecrawl API reference: docs.firecrawl.dev

Related: Migrate from Firecrawl · Self-host a Firecrawl-like API