Claude Code Web Scraping Integration — fastCRW [MCP Server]

Verdict

Claude Code is the agentic coding environment that turns Anthropic's models into a teammate which reads the repo, runs commands, and reasons about external context — but it ships no built-in scraper. fastCRW closes that gap as an MCP server. Once registered with one npx command, fastCRW exposes scrape, search, crawl, map, and crawl-status as native Claude Code tools, so any session, Skill, or slash command can pull live web context. The runtime is a single ~8 MB static Rust binary that is local-first, which makes it sane to run fastCRW right next to Claude Code on a laptop instead of routing every request through a heavyweight cloud scraper.

Who This Is For

• Developers who want grounded answers — let Claude Code scrape API references and changelogs instead of guessing from training data.
• Skill and slash-command authors — encode a repeatable search-then-scrape research routine the whole team can invoke.
• On-call engineers — have the agent pull vendor status pages and advisories during an incident without context-switching.
• Self-hosting / compliance-bound teams — point Claude Code at a private fastCRW instance to scrape internal wikis and dashboards on-premise.

Setup

1. Provision a fastCRW API key

Sign up at fastcrw.com, copy the API key from the dashboard (it starts with fcrw_), and export it:

export FASTCRW_API_KEY="fcrw_..."

2. Register the MCP server

The fastCRW MCP server ships as the crw-mcp npm package. Register it with Claude Code in one command:

claude mcp add fastcrw -- npx -y crw-mcp

Claude Code passes your shell environment through to the server, so FASTCRW_API_KEY is picked up automatically. Restart Claude Code, then run /mcp — you should see fastcrw listed with its scrape, search, crawl, map, and crawl-status tools.

3. Or use a project-scoped .mcp.json

For a setup every collaborator shares, commit a project-scoped .mcp.json at the repo root instead of registering the server per-machine:

{
  "mcpServers": {
    "fastcrw": {
      "command": "npx",
      "args": ["-y", "crw-mcp"],
      "env": {
        "FASTCRW_API_KEY": "${FASTCRW_API_KEY}"
      }
    }
  }
}

Reference the key as ${FASTCRW_API_KEY} rather than hardcoding it — that keeps the secret out of version control while the config stays shareable.

4. Self-hosted fastCRW

To point Claude Code at your own fastCRW instance, add FASTCRW_BASE_URL to the env block:

{
  "mcpServers": {
    "fastcrw": {
      "command": "npx",
      "args": ["-y", "crw-mcp"],
      "env": {
        "FASTCRW_API_KEY": "${FASTCRW_API_KEY}",
        "FASTCRW_BASE_URL": "https://crw.internal.company.com"
      }
    }
  }
}

Using fastCRW Inside a Session

Once the server is registered, Claude Code picks fastCRW tools by name when a prompt calls for live web data. A few patterns:

Read live documentation. Ask the agent to "scrape the latest MCP spec from modelcontextprotocol.io and reconcile it with our server implementation" — Claude Code invokes the scrape tool, fetches clean Markdown, and reasons over it.

Research a topic. "Search for recent posts on prompt-caching strategies, scrape the three most relevant, and summarize the consensus" — the agent chains search then scrape, no glue code required.

Discover a site's shape. "Map docs.example.com so we know which pages exist before I write the crawler" — map returns the URL inventory.

Ingest a docs section. For a whole section, the agent starts a bounded crawl (async — it returns a job ID) and polls it with crawl-status. Because crawl is async, run it as a discrete step rather than blocking the agent loop on it.

Structured extraction

There is no separate extract tool over MCP. To pull schema-shaped JSON instead of Markdown, the scrape tool accepts a json format with a jsonSchema — for example, "scrape careers.example.com/jobs and return each posting as JSON with title, team, and location". Note that any request using the json format adds the managed-LLM token cost on top of the 1 scrape credit (settled from real usage), so reserve it for genuinely structured data.

Skills and Slash Commands

The highest-leverage use is encoding a routine as a Claude Code Skill or slash command. A /research command can fan out a fastCRW search, scrape the top hits, and return a cited synthesis — turning a multi-step pattern into one invocation. Skills that browse work the same way: any Skill that needs web context simply calls the registered fastCRW tools, and subagents inherit the same MCP surface.

Why fastCRW for Laptop-Side MCP

Running an MCP scraper next to the editor only makes sense if the scraper is light. fastCRW is a single ~8 MB static Rust binary — no Redis, no Node.js runtime, no container stack — against a Firecrawl deployment's five containers. On Firecrawl's own public 1,000-URL scrape-content-dataset-v1 (diagnose_3way.py harness, run 2026-05-08), fastCRW recovered the labeled content on 63.74% of 819 labeled URLs, the highest of three tools tested, at a p50 latency of 1914 ms — the fastest median of the three. In fast mode, fastCRW p90 is 4348ms — the lowest of the three. The full p50/p90/p99 split is on the benchmark page.

Limits + Gotchas

• MCP tool calls run inside Claude Code's tool budget. Long crawls can exhaust step budgets — prefer scrape and search in-loop, and run crawls as separate jobs polled with crawl-status.
• The server reads FASTCRW_API_KEY from the environment. Use a per-machine secret manager or ${FASTCRW_API_KEY} interpolation rather than committing keys in .mcp.json.
• Claude Code caches MCP tool definitions per session. After upgrading crw-mcp, restart Claude Code to refresh the tool list.
• fastCRW responses can be large. Ask the agent to summarize before quoting full pages so the context window stays useful.
• Structured JSON extraction via the scrape tool's json format adds the managed-LLM token cost on top of the 1 scrape credit, settled from real usage; plain Markdown scraping is 1 credit.

Related

• MCP integration
• Cursor integration
• fastCRW MCP docs
• fastCRW scrape docs
• fastCRW crawl docs
• Firecrawl alternative