MCP Web Scraping Integration — fastCRW [Firecrawl-Compatible]

Why MCP + fastCRW

The Model Context Protocol (MCP) is the emerging standard for connecting LLM hosts to external tools and data. Anthropic shipped MCP, the major IDEs and assistants adopted it, and the protocol now plays the role HTTP played for early SaaS — a shared substrate that lets any client call any compatible server. fastCRW publishes an official MCP server so any MCP-compatible host can scrape, search, crawl, map, and extract through a single tool surface. The fastCRW runtime is a small single static binary that is local-first, which is what makes the MCP story practical: the server is small enough to run inside a developer's editor process, and the underlying API is small enough to self-host alongside private MCP infrastructure.

Setup

1. Install Node.js 20+ on the machine that will run the MCP server.
2. Provision a fastCRW API key from the dashboard.
3. Export FASTCRW_API_KEY in the environment of the MCP host.
4. Register the fastCRW MCP server in your client's MCP config.

export FASTCRW_API_KEY="fcrw_..."
npx -y crw-mcp --help

The server uses stdio transport by default, which is what every desktop MCP host expects.

Code Example

Generic MCP client config (works for Claude Code, Cursor, Windsurf, Continue, Zed):

{
  "mcpServers": {
    "fastcrw": {
      "command": "npx",
      "args": ["-y", "crw-mcp"],
      "env": {
        "FASTCRW_API_KEY": "${FASTCRW_API_KEY}"
      }
    }
  }
}

For a self-hosted fastCRW instance, override the base URL:

{
  "mcpServers": {
    "fastcrw": {
      "command": "npx",
      "args": ["-y", "crw-mcp"],
      "env": {
        "FASTCRW_API_KEY": "${FASTCRW_API_KEY}",
        "FASTCRW_BASE_URL": "https://crw.internal.company.com"
      }
    }
  }
}

For a remote MCP deployment over HTTP you have two options. Use the managed hosted connector — point any remote-MCP client at https://fastcrw.com/mcp/<YOUR_API_KEY> with no install (see MCP Client Setup). Or self-host: run crw-server, which serves the Streamable HTTP transport at POST /mcp, behind a reverse proxy. (The crw-mcp npm package is a stdio launcher — it has no HTTP-transport flag.) The MCP tools exposed are:

• crw_scrape — fetch a URL and return Markdown, HTML, or links.
• crw_search — web search returning ranked results.
• crw_crawl — async multi-page crawl from a seed URL; returns a job ID.
• crw_check_crawl_status — poll a crawl job for results.
• crw_map — discover URLs on a site via sitemap and link crawl.

A minimal custom MCP client that connects to fastCRW with the official Python SDK:

import asyncio
from mcp import ClientSession, StdioServerParameters
from mcp.client.stdio import stdio_client

async def main():
    params = StdioServerParameters(
        command="npx",
        args=["-y", "crw-mcp"],
        env={"FASTCRW_API_KEY": "fcrw_..."},
    )
    async with stdio_client(params) as (read, write):
        async with ClientSession(read, write) as session:
            await session.initialize()
            tools = await session.list_tools()
            print([t.name for t in tools.tools])
            result = await session.call_tool(
                "scrape",
                {"url": "https://example.com"},
            )
            print(result.content[0].text[:500])

asyncio.run(main())

When to Use This

• Editor-side browsing — Claude Code, Cursor, Windsurf, and Zed users get a unified scrape and search tool through the fastCRW MCP server.
• Custom MCP hosts — internal tools built on the MCP SDK can call fastCRW without hand-rolling HTTP clients.
• Multi-tool agents — combine fastCRW MCP with other MCP servers (filesystem, database, GitHub) in a single host configuration.
• Self-hosted MCP — run the fastCRW MCP server with HTTP transport behind your own auth boundary for enterprise deployments.

Limits + Gotchas

• MCP tool schemas are negotiated at session start. If you upgrade the fastCRW MCP server, restart the host process to pick up new tool definitions.
• Stdio transport requires the host process to spawn the server. Sandboxed hosts may block npx — pin a binary or use the HTTP transport instead.
• Tool outputs flow into the LLM's context. fastCRW responses can be large — instruct the host to summarize before quoting full pages.
• The MCP spec is still evolving. Lock the crw-mcp version in production and watch the changelog for breaking protocol revisions.

Related

• Claude Code integration
• fastCRW MCP docs
• fastCRW scrape docs
• Firecrawl alternative