Overview

If your agent runtime already speaks MCP, you can expose fastCRW as a standard tool instead of wrapping the HTTP API yourself.

{
  "mcpServers": {
    "crw": {
      "command": "crw",
      "args": ["mcp"]
    }
  }
}

When MCP Helps

MCP is useful when the agent host already expects tools to be registered through a standard interface. That reduces one layer of custom glue code between the agent and your scraping service.

Typical fits:

agentic research workflows,
internal copilots that need current website content,
multi-tool assistants that combine search, scrape, and synthesis,
and developer environments such as Claude or Cursor where MCP is already the preferred integration path.

Why It Matters

MCP gives agents a consistent way to discover and call tools. That matters when the workflow moves from finding URLs to scraping content and handing the result to downstream retrieval or reasoning steps.

A Practical Workflow

One common setup looks like this:

the agent identifies a site or page it needs,
it calls an MCP-exposed fastCRW tool,
fastCRW returns scrape, map, or crawl output,
the agent decides whether to continue exploring or move into summarization, ranking, or retrieval.

The value is not just convenience. It also keeps the boundary clear: fastCRW fetches and extracts, while the agent decides what to do next.

Hosted vs Self-Hosted

MCP is an integration layer, not a hosting mode. You can use the same general approach whether fastCRW is:

running as the managed cloud API,
running inside your own infrastructure,
or wrapped into a broader internal platform.

Operational Notes

Keep MCP tool descriptions tight so the agent knows when to use map versus scrape.
Start with read-only scraping tools before exposing anything more complex in the same MCP server.
Log tool usage separately from downstream agent reasoning so debugging stays tractable.

MCP Integration Guide