LangGraph Web Scraping Integration — fastCRW [Firecrawl-Compatible]

Why LangGraph + fastCRW

LangGraph is the right primitive when an agent needs durable state and a reasoning loop with explicit edges. The weakness is that every browse step inside a graph node compounds latency and memory cost. fastCRW is the smallest viable scraping primitive for LangGraph — a 6.6 MB RAM runtime that returns clean Markdown fast enough that a LangGraph node never becomes the bottleneck. Because fastCRW is Firecrawl-compatible, any LangGraph project that already uses Firecrawl can swap in fastCRW with one base-URL change and keep the rest of the graph unchanged.

Setup

1. Install LangGraph and langchain-core for tool definitions.
2. Provision a fastCRW API key from the dashboard.
3. Export FASTCRW_API_KEY in your environment.
4. Import the standard ToolNode and bind a fastCRW tool function.

pip install -U langgraph langchain-core requests
export FASTCRW_API_KEY="fcrw_..."

LangGraph treats fastCRW as a normal tool. There is no LangGraph-specific package to install for fastCRW.

Code Example

import os
import requests
from typing import Annotated, TypedDict
from langchain_core.tools import tool
from langgraph.graph import StateGraph, END
from langgraph.graph.message import add_messages
from langgraph.prebuilt import ToolNode
from langchain_anthropic import ChatAnthropic

FASTCRW_BASE = "https://api.fastcrw.com"

@tool
def fastcrw_scrape(url: str) -> str:
    """Scrape a single URL via fastCRW and return the Markdown."""
    r = requests.post(
        f"{FASTCRW_BASE}/v1/scrape",
        headers={"Authorization": f"Bearer {os.environ['FASTCRW_API_KEY']}"},
        json={"url": url, "formats": ["markdown"]},
        timeout=60,
    )
    r.raise_for_status()
    return r.json()["data"]["markdown"]

@tool
def fastcrw_search(query: str) -> list[dict]:
    """Web search via fastCRW. Returns ranked results."""
    r = requests.post(
        f"{FASTCRW_BASE}/v1/search",
        headers={"Authorization": f"Bearer {os.environ['FASTCRW_API_KEY']}"},
        json={"query": query, "limit": 5},
        timeout=60,
    )
    r.raise_for_status()
    return r.json()["data"]

class AgentState(TypedDict):
    messages: Annotated[list, add_messages]

tools = [fastcrw_scrape, fastcrw_search]
llm = ChatAnthropic(model="claude-opus-4-7").bind_tools(tools)

def call_model(state: AgentState):
    return {"messages": [llm.invoke(state["messages"])]}

graph = StateGraph(AgentState)
graph.add_node("model", call_model)
graph.add_node("tools", ToolNode(tools))
graph.set_entry_point("model")
graph.add_conditional_edges(
    "model",
    lambda s: "tools" if s["messages"][-1].tool_calls else END,
)
graph.add_edge("tools", "model")

app = graph.compile()

When to Use This

• Stateful research agents — multi-step graphs that interleave search, scrape, and reasoning over a persistent state.
• Human-in-the-loop pipelines — pause a LangGraph run for review, then resume the scrape step against fastCRW.
• Map-then-extract workflows — one node maps a domain, another scrapes per URL, another extracts structured fields.
• Replacing heavier scrapers in LangGraph — keep the graph topology and swap the tool implementation to fastCRW for lower step latency.

Limits + Gotchas

• LangGraph step budgets need to account for scrape latency. Set per-tool timeouts to avoid stalling the graph.
• Tool errors must be returned as messages so the model can recover. Catch HTTP errors inside the fastCRW tool functions.
• Long crawls do not fit cleanly inside a single LangGraph node. Run them as background jobs and feed results back through state.
• LangGraph ToolNode expects deterministic tool names. Keep fastcrw_scrape and fastcrw_search stable across deploys to avoid breaking checkpointed graphs.

Related

• LangChain integration
• fastCRW search docs
• fastCRW scrape docs
• Firecrawl alternative