n8n Web Scraping Integration — fastCRW [Firecrawl-Compatible]

Why n8n + fastCRW

n8n is the workflow automation tool engineering teams pick when Zapier hits its ceiling. The friction point is web scraping — n8n's built-in HTML parsing covers static pages but breaks on JavaScript-heavy sites and rate-limited targets. fastCRW fits inside n8n through the official n8n-nodes-crw community node or the standard HTTP Request node and turns any n8n workflow into a production-grade scraper. The fastCRW runtime is a small single static binary, local-first and easy to deploy next to a self-hosted n8n instance, and the Firecrawl-compatible API means existing n8n flows that target Firecrawl port to fastCRW with one URL change.

Setup

1. Open your n8n instance and create a new workflow.
2. Add credentials of type Header Auth with header name Authorization and value Bearer fcrw_....
3. Add an HTTP Request node and bind the credential.
4. Configure the node to POST to https://api.fastcrw.com/v1/scrape with a JSON body.

The recommended path is the official n8n-nodes-crw community node (install it via Settings → Community Nodes), which exposes native fastCRW scrape, search, crawl, and map actions with no manual HTTP wiring. The HTTP Request approach below works as a fallback when community nodes are disabled (e.g. some n8n cloud plans).

Code Example

n8n HTTP Request node configuration for a fastCRW scrape:

• Method: POST
• URL: https://api.fastcrw.com/v1/scrape
• Authentication: Header Auth (Authorization: Bearer fcrw_...)
• Body Content Type: JSON
• JSON Body:

{
  "url": "{{ $json.targetUrl }}",
  "formats": ["markdown"]
}

For an n8n AI Agent node calling fastCRW search as a tool:

{
  "query": "{{ $fromAI('query') }}",
  "limit": 5
}

POST that to https://api.fastcrw.com/v1/search from a Tool subnode under the AI Agent and the agent can search the live web through fastCRW. A typical n8n flow chains: Trigger → HTTP Request (fastCRW scrape) → Set (extract Markdown) → Vector Store node, which is the n8n equivalent of a LangChain ingestion chain.

For workflows that scrape lists of URLs, use the n8n Loop Over Items node to iterate and call fastCRW per row. fastCRW handles concurrency at the API layer, so n8n loops stay simple.

When to Use This

• No-code RAG ingestion — pull marketing pages or docs into a vector store from inside n8n.
• Lead enrichment workflows — scrape company URLs from a CRM trigger and write the structured output back.
• Competitor monitoring — schedule a daily n8n workflow to scrape pricing pages via fastCRW and diff against the last run.
• AI Agent browsing — give the n8n AI Agent node a fastCRW tool so it can fetch URLs during reasoning.

Limits + Gotchas

• The HTTP Request node's default timeout is 5 minutes. For long fastCRW crawls, increase the node timeout or use an async crawl pattern with polling.
• n8n cloud has request body size limits — large scrape responses (huge pages) may need to be written straight to S3 instead of held in workflow state.
• Header Auth credentials encode the API key in plaintext at rest in n8n. Use n8n's credential rotation rather than hardcoding keys in nodes.
• fastCRW rate limits apply per API key. Run the n8n Loop Over Items node with a sensible batch size to avoid throttling.

Related

• Make integration
• Zapier integration
• fastCRW scrape docs
• n8n web scraping guide