Langflow Web Scraping Integration — fastCRW [Firecrawl-Compatible]

Why Langflow + fastCRW

Langflow is the visual builder for LangChain-style flows. Teams that prefer flowchart UX over code reach for Langflow when they want to ship a RAG pipeline or a tool-using agent without writing a Python file. The gap is web scraping — Langflow does not ship a robust scraper out of the box. fastCRW slots in as either a custom component or an API Request node, returning clean Markdown that flows naturally into Langflow's text splitters and vector stores. The fastCRW runtime is 6.6 MB of RAM, which makes it cheap to run alongside a self-hosted Langflow instance, and the Firecrawl-compatible API means existing Langflow flows that target Firecrawl can swap in fastCRW with one URL change.

Setup

1. Install or update Langflow (pip install -U langflow).
2. Provision a fastCRW API key from the dashboard.
3. Decide whether to use the built-in API Request component or to add a custom fastCRW component to Langflow's components folder.
4. Drag the component onto the canvas and bind the API key.

pip install -U langflow
export FASTCRW_API_KEY="fcrw_..."
langflow run

Code Example

A minimal Langflow custom component for fastCRW scrape. Drop this Python file into your Langflow custom components directory:

import requests
from langflow.custom import Component
from langflow.io import StrInput, SecretStrInput, Output
from langflow.schema import Data

class FastCRWScrapeComponent(Component):
    display_name = "fastCRW Scrape"
    description = "Scrape a URL via fastCRW and return Markdown."
    icon = "globe"

    inputs = [
        StrInput(name="url", display_name="URL", required=True),
        SecretStrInput(name="api_key", display_name="fastCRW API Key", required=True),
        StrInput(
            name="base_url",
            display_name="Base URL",
            value="https://api.fastcrw.com",
        ),
    ]

    outputs = [
        Output(display_name="Markdown", name="markdown", method="scrape"),
    ]

    def scrape(self) -> Data:
        r = requests.post(
            f"{self.base_url}/v1/scrape",
            headers={"Authorization": f"Bearer {self.api_key}"},
            json={"url": self.url, "formats": ["markdown"]},
            timeout=60,
        )
        r.raise_for_status()
        markdown = r.json()["data"]["markdown"]
        return Data(data={"markdown": markdown, "source": self.url})

If you do not want a custom component, drag the API Request component onto the canvas and configure:

• Method: POST
• URL: https://api.fastcrw.com/v1/scrape
• Headers: Authorization: Bearer fcrw_...
• Body: {"url": "{{ input }}", "formats": ["markdown"]}

A canonical Langflow RAG flow then looks like: Chat Input → fastCRW Scrape → Recursive Character Splitter → Embeddings → Vector Store → Retriever → Prompt → Chat Output. The same shape works for fastCRW search by switching the endpoint.

When to Use This

• Visual RAG builds — drag-and-drop ingestion from URLs into a vector store via fastCRW.
• Prototype agents — let non-engineers wire up a fastCRW-powered research agent in Langflow.
• Internal tools — package a Langflow flow with fastCRW into a Langflow Endpoint for chat-style scraping.
• Migrating Firecrawl flows — replace existing Firecrawl components with fastCRW custom components for lower runtime cost.

Limits + Gotchas

• Langflow custom components are picked up at startup. Restart the Langflow process after dropping in the fastCRW component file.
• The SecretStrInput field encrypts the key at rest in Langflow but exposes it inside the running flow — treat your fastCRW key like any other production secret.
• The API Request component does not auto-parse error JSON. Add a guard in downstream components for non-200 responses.
• Long fastCRW crawls can stall Langflow's UI run. Use scrape per URL inside a Langflow For-Each component instead of a single deep crawl call.

Related

• LangChain integration
• LangGraph integration
• fastCRW scrape docs
• Firecrawl alternative