Langflow Web Scraping Integration — fastCRW [Firecrawl-Compatible]

Why Langflow + fastCRW

Langflow is the visual builder for LangChain-style flows. Teams that prefer flowchart UX over code reach for Langflow when they want to ship a RAG pipeline or a tool-using agent without writing a Python file. The gap is web scraping — Langflow does not ship a robust scraper out of the box. fastCRW slots in as either a custom component or an API Request node, returning clean Markdown that flows naturally into Langflow's text splitters and vector stores. The fastCRW runtime is a small single static binary that is local-first, which makes it cheap to run alongside a self-hosted Langflow instance, and the Firecrawl-compatible API means existing Langflow flows that target Firecrawl can swap in fastCRW with one URL change.

Setup

1. Install or update Langflow (pip install -U langflow).
2. Provision a fastCRW API key from the dashboard.
3. Decide whether to use the built-in API Request component or to add a custom fastCRW component to Langflow's components folder.
4. Drag the component onto the canvas and bind the API key.

pip install -U langflow
export FASTCRW_API_KEY="fcrw_..."
langflow run

Code Example

A minimal Langflow custom component for fastCRW scrape. Drop this Python file into your Langflow custom components directory:

import requests
from langflow.custom import Component
from langflow.io import StrInput, SecretStrInput, Output
from langflow.schema import Data

class FastCRWScrapeComponent(Component):
    display_name = "fastCRW Scrape"
    description = "Scrape a URL via fastCRW and return Markdown."
    icon = "globe"

    inputs = [
        StrInput(name="url", display_name="URL", required=True),
        SecretStrInput(name="api_key", display_name="fastCRW API Key", required=True),
        StrInput(
            name="base_url",
            display_name="Base URL",
            value="https://api.fastcrw.com",
        ),
    ]

    outputs = [
        Output(display_name="Markdown", name="markdown", method="scrape"),
    ]

    def scrape(self) -> Data:
        r = requests.post(
            f"{self.base_url}/v1/scrape",
            headers={"Authorization": f"Bearer {self.api_key}"},
            json={"url": self.url, "formats": ["markdown"]},
            timeout=60,
        )
        r.raise_for_status()
        markdown = r.json()["data"]["markdown"]
        return Data(data={"markdown": markdown, "source": self.url})

If you do not want a custom component, drag the API Request component onto the canvas and configure:

• Method: POST
• URL: https://api.fastcrw.com/v1/scrape
• Headers: Authorization: Bearer fcrw_...
• Body: {"url": "{{ input }}", "formats": ["markdown"]}

A canonical Langflow RAG flow then looks like: Chat Input → fastCRW Scrape → Recursive Character Splitter → Embeddings → Vector Store → Retriever → Prompt → Chat Output. The same shape works for fastCRW search by switching the endpoint.

When to Use This

• Visual RAG builds — drag-and-drop ingestion from URLs into a vector store via fastCRW.
• Prototype agents — let non-engineers wire up a fastCRW-powered research agent in Langflow.
• Internal tools — package a Langflow flow with fastCRW into a Langflow Endpoint for chat-style scraping.
• Migrating Firecrawl flows — replace existing Firecrawl components with fastCRW custom components for lower runtime cost.

Limits + Gotchas

• Langflow custom components are picked up at startup. Restart the Langflow process after dropping in the fastCRW component file.
• The SecretStrInput field encrypts the key at rest in Langflow but exposes it inside the running flow — treat your fastCRW key like any other production secret.
• The API Request component does not auto-parse error JSON. Add a guard in downstream components for non-200 responses.
• Long fastCRW crawls can stall Langflow's UI run. Use scrape per URL inside a Langflow For-Each component instead of a single deep crawl call.

Related

• LangChain integration
• LangGraph integration
• fastCRW scrape docs
• Firecrawl alternative