Blog category
Tutorial
Hands-on tutorials for scraping workflows, RAG ingestion, MCP setups, and production integrations with fastCRW.
Build a Chat-With-Website Bot (fastCRW + LangChain)
Build a chat-with-website bot with fastCRW and LangChain: crawl a site to clean markdown, chunk, embed, and answer questions over it. Full Python tutorial.
SERP Scraping in 2026: Search the Web With CRW's Search API (Python)
Stop scraping Google's HTML. Use CRW's /v1/search to get ranked results plus full page content in one call. Build a SERP monitor and keyword rank tracker — runnable Python, self-host free under AGPL-3.0.
Structured Web Extraction With JSON Schema and CRW (2026): No CSS Selectors
Extract typed JSON from any page with CRW's /v1/extract and a JSON schema — no CSS selectors, no per-site code. Validate with Pydantic, batch-extract, and handle failures. Runnable Python.
Web Scraping in Elixir: Concurrency on BEAM
Web scraping in Elixir with Req, Floki, and Task.async_stream. BEAM concurrency for fan-out scraping and where a managed scrape API fits the pipeline.
Bash & CLI Web Scraping: One-Off Shell Pipelines
One-off bash and CLI web scraping with curl, pup, and jq. Build interactive shell pipelines, see where they break, and know when to hand off to a scrape API.
Headless Browser Scraping: A Practical Guide
Headless browser scraping in Python with Playwright and Selenium: waiting, infinite scroll, the true cost of a browser fleet, and when to offload to an API.
Build a Competitor Monitoring Tool (Dashboard)
Build a competitor monitoring tool: crawl rival pages on a schedule, diff changes, and surface them in a dashboard. Full Python tutorial with cost math.
Mastra + fastCRW: TypeScript Agents, One Binary
Build Mastra TypeScript agents with live web data via fastCRW. Typed Zod tools, Firecrawl-compatible REST, and a single ~8 MB self-hostable binary.
Build a Jobs Aggregator in Python with CRW (2026): Crawl, Extract, Filter
Aggregate job postings across multiple career pages: crawl with CRW, extract structured roles via JSON schema, normalize, dedupe, and filter by keyword and location. Full Python — AGPL-3.0 self-host.
Build a News Aggregator in Python with CRW (2026): Crawl, Dedupe, Summarize
Build a news aggregator that crawls source homepages with CRW, extracts headlines via JSON schema, dedupes near-duplicates, and writes a daily digest. Full Python — self-host free under AGPL-3.0.
Verify a Firecrawl Drop-In Replacement: Smoke Test
Verify a Firecrawl drop-in replacement with a compatibility smoke test: assert field names, error envelopes, and the divergence matrix before you cut over.
Port a TypeScript Scraper to Python: Skip the Rewrite
Port TypeScript browser automation to Python, or skip the rewrite with a Firecrawl-compatible API. Map Playwright/Puppeteer scripts or call one /v1/scrape.
Migrating from Scrapy to fastCRW: A Practical Guide (2026)
A step-by-step guide to migrating a Scrapy codebase to fastCRW — what maps cleanly, what to keep Scrapy for, incremental migration patterns, and code before/after for spiders, pipelines, and crawls.
Weaviate + fastCRW: Semantic Search From Web
Power Weaviate semantic search with fresh web data: crawl with fastCRW, vectorize clean markdown, and run hybrid search. End-to-end pipeline and credit costs.
Cursor + fastCRW: Live Web Context via MCP
Wire fastCRW into Cursor with the crw-mcp server so your AI coding agent scrapes, crawls, and searches the live web. Setup, config, and credit costs explained.
Sitemap to Crawl: Optimized Discovery at Scale
Go from sitemap to a full crawl on large sites: seed with /v1/map, then cap maxDepth and maxPages. Discovery patterns, caps, concurrency, and credit costs.
Smolagents + fastCRW: Web Grounding, Zero Bloat
Add web search and scraping to Hugging Face smolagents with fastCRW: a single ~8 MB binary keeps the stack lean, plus the highest truth-recall of three tools.
CRW Go Quickstart (2026): Scrape, Crawl, and Search With the HTTP API
A from-zero Go quickstart for CRW using the standard net/http client: scrape to markdown, crawl a site with job polling, map URLs, web search, and a worker-pool batch. Self-host free under AGPL-3.0.
Web Scraping in Go (2026): Goroutines, Backpressure, and When to Stop Building It Yourself
A practical guide to web scraping in Go — colly/goquery, goroutine concurrency and backpressure, the anti-bot wall every Go scraper hits, and how to call a Rust scraping engine from Go without losing the static-binary ethos.
Web Scraping in Java (2026): JSoup, the JVM Footprint Tax, and the Sidecar Pattern
Web scraping in Java for backend teams — JSoup and HtmlUnit, Selenium's heavyweight reality, the JVM memory tax for scrape workers, and why a single-binary scraping sidecar beats fattening your service.
Pointing the Firecrawl SDK at Any Backend: The api_url Swap, Done Right (2026)
A hands-on guide to redirecting the official Firecrawl Python and Node SDKs at a Firecrawl-compatible backend via api_url — including LangChain/LlamaIndex, config patterns, and a parity test harness.
CRW Python Quickstart (2026): Scrape, Crawl, Map, Search in 15 Minutes
A from-zero Python quickstart for CRW: install, scrape a page to markdown, crawl a site, map URLs, search the web, and extract JSON. Async batch included. Self-host free under AGPL-3.0.
Scrape-to-RAG with LlamaIndex and CRW (2026): A Production Ingestion Pipeline
Build a production scrape-to-RAG pipeline: crawl a docs site with CRW, chunk clean markdown, embed with OpenAI, and query with LlamaIndex. Full runnable Python — self-host for $0 under AGPL-3.0.
E-Commerce Stock & Restock Monitoring in Python with CRW (2026)
Build a restock monitor: poll product pages with CRW, extract stock status via JSON schema, detect in-stock transitions, and fire instant alerts. Full runnable Python — self-host free under AGPL-3.0.
Build a Perplexity-Style Search Answer Engine in 50 Lines (with Citations)
fastCRW v0.7.0 ships answer: true on /v1/search — one call gives you a synthesized answer plus validated citations, powered by the managed LLM on paid plans. Full Python and TypeScript tutorial.
fastCRW AI Web Summaries: A Managed-LLM Scrape-Summary Tutorial
Build a production AI web summarizer with fastCRW's managed LLM. Add a summary format to /v1/scrape — no LLM key to manage, usage metered in CRW credits on paid plans. Full Python and TypeScript code.
Build a RAG Pipeline with LangChain and CRW in 5 Minutes
Use langchain-crw to crawl a docs site, chunk the content, embed into a vector store, and answer questions — all with LangChain's native interface.
$5 VPS Web Scraping: Run CRW Where Firecrawl Can't
Deploy a full Firecrawl-compatible scraping API on a $5/month VPS with 512 MB RAM. CRW's tiny single-binary memory footprint makes it possible — here's the complete guide.
How to Build a Job Board Scraper with CRW and OpenAI
Build a job board scraper with CRW and OpenAI — extract listings, match against your resume, and automate your job search.
Exa Search API Guide for AI Agents: Search Types, MCP, Pricing, and Alternatives
A practical guide to the Exa Search API: search types, contents, MCP, pricing, and when fastCRW is a better production choice for AI agents.
How to Build a Web Scraping Agent with LangGraph and CRW
Build a web scraping agent with LangGraph and CRW — graph-based orchestration, state management, and conditional routing.
How to Connect CRW to n8n for Automated Scraping Workflows
Connect n8n to CRW's API for automated web scraping — build scheduled scrapers, data pipelines, and alerts without code.
JavaScript Web Scraping in 2026 — 4 Approaches Tested (Cheerio, Puppeteer, Playwright, fastCRW)
JavaScript web scraping compared: Cheerio (fastest parsing), Puppeteer, Playwright, fastCRW API. Code examples in Node.js + TypeScript with cost, RAM, and reliability tradeoffs. Pick the right tool for your scraper.
How to Build a RAG Chatbot with Langflow and CRW
Build a visual RAG chatbot pipeline in Langflow using CRW as the web scraping data source — no coding required.
How to Automate Web Scraping with Make.com and CRW
Step-by-step guide to building automated web scraping workflows in Make.com using CRW's Firecrawl-compatible API — no code required.
How to Use CRW with Lovable for AI App Prototyping
Build a web app prototype with Lovable's AI app builder that uses CRW/fastCRW for live web scraping — from prompt to working app in minutes.
Add Web Scraping to OpenClaw Agents with CRW
Install the CRW plugin for OpenClaw and give your WhatsApp, Telegram, and Discord AI agents the ability to scrape, crawl, and map any website.
Build a RAG-Powered Research Agent with CrewAI and CRW
Combine crewai-crw web scraping tools with a vector store to build a CrewAI agent that crawls sites, builds a knowledge base, and answers questions with RAG.
How to Build a Lead Enrichment Pipeline with CRW
Build a lead enrichment pipeline that scrapes company websites, extracts structured data like industry, size, and tech stack, and enriches your CRM using CRW.
How to Scrape Cloudflare-Protected Sites with CRW's Stealth Mode
CRW v0.0.11 adds automatic stealth JavaScript injection and Cloudflare challenge retry. Here's how it works under the hood, and how to configure it for maximum success rate.
How to Use CRW with OpenAI Agents SDK for Web-Aware AI
Integrate CRW as a tool in OpenAI's Agents SDK. Build web-aware agents with function calling, handoffs, and real-time web scraping capabilities.
How to Add Web Scraping to Claude Code in 30 Seconds
Give Claude Code web scraping superpowers with CRW's built-in MCP server. One command, zero config — scrape any website directly from your terminal AI assistant.
How to Use CRW with CrewAI for Multi-Agent Web Scraping
Build a CrewAI crew with specialized agents for web scraping and data analysis. Use crewai-crw — the CRW tool package — for fast, clean content extraction.
Browser Automation for AI Agents: Playwright, Stagehand, Browser Use, and APIs (2026)
Playwright, Puppeteer, Stagehand, Browser Use, Browserbase, or a scraping API? A practical guide to browser automation for AI agents in 2026.
Building AI Agents with Google ADK and CRW
Use Google ADK with CRW for web scraping — learn function declarations, tool registration, and Gemini-powered scraping agents.
How to Monitor Competitor Websites with CRW
Set up automated competitor website monitoring with CRW — detect changes, compare snapshots, and generate AI summaries of what your competitors are up to.
Build an AI Price Tracker in Python (2026) — 50 Lines, Zero API Cost [Self-Hosted]
Build an AI price tracker in 50 lines of Python: scrape with fastCRW, extract structured prices via LLM, store in SQLite, alert on drops. AGPL-3.0 self-host, zero per-request cost — full code included.
Web Scraping for Beginners: From Zero to Production (2026)
Beginner-friendly introduction to web scraping — what it is, how it works, legal considerations, tools overview, and hands-on examples with CRW's API.
How to Build a Deep Research Agent with CRW
Build a deep research agent that searches, scrapes, and synthesizes findings into structured reports using CRW's scraping API.
Python Web Scraping: The Complete Guide with CRW (2026)
Python web scraping guide — requests, Beautiful Soup, Scrapy, and the modern API approach with CRW. Code examples included.
How to Self-Host a Firecrawl-Like API with a Single Binary
Run a Firecrawl-compatible scraping API on your own server in under 60 seconds using CRW's single Docker image.
How to Convert Websites to Clean Markdown for LLMs
Turn any web page into clean, noise-free markdown ready for LLMs using CRW's scrape endpoint. No selectors, no regex.
How to Expose Web Scraping to AI Agents with MCP
Connect CRW's built-in MCP server to Claude, Cursor, or any MCP-compatible AI agent for live web scraping in agentic workflows.
How to Build a RAG Pipeline from Websites Using CRW
Step-by-step guide to scraping websites, converting to clean markdown, and feeding into a RAG pipeline using CRW's API.
Browse more
Jump back to the full archive
This category contains 54 of 156 total posts in the fastCRW blog archive.
View all blog posts