The web data API
for AI agents.
Search, scrape, crawl, extract, and monitor the web through one managed cloud API. No infrastructure to run.
500 one-time credits · No credit card required · also open source
Integrations
Works with your stack
Search, scrape, crawl, and extract
1# pip install crw2from crw import CrwClient34crw = CrwClient(api_key="crw_live_...")56# Search the web, get results with content7results = crw.search("latest LLM coding benchmarks")8print(results[0]["title"], results[0]["url"])1{2 "success": true,3 "data": [4 { "title": "SWE-bench Verified leaderboard", "url": "https://swebench.com" },5 { "title": "Claude 3.5 Sonnet — coding evals", "url": "https://anthropic.com/news" },6 { "title": "GPT-4o system card", "url": "https://openai.com/research" },7 { "title": "LiveCodeBench", "url": "https://livecodebench.github.io" },8 { "title": "DeepSeek-Coder V2", "url": "https://arxiv.org/abs/2406.11931" },9 ...10 ]11}Benchmarked against Firecrawl, Tavily and Exa
Benchmarks
Where we win,
measured.
Public datasets, public harness. Reproduce every number with crw bench.
Recall · 191 paper-retrieval queries
higher is better ↑
fastCRW
ExaHighest recall on the board · +7.7 pts over Firecrawl
Works where your agents already live
One command · installs everything
Auto-detects your agent and wires the skill + MCP server. No config.
Prefer to wire it yourself?
Per-agent installers do the same, one at a time.
Claude Code, Cursor, Codex & moreMCP: drop-in config
Python
crw on PyPI: scrape, crawl, and map from one client.
Node.js
crw-sdk on npm: the same API, fully typed.
Prefer to run it yourself? Self-host docs
We handle the hard stuff. You ship.
Media parsing. fastCRW turns web-hosted PDFs into clean markdown: CPU-only, no LLM, billed per page.
Smart wait. fastCRW intelligently waits for content to load, making scraping more reliable.
Cached, when you need it. Selective caching: you choose your caching patterns, growing web index.
Actions. Click, scroll, write, wait, press and more before extracting content.
Anti-detection. Rotates headers, handles CAPTCHAs, and mimics real browsers. No proxy setup needed.
Source attribution. Every response includes source URLs and metadata: your users can verify, your team can debug.
Free tier, then usage-based from $13/mo. Pay only as you scale. Start with 500 one-time credits.
See pricingBuilt for teams shipping
production AI
AI agents
autonomous web access → MCP
AI Chat & RAG
clean markdown → search + scrape
Structured extraction
prompt or schema → JSON → extract
Change monitoring
track page changes → monitor
Market research
competitors & trends → crawl
Deep research
full-page extraction → crawl + extract
Content aggregation
feeds, blogs & forums → crawl
Lead enrichment
company & profile data → scrape
Audit the code.
Then let us run it.
Auditable, not a black box
Every request path, parser, and ranking rule is public on GitHub under AGPL-3.0. Read exactly what happens to your data before you trust it.
Proven, and reproducible
Our benchmark harness is public, and CRW leads Firecrawl's research-recall index on ArXivQA. We ship the test, not just the claim.
Zero ops
We run the proxies, the browsers, the anti-bot, and the scaling. You call one endpoint and get clean data back. No infrastructure to babysit.
Frequently
asked questions
Ready to give your agents the web?
Start free with 500 one-time credits, no credit card. Usage-based plans from $13/mo when you're ready.
Open source, AGPL-3.0, audit every line on GitHub.
Are you an AI agent? Get an API key here