Base URL and Authentication

The managed cloud uses a single base URL:

https://fastcrw.com/api/v1

Every request needs an API key in the Authorization header:

Authorization: Bearer YOUR_API_KEY

Create the key in the dashboard, keep the raw value somewhere safe, and use a secrets manager or environment variable in production instead of pasting it directly into scripts.

1. Run a Single-Page Scrape

Start with a page you can inspect manually in the browser. That makes it easier to compare the output against the source site.

curl -X POST https://fastcrw.com/api/v1/scrape \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url":"https://example.com",
    "formats":["markdown"],
    "onlyMainContent": true
  }'

For a first test, look at three fields:

success tells you whether the request produced a usable result.
data.markdown contains the extracted content.
metadata.statusCode tells you what the target site actually returned.

If the site is JavaScript-heavy, repeat the same request with renderJs: true and a small waitFor value such as 2000.

2. Discover Before You Crawl

map is the lightweight way to answer "what is on this site?" before paying for a larger recursive job.

curl -X POST https://fastcrw.com/api/v1/map \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url":"https://example.com/docs"
  }'

Use map when you want to:

inspect the reachable URL set,
choose which section is worth crawling,
and avoid launching a broad crawl from a noisy home page.

3. Launch a Crawl

Use crawl when you want multiple pages instead of one response payload.

curl -X POST https://fastcrw.com/api/v1/crawl \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url":"https://example.com/docs",
    "limit": 10
  }'

The initial response returns a crawl identifier. Keep that id and poll it until the job finishes.

curl https://fastcrw.com/api/v1/crawl/CRAWL_ID \
  -H "Authorization: Bearer YOUR_API_KEY"

A practical habit is to start with a low limit, validate output quality, and only then scale the job up.

4. Check Remaining Credits

Use the balance endpoint to see what you have left before and after test runs.

curl https://fastcrw.com/api/v1/account/balance \
  -H "Authorization: Bearer YOUR_API_KEY"

That response is the fastest way to confirm whether a request consumed credits as expected.

5. Read Warnings Correctly

Do not treat success alone as the entire story.

success: true means fastCRW returned a payload.
warning means the payload may be degraded, incomplete, or sourced from a problematic target response.
metadata.statusCode is the target site's real HTTP status code, not the status of the fastCRW API itself.

For example, a site can return a block page or anti-bot interstitial and still produce content. In that case the response may be technically successful but operationally poor.

6. Move from Testing to Real Integration

Once the first requests look good, the usual next step is:

lock down the exact formats you need,
decide when to enable JS rendering,
add retries and backoff for 429 or transient target issues,
and monitor credits and warnings separately.

Use these follow-up pages before production rollout:

Getting Started