fastCRW Benchmark Methodology

Executive Summary

The benchmark center exists for one purpose: make fastCRW's claims auditable.

Every claim on the marketing site should fall into one of two buckets:

1. Internal benchmark result: measured by the fastCRW team and clearly labeled as such.
2. Third-party benchmark context: sourced from an external publisher and described as market context, not as first-party proof.

If a claim cannot be placed into one of those buckets, it should not appear as a benchmark claim.

This discipline matters because scraper benchmarks are easy to abuse. A useful benchmark page should tell you what was tested, how it was framed, and where the result stops being reliable.

Metric Definitions

Coverage

Coverage refers to whether the system extracts a useful primary content result from the benchmark URL set. It is not a claim about every possible target site on the internet.

Average latency

Average latency refers to the mean response time observed in the benchmark setup. It should always be paired with:

• the dataset or workload,
• the environment framing,
• and the competitor set.

Idle RAM

Idle RAM is especially important for fastCRW because the product differentiates on operational weight. In this site, the canonical fastCRW framing is:

• 6.6MB idle RAM for the server plus the optional LightPanda sidecar

That is a product metric for this deployment framing, not a universal guarantee across every deployment shape.

Why This Methodology Is Narrow On Purpose

The goal is not to simulate every possible internet page or every deployment configuration. The goal is to produce claims that are:

• specific enough to verify,
• useful enough to guide evaluation,
• and narrow enough to stay honest.

A smaller honest benchmark is more useful than a bigger benchmark that quietly mixes workloads, environments, and unsupported inferences.

Dataset and Source Provenance

The main internal dataset used in this release is the Firecrawl scrape-content dataset. That matters because it grounds the comparison in a public source rather than a hand-picked marketing demo.

The market-context section also references third-party reporting, especially where it supports:

• public throughput comparisons,
• success-rate context,
• or high-level operational differences.

Interpretation Rules

To keep the site honest, these rules apply across all benchmark and comparison pages:

• Never mix internal and external benchmark numbers in the same unlabeled table.
• Phrase internal claims as "In our 1,000-URL benchmark..."
• Use third-party sources to frame the market, not to fabricate a synthetic fastCRW win where none was measured directly.
• Include a section on where fastCRW does not obviously win.

Reading Third-Party Sources Correctly

Third-party benchmark material is useful, but it should be handled carefully:

• use it to understand how the market is described externally,
• use it to validate or challenge broad claims,
• but do not merge it into internal tables as if it came from the same test run.

Where Methodology Limits the Claims

This benchmark center is designed to support decision-making, not absolutist claims.

That means:

• fastCRW can publish a strong Firecrawl alternative case,
• fastCRW can publish a credible operational-efficiency case,
• but fastCRW should not claim a universal win over every crawler, every workload, and every runtime.

That discipline is what makes the benchmark center useful instead of noisy.