By the fastCRW team · Catalogue figures checked 2026-11-05 against GET /v1/data · Verify independently before relying on any figure.
We named our public data catalogue Pergamon after the ancient library that kept collecting. The name describes the job: gather sources from many countries, read their terms, and keep them reachable through one interface. This post covers the history behind the name, what we took from it, and where the catalogue stands today. Pergamon lives at fastcrw.com/pergamon.
The library of Pergamon
Pergamon was a Greek city in Anatolia, near the modern town of Bergama in western Turkey. It was ruled by the Attalid dynasty, and the library was built under King Eumenes II, who reigned from 197 BC. It became one of the most important libraries of the ancient world, second only to its rival library in Egypt. The two competed for books, for scholars and for parchment. The Library of Pergamum entry gives the background and the sources behind it.
The library is also tied to the material it wrote on. The word "parchment" derives from the city's name, through the Latin pergamenum. According to tradition (Pliny the Elder, citing Varro, is the usual source), Egypt held back papyrus exports to slow the library down, and Pergamon developed parchment as a substitute. Treat that as tradition, not fact. Parchment was in use in Anatolia long before Pergamon rose, and the encyclopedia entry calls the invention story a myth. What is well established is that Pergamon was a thriving centre of parchment production.
What we took from it
The hub page describes the choice in one line: "Named after the library of Pergamon, whose scribes gave us parchment and kept collecting." Three working habits follow from that line.
- Collect from the source. Each entry in the catalogue is called through the source's own published API, not through a scraped copy of its pages.
- Read the terms first. A provider whose terms nobody has read is not reachable from the catalogue, and caching follows the licence.
- Keep the licence with the text. Every response carries the source's licence and attribution, so the citation travels with the data.
An agent that cites a public statistic is only as useful as the record behind the citation. Collecting a source is the easy part. Keeping its terms attached to every copy is the work.
What the catalogue is today
The catalogue lists more than 430 sources and 4,200 capabilities, grouped into 13 categories. More than 70 of those sources run on your own machine through the local crw-mcp, and the data for those calls never passes through fastCRW. The rest run on our side. These numbers move as sources are added or switched off, so the live catalogue is the one to quote.
Open data portals is the deepest category, with more than 270 city, state and national portals, so one agent can query a city portal and a national statistics office with the same two tools. The other large groups are research, developer tools such as package registries, finance, health, reference works and public records.
Every source row in the catalogue records the fields that matter before you build on it: the licence and its URL, the attribution text, whether commercial use and redistribution are allowed, and whether the source runs on our side or on your machine. We do not summarise a licence as "free". The row says what the publisher wrote, and the response carries the same text.
A find call is the quickest way to see the breadth. This query searches the Reference category for museum collections:
curl "https://api.fastcrw.com/v1/data/find?query=museum%20collection&category=Reference&limit=3" \
-H "Authorization: Bearer $CRW_API_KEY"Our run returned three capabilities from the Met's collection API, including its collection search, its departments list and its list of object IDs. Find calls are free and never billed.
Each item in the response names the provider and capability, its creditsCost, whether a keyRequired, the options it takes under options and the licence under provenance. That is enough for an agent, or for you, to decide what to call next without reading the source's documentation first. The reference for the full call format is on the data sources docs page.
Five example providers from different fields are Hong Kong Observatory, the UK Office for National Statistics, NASA, GBIF and The Met. Each provider page shows its own licence line, so the five of them already carry five different sets of terms.
What stays open
Some parts of the stack are open by design. The fastCRW engine is open source under AGPL-3.0, so you can read how it works and run it yourself. The local crw-mcp runs the client-run sources on your own machine, under each source's own terms, at 0 credits. Catalogue browsing and find calls are free. For what the AGPL means in practice, read our AGPL explainer. For how licences apply to public API data, read the licence guide.
Open does not mean unlimited. Upstream sources keep their own rate limits and their own terms, and a call that fails upstream is not billed.
Caching follows the licence
Our hub states one rule that shapes the whole catalogue: caching follows the licence. A response served from cache says so in its provenance: on REST, is_stale is true when the body came from the cache, and cache_age_seconds says how old it is. It does not mean the upstream data is out of date.
Bring a source
If you run an API, a registry or a dataset that agents should be able to reach, the provider intake is open at become a provider. Four things are needed before we can serve a source:
- A documented endpoint we can call without scraping you.
- Terms that say, in your own words, whether a paying customer of ours may use what we return.
- The attribution line you want displayed, if any.
- A contact who can answer a licence question in writing.
In return, your licence and attribution line travel with every response from your source, and callers do not need an integration of their own. We will not list a source whose terms we cannot read. If you have a question about a source that is already listed, contact us through support.
To see how an agent uses the catalogue, start with Public data API for AI agents. For the story of the engine underneath, read the story behind the engine.
Related: Public data API for AI agents · Can you use public API data commercially? · Pergamon catalogue