One MCP Server for 430+ Public Data Sources

By the fastCRW team · Catalogue figures checked 2026-11-05 against GET /v1/data · Verify independently before relying on any figure.

An MCP server for public data gives an agent access to many public sources through one server and a small, fixed set of tools. Pergamon, the public data catalogue on fastCRW, does that with two data tools, crw_find_data and crw_get_data, across more than 430 sources. The agent searches the catalogue first, then loads the options for the one capability it needs. It never has to load a schema for every endpoint.

This post covers why one server per source breaks down, how the two tools work, how to install them in a hosted or local setup, and what an agent does when a call fails. The calls shown were run on 2026-11-05.

One server per source does not scale

A single-source MCP server is useful for its own source. It also adds an install, an auth flow and a tool list that the model reads on every turn. An agent that needs ten sources ends up with ten servers and a long list of tool descriptions competing for context.

Developers have noticed. One r/mcp thread is titled "Gave my agent 140 MCP tools at once and watched it get dumber" (reddit.com). Another post is titled "I connected 8 MCP servers to my agent. 3 are worth it. The rest are context bloat." (reddit.com). An AWS engineering write-up on MCP tool design covers the trade-offs (aws.amazon.com).

The fix is to load tool detail on demand. Pergamon's answer is a catalogue search and a generic call, not one tool per source.

Two tools: find, then get

The two data tools are read-only. Their names and inputs are:

• crw_find_data takes query, category, provider, limit (1 to 50) and offset. It returns matching capabilities with their requiredOptions, optionalOptions, creditsCost, keyRequired and licence.
• crw_get_data takes provider, capability and an options object. It returns the data with a camelCase provenance block.

The agent calls find with a plain-language query, reads the options for the best match, then calls get with exactly those options. That keeps the context small, because the agent loads the options for one capability, not the schemas for more than 4,200 capabilities.

A find call from an agent looks like this:

{"name": "crw_find_data", "arguments": {"query": "SEC full text search", "limit": 2}}

The first item returned is secfts/search, the EDGAR full-text search capability. Its entry lists one required option, q, and optional filters such as forms, startdt and enddt.

How find narrows the search

Find takes a plain-language query, and three optional inputs narrow it further. category limits the results to one category, and the match is case-insensitive. provider limits them to one source by its slug. limit sets the page size, from 1 to 50, with a default of 10. A value outside that range is refused with invalid_options instead of being clamped, so the agent should send a valid number.

Results come back in pages. When more matches exist, the response carries a nextOffset value, and the agent passes it back as offset to read the next page. The response also reports a total, so the agent knows how many matches there are before it pages through them.

What one result tells the agent

Each item names the provider and the capability, and it gives a short description, the category and the cost in creditsCost. The options are split into requiredOptions and optionalOptions, so the agent knows what it must send before it calls. keyRequired says whether the source needs a key of your own. The licence line says what the data may be used for.

Over MCP the item also carries keyConfigured and requiresLocalInstall. Those two fields tell the agent whether the call can run from this client at all. Read the item before the get call, not after a failure.

Install: hosted vs local

There are two ways to connect, and they differ in which sources they can run.

• Hosted at https://fastcrw.com/mcp. It signs in with OAuth, and an API key in the path (/mcp/<API key>) also works. The hosted endpoint runs server sources only. A client-run source is listed with requiresLocalInstall: true, and a call to it returns client_execution_required.
• Local, installed with one command. It runs client-run sources on your machine as well. Without a key, the local server starts in embedded mode and has no data tools.

# Local server, runs every source including the ones that run on your machine
npx -y crw-mcp install --api-key <your fastCRW API key>
# Hosted (web MCP clients): https://fastcrw.com/mcp  (server-run sources only)

Both setups expose the same two data tools. Setup steps for each supported client are in the MCP client documentation. There are integration pages for Claude Code and Cursor.

Pick the hosted endpoint when every source you need is server-run and your client can only reach a URL. Pick the local install when you need a client-run source, or when you want those calls to stay on your own machine. It is the only way to reach the full catalogue.

A worked session: "Which recent 10-K filings mention supply chain?"

This is the path an agent takes for a real question. It takes three steps, and the find step is free. The same steps work for any source in the catalogue. Only the find query and the options change.

Step 1, find the capability. The find call above returns secfts/search.

Step 2, get the filings. The agent calls crw_get_data with the phrase in quotes and the form filter:

{"name": "crw_get_data", "arguments": {"provider": "secfts", "capability": "search", "options": {"q": "\"supply chain\"", "forms": "10-K"}}}

Over the hosted endpoint on 2026-11-05 this call returned creditsUsed: 1 and a result of about 65 KB. The first hit, trimmed, looks like this. It is the same query run over REST on the same day:

{
  "_id": "0000931763-02-002489:dex21.txt",
  "_source": {
    "display_names": ["FEDEX CORP  (FDX)  (CIK 0001048911)"],
    "form": "10-K",
    "file_date": "2002-07-19",
    "adsh": "0000931763-02-002489",
    "file_description": "SUBSIDIARIES OF REGISTRANT"
  }
}

Read the hit as an identity. It gives you the accession number, the entity, the form and the file date. It does not give you the sentence that matched. The catalogue description says so directly: no matched-text snippet is returned, only the identity of the matching document. The agent has to open the filing to quote the text.

Step 3, go deeper on one filer. Take a CIK from the hit and call sec/submissions with it to read that company's filing history. Pad the CIK to 10 digits first.

Errors the agent can act on

Every error starts with a code in brackets, followed by a sentence the agent can pass on. The codes are the same wherever the source runs, so an agent can handle a client-run failure and a server-run failure with one branch. These are the six codes an agent is most likely to meet. The meanings come from the data sources documentation.

Code	Meaning	What the agent does
invalid_options	A missing or malformed option, or a bad provider or capability.	Use the ids and options from crw_find_data.
unknown_data_source	No such source or capability.	Call crw_find_data again with a different query.
connector_key_missing	The source needs your own key.	Run npx -y crw-mcp connectors set-key <provider>.
client_execution_required	Hosted endpoint only. The source runs on your machine.	Install the local server.
upstream_rate_limited	The source is limiting requests.	Wait, then try again.
result_too_large	The result is over 256 KiB.	Narrow the options and call again. For a server source the call is refunded.

Bring your own key for client sources

Some client-run sources need a key from the source itself. FRED is one example. It needs a free key from the FRED API key page. Store it once with the connector command:

npx -y crw-mcp connectors set-key fred

The item for a source reports keyConfigured, so the agent can see whether the key is in place before it calls. The key stays on your machine, and it is never passed as an argument to a tool call. Run npx -y crw-mcp connectors list to see which sources need a key and which already have one.

The connector list has one row for each client-run source. Three of them, fred, discogs and fec, show "required, missing" until you set a key. musicbrainz, openmeteo and openlibrary show "not needed". The list reflects the keys stored on your machine, so your output may differ.

Working rules for the agent

• Call find before get, unless the exact provider, capability and options are already known from this session.
• Read creditsCost and keyRequired before a loop, so a batch does not surprise you with a key error halfway through.
• Narrow the options before the call, so the result stays under the 256 KiB cap.
• Pass nextOffset back as offset to read the next page of find results, instead of guessing a page number.
• Keep the attribution with every figure the agent reports, so a reader can see which source the number came from.

Honest limits

• Two tools are a design choice, not a magic index. Find only returns what the catalogue describes, so a source that is not listed cannot be reached.
• Over MCP a result is capped at 256 KiB. Large bodies such as the full SEC ticker list or the KEV catalogue are refused. Use the options to narrow the call.
• The hosted endpoint cannot run client-run sources. For those you need the local install.
• Upstream rate limits still apply. A source that limits requests answers upstream_rate_limited, and the agent has to wait before it retries.
• The catalogue changes. Counts and capabilities move as sources are added or switched off, so do not hard-code a list of sources into a prompt.

Sources

• fastCRW MCP client setup
• fastCRW data sources documentation
• AWS: MCP tool design, practical approaches and tradeoffs

Related: Public data API for AI agents · MCP web scraping · MCP vs REST API for agents · Best MCP servers for web scraping · Pergamon catalogue