By the fastCRW team · Engineering post · Catalogue figures checked 2026-11-05 against GET /v1/data · Verify independently before relying on any figure.
An LLM that writes "US GDP was 27.8 trillion dollars in 2023" has given you a number with no way to check it. The fix is not a better prompt alone. It is a pipeline where every figure leaves the API with its source attached, and the model is only allowed to repeat what arrived with that source. This post shows how to keep the provenance from an official public API all the way to the final answer, in Python.
Why numbers need a source
A figure with no source cannot be checked, and a reader cannot tell a fresh value from a stale one. Official data is not always clean either: one Hacker News thread is titled "Twice this week, I have come across embarassingly bad data" (the misspelling is theirs). When a number looks wrong, you want to see exactly where it came from. The Pergamon catalogue returns a licence, an attribution line and the upstream URL with every response. If your pipeline drops those fields, the agent is left guessing. If it keeps them, the answer can point to the exact record.
The provenance block, field by field
Every successful call returns a provenance block next to data. A REST response uses snake_case. This is a real response from the SEC EDGAR submissions capability for Apple, taken on 2026-11-05. It is the provenance block only:
"provenance": {
"source": "SEC EDGAR",
"upstream_url": "https://data.sec.gov/submissions/CIK0000320193.json",
"licence": "A work of the United States Government, not subject to domestic copyright under 17 U.S.C. 105, published by the agency as open data.",
"licence_url": "https://www.sec.gov/about/privacy-information#security",
"attribution_text": "Source: U.S. Securities and Exchange Commission, EDGAR",
"retrieved_at": "2026-11-05T10:19:52.312Z",
"adapter": "rest",
"cache_age_seconds": 435,
"is_stale": true
}
sourcenames the publisher. Show it next to the figure.upstream_urlis the page or endpoint the value came from. Link it.licenceandlicence_urlare the source's own terms. Quote them; do not summarise them as "free".attribution_textis the line the source asks you to show. Copy it as given.retrieved_atis when fastCRW fetched the body.cache_age_secondsandis_staletell you whether the body came from our response cache. More on this below.
The Python SDK returns the same information in camelCase, with fewer fields. It has source, upstreamUrl, licence, licenceUrl, attributionText and retrievedAt. It has no cache fields, so the helper below uses retrievedAt for freshness.
Keep the four fields together
The pattern is simple. Whenever you turn an API result into a number the model can see, carry four things together: the value, the upstreamUrl, the attributionText and the retrievedAt. Store them as one record, and never let the value travel alone. The helper below does that for three sources. It wraps get_data and returns citation-ready records for World Bank GDP, BLS inflation and SEC revenue.
# pip install crw
import json
from crw import CrwClient
client = CrwClient() # reads CRW_API_KEY
def cite(result: dict, value: object, period: str) -> dict:
prov = result["provenance"]
return {
"value": value,
"period": period,
"upstreamUrl": prov["upstreamUrl"],
"attributionText": prov["attributionText"],
"licence": prov["licence"],
"retrievedAt": prov["retrievedAt"],
}
def gdp_usd(country: str, year: str) -> dict | None:
result = client.get_data(
"worldbank", "indicator",
{"country": country, "indicator": "NY.GDP.MKTP.CD", "date": year},
)
rows = result["data"][1] or []
if not rows or rows[0]["value"] is None:
return None
return cite(result, rows[0]["value"], rows[0]["date"])
def cpi_all_items() -> dict:
result = client.get_data("bls", "series", {"seriesId": "CUUR0000SA0"})
latest = result["data"]["Results"]["series"][0]["data"][0]
return cite(result, latest["value"], f"{latest['year']}-{latest['periodName']}")
def apple_revenue() -> dict:
result = client.get_data("sec", "companyconcept", {"cik": "0000320193", "concept": "RevenueFromContractWithCustomerExcludingAssessedTax"})
facts = sorted(result["data"]["units"]["USD"], key=lambda fact: fact["end"])
latest = facts[-1]
return cite(result, latest["val"], f"{latest['start']} to {latest['end']}")
if __name__ == "__main__":
citations = [gdp_usd("US", "2023"), cpi_all_items(), apple_revenue()]
print(json.dumps([c for c in citations if c], indent=2))
Run it with uv run --with crw python cite_helper.py. If the World Bank value is null, for example for a year with no observation, the helper drops that record instead of passing an empty figure on. Real output from 2026-11-05, first record only, trimmed. The BLS and SEC records have the same fields:
[
{
"value": 27811517000000,
"period": "2023",
"upstreamUrl": "https://api.worldbank.org/v2/country/US/indicator/NY.GDP.MKTP.CD?format=json&date=2023",
"attributionText": "Source: World Bank Open Data (CC BY 4.0)",
"licence": "The World Bank Group licenses its datasets under Creative Commons Attribution 4.0 (CC BY 4.0), which allows copying, modification and distribution for any purpose including commercial use, with attribution.",
"retrievedAt": "2026-11-05T10:22:10.641Z"
},
...
]
The BLS record's upstreamUrl shows registrationkey=REDACTED, so that link will not run as written. It shows where the figure came from, not a URL to open.
The prompt rule
Keep the citation rule in the system prompt of the agent, so the model knows what to do with a record it receives. A rule that works in practice reads like this:
Cite the attributionText and link the upstreamUrl for every figure you state.
Use only figures that arrive as citation records. If a figure has no record,
say that the figure is missing. Do not estimate it or fill it from memory.
The rule does two jobs. It stops the model from adding a figure of its own, and it turns a gap in the data into a visible sentence instead of a made-up number. Check the answers with a second pass that matches every figure to a record.
Freshness
The REST fields tell you when a body was retrieved and whether it came from cache. retrieved_at is the fetch time. cache_age_seconds says how old a cached body is. is_stale is true whenever the body was served from fastCRW's response cache. It does not mean the upstream data is out of date.
The block above shows the difference. is_stale is true and cache_age_seconds is 435, so that body had been in our cache for 435 seconds when it was served. SEC may or may not have changed the record in that time. The flag tells you the body was not a fresh fetch, and the age tells you by how much. If the figure must be current, call again and check the flag on the new response.
The SDK does not expose these cache fields. If your pipeline depends on them, call the REST endpoint, or show the retrievedAt time to the reader and let them judge.
Showing attribution in a UI
When a person reads the answer, put the attribution where they can see it without digging. Put the value, then the source name and the attribution line directly under it, with the upstream URL as the link. Keep the licence one click away, in a tooltip or a footer. Do not hide it behind an icon that needs a hover to reveal the source.
The pipeline in five steps
- Call every API through a helper such as
cite(), never straight from the prompt builder, so no bare value leaves the fetch step. - Store each citation record next to the value it describes, in the same object.
- Pass the model the records. The figures it can see then all carry a source.
- Ask the model to tag each figure with the index of its record, for example
[1]. The tag makes the check in the next step simple. - Before the answer reaches a reader, check that every tagged figure maps to a record and that every record has a usable URL, attribution text and fetch time.
The last step can be a short function. It runs against the same records the helper returns, and it fails loudly on the first record that is missing a field. This is the check that ran against the three live records above, and it passed:
def check_citations(citations: list[dict]) -> None:
for c in citations:
assert c["upstreamUrl"].startswith("https://"), c
assert c["attributionText"], c
assert c["retrievedAt"], c
Output on 2026-11-05: checked 3 citations.
Honest limits
- Provenance records where a value came from. It does not check that the value is right. An upstream error or a bad record passes through with its citation intact.
- The helper has no retry and no batching. It is a pattern to copy, not a production client.
- Licence text changes. Read the
licence_urlbefore you ship a product that shows the figure to customers.
Sources
Related: Public data API for AI agents · RAG pipeline with crw · Web context layer for AI agents · Public API data and commercial use · Due diligence agent with public data · Pergamon catalogue
