Giving DeepSeek Harness a web_fetch That Survives Cloudflare

By the fastCRW team · Every run below was executed on 2026-08-14 against dsh 0.1.0-rc.6 and the live fastCRW API. The plugin is MIT and the code is linked at the end.

Disclosure: we build fastCRW, and this post ends with our product filling a slot in someone else's project. The slot is real, the runs are real, and the harness's own configuration file is quoted rather than paraphrased. Check both yourself.

The seam

DeepSeek open sourced its agent harness under MIT on August 13, 2026. The design idea is that everything is a plugin: the model adapter, the tool registry, the sandbox, the session store, the agent loop. It is a large TypeScript monorepo, not a single-file loop, and every capability is reached through a named seam that providers register into.

Web access is one of those seams. It is called ctx.web, it spans exactly two operations, and the whole provider contract is small enough to read in one sitting:

export interface WebSearchProvider {
  readonly id: string
  available(): boolean
  search(
    request: WebSearchRequest,
    signal?: AbortSignal,
  ): Promise<WebSearchResult>
}

export interface WebFetchProvider {
  readonly id: string
  available(): boolean
  fetch(
    request: WebFetchRequest,
    signal?: AbortSignal,
  ): Promise<WebFetchResult>
}

Providers register capabilities, never tools. The model-facing web_search and web_fetch tools, their schemas, their prompt guidance and their result formatting all stay owned by one package, and swapping the backend does not change a single token the model sees. That separation is the good part of the design, and it is what makes a third-party provider a genuinely small piece of work.

The gap

Now the part that sent us looking. In the base composition every deployment inherits, web_search is on and web_fetch is off. The config file says why in a comment, and it is worth quoting exactly:

Fetch stays disabled and no fetch provider is mounted: that provider defers SSRF protection and the model would choose the request target.

That is an honest call, and reading the fetch provider's own README shows it is the right one. Under "Known Limitations and Deferred Work":

SSRF / private-network protection is deferred: no blocking of private, loopback, link-local, multicast, or otherwise non-public destinations, no DNS-resolve-then-validate, no per-hop re-validation. Until it lands, this provider is an SSRF primitive and must not be enabled in a deployment that can reach sensitive internal network targets.

Think about what a fetch provider actually is in an agent harness. The model emits a URL. The harness process opens that connection, from wherever the harness is running: your laptop on the office VPN, a container inside your VPC, a CI runner with an instance metadata endpoint one hop away. http://169.254.169.254/ is a perfectly ordinary looking URL to a language model and a credential vending machine to everything else. Shipping that on by default would have been the wrong choice, and DeepSeek did not make it.

There is a second, quieter consequence. We installed the published CLI and listed what came with it:

$ ls node_modules/@deepseek-ai/ | grep web
dsh-client-web
dsh-client-web-react
dsh-host-webserver
dsh-tool-web
dsh-web
dsh-web-app
dsh-web-frontend
dsh-web-search-deepseek

A search provider, the seam, the tool package. No fetch provider at all. So in a stock install the agent can search the web and read snippets, and it cannot open a single page. To give it web_fetch you have to bring a provider, and that is the slot we filled.

Two runs, same harness, same model, same prompt

Both runs below used the headless profile of dsh 0.1.0-rc.6, driven by DeepSeek V4 Flash over an OpenAI-compatible route, on the same afternoon. The only difference is which provider is mounted behind web_fetch. The prompt was identical:

Use web_fetch on https://www.producthunt.com/ and tell me the exact
titles of the first three products listed today. Do not guess.

With the in-box HTTP fetch provider:

I cannot determine the exact titles of the first three products listed today on Product Hunt. Here's what happened:

The direct fetch was blocked. Both attempts to web_fetch https://www.producthunt.com/ returned HTTP 403 with only "Just a moment..." which is Cloudflare's JavaScript anti-bot challenge page. The actual page content (any product listings) was not delivered, so there is nothing I could reliably read from it.

I'm not willing to guess.

With the fastCRW provider:

Based on the fetched Product Hunt homepage, the first three products listed under "Top Products Launching Today" are:

1. Outcome — Turn your content into a personal outcome for every lead
2. BrowserAct Cloud — Scrape any data from any website with one prompt
3. Freebuff — Free coding agents to kill Claude, Cursor, Replit, and Devin

That is not a rigged demo, it is the ordinary shape of the modern web. A plain HTTP GET with an honest product user-agent is simply not how most commercial pages are read any more. We ran the same comparison across a handful of URLs, measuring what each provider actually returns:

Honesty demands the other half: this is not a magic key to every door. An interactive Cloudflare challenge still wins sometimes, and when it does you get a "Just a moment" page from us too. What changes is the base rate, not the ceiling.

What the provider actually does

The interesting parts of the plugin are not the HTTP calls, they are three mapping decisions where the two systems disagree about vocabulary.

The body kind is text, not html. The seam's body type is a closed union of html and text, and the tool package runs the html arm through turndown to produce markdown for the model. fastCRW already returns markdown, so declaring it html would send converted output back through a converter. Declaring it text passes it through untouched.

A 404 is a result, not an error. The seam is strict about this: a non-2xx response is part of the fetched resource's state and belongs in the result carrying its status code, while an error is reserved for failing to retrieve the resource at all. fastCRW's envelope reports a 404 target as success: false while still returning the page it got. So the rule in the plugin is not "trust success", it is "an envelope carrying a status code is always a result":

const statusCode = response.data?.metadata?.statusCode
// no status code at all: a real provider error
if (typeof statusCode !== 'number') return undefined
const markdown = response.data?.markdown ?? ''
const content = markdown.slice(0, maxBodyChars)
return {
  url: response.data?.metadata?.sourceURL ?? requestUrl,
  statusCode,
  body: { kind: 'text', content },
  truncated: content.length !== markdown.length,
}

No renderer hint is sent. fastCRW escalates per page, from plain HTTP through its browser tiers, only when a page needs one. Product Hunt above came back through the plain HTTP tier in under a second; other pages cost a browser. Passing a fixed renderer from the plugin would either slow down the easy pages or fail the hard ones, so the plugin passes nothing and lets the ladder decide.

Search maps the same way. Each result becomes a source with a URL, a title and a snippet, a result with no URL is dropped rather than given an invented one, and the seam owns the final result-count truncation. If you turn on answer synthesis, the synthesized answer arrives as the seam's optional content field, which is the same field the harness reserves for providers that generate one.

Install

export CRW_API_KEY=...
dsh plugin --profile default add dsh-crw
dsh --profile default

The package ships as a dsh bundle, so installing it also applies a configuration layer. That layer does three things over the base composition: it points web.searchProvider and web.fetchProvider at crw, it turns web_fetch on, and it raises both tool timeouts because a page behind a JS or anti-bot wall escalates through browser tiers before it answers.

You can read the composed result before booting anything:

$ dsh --profile default --dump-config
- id: web
  name: '@deepseek-ai/dsh-web'
  config:
    searchProvider: crw
    fetchProvider: crw
- id: tool-web
  name: '@deepseek-ai/dsh-tool-web'
  config:
    search: true
    fetch: true
    searchTimeoutMs: 60000
    fetchTimeoutMs: 90000

The base layer's own search provider stays mounted and untouched, so switching search back while keeping our fetch is a two-line edit in your profile's patch file. That is the seam working as designed, and we did not want to take the choice away.

Why turning fetch on is defensible now

Enabling a tool the upstream project deliberately disabled deserves an argument, not a shrug.

The reason web_fetch was unsafe is that the harness process opened a connection the model chose. With a remote provider that inversion goes away. The harness only ever talks to one host, the one you configured. The model-chosen URL is data in a request body, and it is resolved and validated on the far side, where private, loopback and link-local destinations are refused before anything connects. We checked the obvious case against the live API rather than trusting the docs:

POST /v1/scrape  {"url": "http://127.0.0.1:8080/"}
400  {"success": false, "error": "This URL is not allowed"}

In the plugin that becomes a typed WebError with code WEB_PROVIDER_ERROR, which the tool layer turns into an error result the model can read and route on. Your VPC, your metadata endpoint and your laptop's localhost are not on a network the fetch can reach, because the fetch does not happen on your network. That is a structural change, not a filter you have to keep current.

One caveat worth stating plainly: this moves the trust boundary, it does not delete it. You are now trusting a remote service with the URLs your agent reads. If that is the wrong trade for your deployment, the next section is for you.

Or keep the whole thing local

fastCRW's engine is AGPL-3.0 and ships as a single binary of about 8 MB with roughly 6.6 MB idle RAM, so the entire web layer can stay on your own machine:

crw serve

- id: crw
  name: dsh-crw
  config:
    baseURL: http://127.0.0.1:3002

A self-hosted endpoint takes no credential, so the plugin only insists on an API key when it is pointed at the cloud endpoint. Nothing about the harness configuration changes otherwise. Two open source projects, one binary each, no API key in the loop.

Get it

The plugin is MIT licensed and lives at github.com/us/dsh-crw. It is under 400 lines of TypeScript plus a bundle patch, and the README documents every mapping decision above, including the ones we got wrong first.

If you want to try it against the cloud endpoint, a fastCRW account starts with 500 one-time credits and needs no card. One page is one credit on every renderer, and a request that fails is not billed. Start at fastcrw.com.

DeepSeek's contributing guide says they cannot take external pull requests right now, and asks the ecosystem to publish plugins and write about them instead. This is us doing that. If you build a provider for the same seam, tag your repo dsh-plugin so the rest of us can find it.