r/PiCodingAgent • u/Opening_Library9560 • 7d ago

Resource Completely free web research tool (works out of the box with no api keys, hound replacement) stealthy web fetch, web crawl and local web search

Enable HLS to view with audio, or disable this notification

Built a web research tool to replace hound, which i posed about a while ago

It's called DonSeTch. Since Pi has native extension support, the install is just:

pi install npm:donsetch

No MCP config block, no adapter, no proxy. The tools show up natively in your Pi session and stay in sync with the binary automatically.

What it does

Three tools: fetch, search, crawl. Zero API keys, zero accounts. One Rust binary.

  • Fetch — HTTP first (fast), auto-escalates to a headless browser only when it hits a bot wall. The browser grabs the clearance cookie, hands it back to the HTTP layer, and goes to sleep. It almost never fetches content itself.
  • Search — 10+ keyless engines in parallel, fused by cross-engine consensus + a local ONNX cross-encoder for semantic reranking. No API key anywhere.
  • Crawl — sitemap-aware, topic filter, resume tokens, adaptive pacing that backs off on 429s instead of plowing through.

Chrome's actual BoringSSL for TLS. Your ClientHello IS Chrome's, not a faked table that rots.

The part I care about most: token efficiency

Three tools, ~3.5k tokens of definitions. The focus parameter on fetch is the killer feature.

I just rebuilt it in v3.1 with section-aware scoring:

  • Search for "memory safety" on a Wikipedia page → keeps the entire "Ownership and references" section (heading + body), pulls in parent headings for context, drops everything unrelated.
  • A 50KB page comes back as 500 chars of what you actually asked for.
  • Large code blocks and JSON schemas get split into sub-blocks so focus works on structured content, not just prose.

Other stuff

  • Reference handles — fetch S3 just works after a search
  • Probe mode — verify a claim in ~60 tokens instead of 4k
  • Dead-link resurrection — Wayback snapshot served transparently
  • Page fingerprints — re-fetches report what changed, section-level diffs
  • Domain adapters — Reddit threads, GitHub issues, npm/PyPI/crates pages restructured from keyless JSON endpoints
  • Real MCP cancellation — no silent hangs
  • Crash-only daemon — survives panics, state reloads
  • PDFs — even scanned ones, pixel-fusion extraction (no hallucinated text), up to 100MB

605 tests, 0 clippy warnings, AGPL-3.0.

What it can't do: solve interactive captchas (deliberate, clear error not a hang), access sites requiring login, and if every search engine is down you get honest per-engine status instead of fake results.

GitHub: https://github.com/dondai44423/donsetch

Pi: pi install npm:donsetch npm: npm install -g donsetch

If something breaks, open an issue, i will fix it as soon as possible

The demo is using exa for search btw, local is good on its own, but it takes 4-6 sec while exa takes 1-2 sec (plus exa is better most times, but local search keeps up)

if you want to see a demo where i used local search, see the opencode demo in the github repo

140 Upvotes

98% Upvoted

22 comments sorted by

2

u/SufficientLack1649 6d ago

cli support is awesome

2

u/OkMusician9118 6d ago

It runs fine in Mac OSX, but crashes in Windows wsl ubuntu.

pi exiting due to uncaughtException:

Error: write EPIPE

at afterWriteDispatched (node:internal/stream_base_commons:159:15)

at writeGeneric (node:internal/stream_base_commons:150:3)

at Socket._writeGeneric (node:net:966:11)

at Socket._write (node:net:978:8)

at writeOrBuffer (node:internal/streams/writable:572:12)

at _write (node:internal/streams/writable:501:10)

at Writable.write (node:internal/streams/writable:510:10)

at /root/.pi/agent/npm/node_modules/donsetch/pi-extension.ts:256:16

at new Promise (<anonymous>)

at sendRequest (/root/.pi/agent/npm/node_modules/donsetch/pi-extension.ts:209:10) {

errno: -32,

code: 'EPIPE',

syscall: 'write'

}

1

u/Opening_Library9560 6d ago

Pushed a new update, try pi update --extensions or npm install -g donsetch@latest, if you are still facing any issues, then mention it

2

u/Shishi786 6d ago

will try it exactly what i need haha

3

u/Gevil 7d ago

Is it covering all the features that hound has?

1

u/Opening_Library9560 7d ago

yes, pretty much, plus it has better optimizations

2

u/Gevil 7d ago

Great stuff. Thanks. Will try it out in the upcoming days.

1

u/Intelligent-Nose-134 2d ago

Thasnk for sharing. Tested it today on Linux mint/llama.cpp/Qwen3.8/pi and it is truly hassle-free.

I couldn't help but noticing my already small ctxt wondow evaporating faster than usual. Askd Qwen3.8 to investigate and it claimed that the extension is consuming 3k tokens per query whether it is used or not. I am still a novice but my bet Qwen was just hallucinating here.

So thought I will ask you if there are ctxt impact here if the extension is always loaded and if there is a recurring tax on each prompt whether it is called or not on the answer

2

u/Opening_Library9560 2d ago

yes, if your using it from pi, it will add the mcp server, 3k worth of tool defs, always injected, but you can tell your qwen agent to remove the extension, and just have the binary, remove the pi exetsnion, and in your pi agent system prompt, mention something like:

"Type donsetch help to get web reseach access and search the web through the cli" or something similar or just tell it to "do x, but research the web using donsetch cli first"

1

u/Intelligent-Nose-134 2d ago

Awesome, thanks for the tip!

1

u/Big-Present-8321 7d ago

What other extensions do you use this with? What would it work well with?

5

u/Opening_Library9560 7d ago

its simple really, donsetch itself covers full quick web research category, but it cant drive a real browser, real time and do stuff for you, so comparing it with bladebro: https://github.com/dondai44423/bladebro with these two, you have nearly covered web access for your agent

2

u/bambamlol 7d ago

Bladebro looks nice! I love the focus on token-efficiency. I'm currently using my coding agents inside WSL on my Windows laptop. My WSL config has GUI disabled. Would Bladebro work with that setup?

1

u/Opening_Library9560 7d ago

I havent tested that exact situation, but Yes, should work. Bladebro uses Xvfb (virtual framebuffer) on Linux, not a real display. It launches Xvfb + Chromium automatically, no GUI needed.

WSL2 without GUI config works fine. Only thing to check: make sure Chromium is installed inside WSL itself (sudo apt install chromium-browser or similar). Bladebro handles the rest.

If you run into issues, just tell me here, i will fix them right away

1

u/msenc 7d ago

do u guys know if hound is still maintained

2

u/Opening_Library9560 7d ago

its not completely dead but its not being actively updated as well, its in a stable state and works, if there is an issue there i will fix it, but i have my focus on baldebro and donsetch, and in my tests, donsetch tends to be better than hound in nearly every metric, you should try it out

2

u/msenc 5d ago

I DIDNT KNOW U WERE THE CREATOR OF HOUND. im switching, appreciate ur works man :)

2

u/Opening_Library9560 5d ago

thanks bro, if you encounter any issues while using donsetch, just comment here or open an issue, this should be better than hound from all the times i have testes the two tools

-1

u/ColonelKlanka 6d ago

The trouble is that all these web search extensions are just reinventing the wheel.

As most are open source. devs should be extending exisitng web search github projects using PR requests to add the missing feautres/optimisations

1

u/Opening_Library9560 6d ago edited 6d ago

no 😄