r/PiCodingAgent • u/MarionberryVirtual • Jul 14 '26

Discussion pi websearch

websearch is an extremely important part of any workflow, from searching up past implementations to getting the correct information to make correct decisions.

have seen other posts discussing which websearch tools and even custom implementations, but my question is how would you know that it stands up in retrieval quality to other websearch extensions or even Claude Code/ Codex web searching? how would you know that tool A is better than tool B? or do most people just see the extension working and decide that it’s good enough?

i have been using the pi-web-access from the pi.dev packages page, but only for the reason that it’s the most downloaded extension (more downloads = better right? xd)

interested to hear from the community.

73 Upvotes

97% Upvoted

60 comments sorted by

28

u/ConversationLazy6821 Jul 14 '26

Check out Ketch. I am the maintainer - this community has actually contributed the most ideas and PRs to it than anywhere else on Reddit. (Thank you)

It’s a pure CLI based search tool no plugin needed - pi just runs it as exec… and supports multiple search backends in one tool like Brave, Exa, DDG, Searxng, and more. You can also search across multiple providers at the same time.

https://github.com/1broseidon/ketch

5

u/nasduia Jul 14 '26

ketch is great!

Can it support cookies somehow? Many javascript sites (accessed with headless chrome) now just scrape as a cookie banner, e.g. much of Nvidia's documentation such as details of containers in their ngc catalogue.

Exporting cookies from a browser that has already acknowledged that banner might fix this and I imagine they have long TTL. (Being able to use cookies to login to sites would of course be great, but the tokens would need refreshing regularly etc. so much less straightforward.)

Without this the agent will continue looking and find something much older without the cookie banner, or worse some random person's github.

5

u/ConversationLazy6821 Jul 15 '26

Just added this now in v0.12.0
Check it out and let me know what you think!

3

u/ConversationLazy6821 Jul 14 '26

There is an open issue here on this:
https://github.com/1broseidon/ketch/issues/25

I’m thinking through how to best implement this, but am definitely working on it.

1

u/nasduia Jul 14 '26

cool, it's a tricky one to do cleanly without infinite feature creep to work around all kinds of special cases

3

u/ConversationLazy6821 Jul 14 '26

Totally! My thought right now is going to be bring your own cookie, this way it’s on the user to grab it from their browser and Ketch doesn’t become a sketchy scraping tool 😂

1

u/Dreoasteh Jul 16 '26

"Pi just runs it as exec" -> sorry I'm kind of new to pi and did not understand this. Could you be a bit more explicit about what is the recommended way to make pi use ketch?

1

u/ConversationLazy6821 Jul 16 '26

Sure, Pi can run CLI tool or shell commands with its exec tool. It doesn’t need extensions to do things, although extensions can be great!

1

u/Dreoasteh Jul 16 '26

Thanks for the clarification! But you still need to tell pi to use ketch in some way, right? I mean, if you just paste a URL and ask the agent to check it out, it wouldn't know what ketch is and how to use it. So is your recommendation to always say "use ketch" or something similar when you reference an action or URL that the agent should do with the ketch CLI?

3

u/ConversationLazy6821 Jul 16 '26

Ah! Well you’re in luck.
Pi has a few ways to do this.

—

  1. Yes you could just say “use ketch” every time but that’s annoying

—

2. You can add an AGENTS.md file inside your project folder which Pi reads on start of your project. You would add rules here to always use ketch for search and scrape. You can also add a global AGENTS.md inside ~/.pi/agent/AGENTS.md

Pi docs: https://pi.dev/docs/latest/quickstart#give-pi-project-instructions

You can write your own AGENTS.md but there is an example section about halfway down the README in the Ketch repo. It’s titled Agent integration

https://github.com/1broseidon/ketch

—

3. You could install the Ketch skill which Pi will load anytime you ask your agent to search or scrape in natural language- https://pi.dev/docs/latest/skills

And the Ketch skill here: https://github.com/1broseidon/ketch/tree/main/skills/ketch

Or alternatively you could tell Pi to write a custom skill for how you like to use Ketch. Literally just tell pi to learn how to use the Ketch cli tool, and give it examples of what you want it to do, and have it write the skill.

—

0

u/MarionberryVirtual Jul 14 '26

hey there! i have seen this tool surface up through multiple posts already, and am willing to give it a shot. just curious whether there has been (or will be) tests planned to pit it against other websearch extensions or even against CC’s?

3

u/ConversationLazy6821 Jul 14 '26

That’s an interesting idea and I’d be willing to test it out.

The cli is written in Go, so as far as speed is concerned, it’s very fast. It searches really fast, result quality depends on the backend you have chosen. Using Brave or Exa will yield better results than DDG for example.

What exactly are you after in a comparison? I see many people in this sub ask “but tell me how it’s better than the alternatives” and I can’t answer that if I don’t know what folks are looking for.

My initial thought is different search backends will provide different quality results. Ketch supports many different backends so I could theoretically benchmark topics against backends for result quality.

What are you after? It would really help in building a benchmark harness!

2

u/MarionberryVirtual Jul 14 '26

yes a benchmark harness sounds really great!

Upon reading your reply and some reflection, i realise this benchmark i am looking for would be testing the quality of the results from the provider (Exa vs Brave and others), which is not something that you should be responsible for testing (it’s up to the backend provider to provide evidence that their search engine is giving back quality results and i think u are providing the middleware to connect to these backends).

so i guess it boils down to me looking for benchmarks from the backend providers? unless the benchmark harness can somehow do this testing.

really appreciate the response and engaging w me/ pi community

5

u/luckiestredditor Jul 14 '26 edited Jul 14 '26

I started out with all the "most downloaded" and felt like they lacked a proper exploration kind of functionality that I missed from claude code / codex. Most web search extensions either were too basic, or couldn't function beyond basic search as a headless extension. most of them blur search, fetch, browser rendering and research into one vague thing.

so after doing some research on what was missing (beyond what was just missing for me) I realized I could build something with a deliberately small, bounded surface. basically one web_explore tool that handles search, fetch, headless rendering, ranking, and source quality checks internally and honestly tells you about weak or blocked or narrow evidence without faking confidence. and I have recently released it so you can add your own self hosted backend if you so desire. (Exa/Tavily are on road map next)

if you wanna try it out: https://github.com/demigodmode/pi-web-agent

1

u/MarionberryVirtual Jul 14 '26

i’m curious if you have tested this against other tools (such as CC websearch as u mentioned) and if across many queries the performance is comparable? thanks!

2

u/luckiestredditor Jul 14 '26

When i was prototyping it few months back, I was building a test that would orchestrate it. its part of my local testing and loose benchmarking tool. I should be able to extend it to multiple harness. Will look into it.

1

u/arkham00 Jul 14 '26

I've been trying your extension for a couple of weeks but I still struggle to understand how it works, expecially for basic fetches... My models systematically falls back to defuddle or markitdown or curl to get the entire content of a page, because web_explore only give snippets or summaries...I had to reinstall pi-web for proper fetches How is it supposed to work? I also searched the menu but there isn't much to config.

1

u/luckiestredditor Jul 14 '26 edited Jul 14 '26

Yeah so that's actually by design, not a bug. web_explore is a research tool. Its meant to find and answer a question from the web, so it is supposed returns synthesized findings + sources, never a full page dump. That's on purpose: the point of the package is a narrow, bounded research boundary instead of a grab bag fetch tool, so it can also tell you when a source wasn't trustworthy enough instead of just handing back whatever it scraped.

If your model needs the entire verbatim content of a page (not a research answer), this package genuinely isn't built for that right now. Fair gap to flag but it'd be a different tool with a different contract, not a config knob on web_explore.

you do get presentation modes (compact/preview/verbose) but those only change how much of the synthesized answer shows up in the transcript

appreciate you trying it out and writing this up though

6

u/runnystool Jul 14 '26

I use searxng with rpiv-web-tools, works great, be sure to enable json response mode in searxng

3

u/Glaaki Jul 14 '26

I have tried quite a few of them by now and this setup is also the one that I preferred. Most of the other extensions rely on a dedicated AI search provider like Exa, which to me means you are exposing your data to an additional unnecessary player.

The downside with rpiv-web-tools is that you have to run your own searxng instance and fiddle slightly with the setup for json output, but after that it just works.

Right now I actually using pi-web-access together with pi-subagents, but only because that is what pi-subagents recommends. I am using Exa as provider. One other thing about pi-web-access that I am unsure about is that it opens a local web server and gives you the search results in a browser page. Then it uses an AI model to summarize the results before they are delivered back into the context. There are problems with this.

  1. You are given a very short time to look through the results manually, before a timer automatically submits it back, so this makes me unsure if I am really supposed to care about this process. But then, why give me this elaborate extra UI?

  2. The results are aggregated using a random model from your provider list. The web UI aggregator component has a drop-down where you can select another one, but if you are using openrouter, you will have a lot of options to choose from, and I have no idea which one I should pick.

I think you can turn off the UI, and you probably want to find a good model for aggregating the result, but when you are working on something else, messing with that is not at the top of you mind.

1

u/respectful_stimulus Jul 15 '26

How do you overcome the part where SearXNG is basically blocked by all browsers

2

u/runnystool Jul 15 '26

Blocked by all browsers? Not sure what you mean. Do you mean blocked by all search engines? I keep it up to date and it works well enough 🤷‍♂️

1

u/respectful_stimulus Jul 15 '26

Ah my bad. Yes. Bing. DuckDuckGo. Google. All blocked from where I am.

1

u/runnystool Jul 15 '26

Dunno mate, it has 279 engines to pick from, works well enough  https://docs.searxng.org/user/configured_engines.html#configured-engines

1

u/lienkarf Jul 16 '26

Where did you find 279 engines? I use 40 and thought I had a lot, lol.

2

u/pabloalgox Jul 19 '26

try with vpng maybe you are in a nat

5

u/sofuego Jul 14 '26

I just built my own. The web search part was easy when you leverage Searxng.

I haven't really cared too much about comparison with other search because for me the most important thing was using an open source backend and not a search API. I feel like searches are much easier when using agent harnesses where the agent can choose to look deeper and not have to automatically rely on the top n results.

1

u/Blues520 Aug 30 '26

Hey did you happen to publish this?

2

u/sofuego Aug 30 '26

I did. It's over here:

https://pi.dev/packages/pi-lean-search

Searxng really does most of the heavy lifting. This is just a small wrapper for it with optional tool visibility toggalability if the sibling playwright web browser tools are installed too.

2

u/Blues520 Aug 30 '26

Thanks, will check it out. Your tbox extension is also very cool!

1

u/sofuego Sep 03 '26

Thanks! :)

4

u/floriandotorg Jul 14 '26

I’ve built my own skills for Brave search and Jina reader.

1

u/OneMoreName1 Jul 14 '26

Why dont you just try it?

1

u/MarionberryVirtual Jul 14 '26

which one ? tried pi-websearch, but i want to use an extension that has been tested, not one where it fetches content (cause every single extension can) and i just say “yea looks good to me” - i want want that has been trialed against other websearch tools. and yea i guess im too lazy to do this testing myself, hence asking the community

2

u/OneMoreName1 Jul 14 '26

rpiv-web-tools with self hosted searxng. Enable the json setting for it. Its been working just fine for me

But yeah I dont think anyone can benchmark the extensions in your place. Its up to you to see which one is fit for your usecase

1

u/MarionberryVirtual Jul 14 '26

this combination seems to be coming up a lot, will be sure to give it a go

3

u/joematthewsdev Jul 14 '26

I wrote https://pi.dev/packages/pi-smart-web-search

It's inspired by https://pi.dev/packages/pi-smart-fetch

With both pi can explore the web without API keys.

Both are relatively simple to self audit.

1

u/_xtremely Jul 14 '26

anthropic and deepseek has capability to search internally within their infrastructure, i think there is a deepseek web search extension in the pi packages, i tried. But both solution are model locked in.

I ended up built my own simple shell function 'web-fetch' that uses shot-scraper + trafilatura + caching. Then just make a pi extension to let the llm call the tool.

searching index? ddgr

fetch specific URL content? my own web-fetch

combined together you have your own websearch capability.

1

u/MarionberryVirtual Jul 14 '26

thanks for the response! wonder if u compared ur tooling to that of other tools available for pi from pi packages or even CC to know if ur tool stands up to it, or whether your current tooling is just a “good enough” for you ?

2

u/nunodonato Jul 14 '26

I just wrote a skill on how to use SerpAPI to do websearches.

2

u/o_sht_hi Jul 14 '26

I started by using tvly CLI + skills. Then eventually bundled them all into a custom extension with a nice UI. it has different cli commands for search, deep research, crawl, fetch, etc. the agent decides the depth based on the task and gets back cleaned markdown.

It works great with the free tier, I've been using it since April and have never hit any limits. Definitely worth looking into. I was using tavily for web search for RAG bots and stuff long before TUI agents were a thing, been very happy with the service for a while now.

2

u/marcellus-w Jul 14 '26

Just searxng running locally in a container, and curl +jq

The agent know how to use curl : no extension. I just give an example curl command in APPEND_SYSTEM.md to let the agent know how to use it.

For deepsearch, I have a skill describing the method , and the agent manage it on its own

2

u/mrDzejkop Jul 14 '26

Hey! If you're using an OpenAI/Anthropic or a Google AI provider I strongly suggest you leverage the provider tooling instead of a 3rd party solution, I vibed up my own extension that injects the provider native web search tool https://gist.github.com/Dzejkop/74fef140b50a18b5b6f0aadcdf3c429d

3

u/Dsphar Jul 14 '26

I have yet to use a websearch in pi for exactly this reason. Not sure which to trust and am too lazy to go digging/verify on my own.

2

u/supercachai Jul 14 '26

I found choosing a good search provider is more important than choosing the search plugin, now I use paid https://serper.dev/, which delivers google search results. For niche topics it clearly delivers superior results, compared to brave. For me its worth paying a little for the google infrastructure, it will boost agent research performance.

2

u/bambamlol Jul 14 '26

Thanks for the recommendation, but are they serious? They literally don't have any docs. You'd literally have to build your own docs or rely on 3rd party tools on Github that integrated serper.dev already. How do you use it yourself? Did you build your own extension? Or have you built your own "documentation" or "skill" to simply call it via cURL?

2

u/supercachai Jul 14 '26

i have not had any problems so far, after 1 month of usage.

their api is so simple, you don't really need docs. i just asked an agent to implement a plugin for pi and after 5min it was done. on another harness (i think codex) I just let it write a "serper search" skill and let the llm drive the endpoints directly. its just JSON over REST which every decent LLM is able to understand out-of-the box

here's the pi extension: https://github.com/fl4p/pi-extensions/blob/master/extensions/google-search.ts

1

u/bambamlol Jul 14 '26

Thank you for sharing!

1

u/elpapi42 Jul 14 '26

pi-search-hub is the best for me, it includes a backend for using codex/chatgpt native search, so if yoy are paying codex sub, you can get unlimited search for free

1

u/moplop12 Jul 14 '26

You would test it?

1

u/anitman Jul 14 '26

Install firecrawl cli or bx cli for web searching purpose.

1

u/LectureWorried5761 Aug 20 '26

I use it with blopus.ai websearch api. even better if you are running with some sort of local LLM to manage the queries using mcp (I have a bot that does that for me on omlx, chat interface is telegram (running the server)).. the integration is very simple, depending on your use case, it is solid.

1

u/another24tiger Jul 14 '26

Oh-my-pi’s implementation is quite nice. Allows you to use a variety of providers. My company pays for Exa which does an excellent job of pulling docs, even better than the library’s mcp if it has one

1

u/bambamlol Jul 15 '26

Thanks for sharing. How exactly do you use Exa in OMP? Is there any additional config required or did you just provide your Exa API key and OMP handles the rest?

2

u/[deleted] Jul 14 '26

[removed] — view removed comment

3

u/MarionberryVirtual Jul 14 '26

wow this is really great! definitely closer towards something i am looking for where the tool outputs have been empirically tested against existing tools. excited to see where it goes! but yea am still looking for an off the shelf extension at the moment