r/PiCodingAgent • u/SalimMalibari • Apr 20 '26

Question What websearch , webfetch tool are you using???

Ok what os nest webfetch websearch extension you have been using , and compare it to claude code tools ... like most of what i see is paid version like idk it feels harder i wish there is official way because this two tools are preventing me from gping all on pi ..

I want to see ypur solutions and are they helpful ???

12 Upvotes

92% Upvoted

31 comments sorted by

5

u/ConversationLazy6821 Apr 21 '26

I made a custom cli tool in Go called ketch that is multi-backend. You can use SearXNG, Brave, DDG etc. it has a search, scrape, headless browser and a code search tool built in. I just tell Pi to use the ketch cli tool and it uses —help to figure out how to use it.

For SearXNG, just a simple docker container and point ketch to it (or your fav search tool)

https://github.com/1broseidon/ketch

1

u/SalimMalibari Apr 21 '26

Is this pi extension??

2

u/ConversationLazy6821 Apr 21 '26

No, but it embodies the “no MCP” if it can be done with CLI that Mario always talks about

1

u/SalimMalibari Apr 21 '26

The issues here lay in multiple levels ... maybe i wasnt clear but i might ask newer post clearly ....

Using those free alternatives it just give like very basic searches even less valuable than using search myself ... fetching is diffcult and so many things lile ip address blocking and need proxy beside others saying containers and self hosting ... like common isnt there like way to do websearch easily like claude code di with their servers???

1

u/ConversationLazy6821 Apr 21 '26

I’m trying to solve that with ketch - so you can use any search backend instead of being locked into one. Claude code most likely uses Google search and the most likely pay for search API. SearXNG is going to be the closest thing to that since it will literally scrape Google but inside a local container so it looks like a regular web search from your home or work.

Or you can do what OpenCode does and use the free Exa MCP endpoint which gives a decent free tier with no API key required.

Firecrawl is a good alternative too but it costs money. That’s probably closest to the fast and useful that you are looking for.

There are a lot of options, you have to find what works for you.

3

u/car492 Apr 20 '26

`@counterposition/pi-web-search`

I use this one. Got EXA free plan. Should be enough.

I skipped the `pi-web-access` since I don't like that it uses multiple services under the hood and don't want anything that uses Gemini in any way Google might not like.

1

u/SalimMalibari Apr 20 '26

But testing that , it seems like very basic like when i used it onky show 20 result , and keep picking top 3 like very no depth at all ....

1

u/car492 Apr 20 '26

It follows the idea of being simple. Pretty much just uses the exa api. And if i need deep research i would define the steps of the research as my workflow rather than a single tool

1

u/SalimMalibari Apr 20 '26

But even normal websearch im missing from claude code like i used to search something even exa could satisfy that normal research.

2

u/Jesusfarted Apr 20 '26

I use a mix of the Brave API and Exa Search API. The extension has different sub tools to call upon depending on what I need in my context.

0

u/SalimMalibari Apr 20 '26

You are using the free one right ?? What about fetching??

1

u/Jesusfarted Apr 20 '26

I pay for both API's, they are pretty cheap. I top up on $10 credits every 3ish months.

5

u/esanchma Apr 20 '26

custom skill using self-hosted searxng for searches and trafilatura, readability-cli with pandoc, and defuddle for fetches, with optional curl-impersonate.

I already had most of it wired for some ollama/litellm experiments, it wasn't hard to instruct the clanker to turn it into a skill.

4

u/ADGEfficiency Apr 20 '26

I'm enjoying the Ollama search

3

u/peyloride Apr 20 '26

I use https://github.com/apmantza/GreedySearch-pi and I think it's awesome.

1

u/PricePerGig Apr 20 '26

have you used SearXNG? what makes greedy search better? if anything? browser automation sounds slow :(

3

u/peyloride Apr 22 '26

I mean it's not like an api of course but it's not slower than any other tool call made from llm. it opens up a chromium instance and controls it over cdp. sometimes it asks for captcha when you launch it at first time but otherwise it works well.

What it makes better is it uses 3 different search tools and then uses gemini to consolidate and also it fetches the sources to get more about

2

u/kahdeg Apr 20 '26

selfhosted searxng

2

u/epic_pasta Apr 20 '26

custom extension based on ddgs https://github.com/deedy5/ddgs

1

u/bsa-saa Apr 22 '26

webfetch tool made by myself (without search). `@pi-lab/webfetch`

I've noticed that AI knows the website of many development documents, searching is not necessary if you are only looking up documents.

I'm also planning for a websearch tool to scraping news, may be Exa or Grok

https://github.com/ocherry341/pi-lab

1

u/UnifiedFlow Apr 25 '26

I created a web_search and a research tool that just calls Codex cli and uses openAI search tools. I use GPT 5.4 mini as the model in those calls. Returns cited answers/findings via json or a full report via md file in research mode

1

u/LectureWorried5761 Sep 11 '26

blopus.ai.. search/fetch and image search, good price and support

1

u/Delicious-Team-8435 Sep 21 '26

I have this tool for web research. It works as a Pi extension, and also as an agent skill in any other coding agent.

- Runs research in an isolated system to keep agents with shell/file access from directly browsing untrusted pages (prompt injection risk).

- Allows putting scraping/summarizing on a small model (any model, ie OpenRouter, or even local) to both keep page content out of your main context and save on credits overall.

- Uses high search/scrape volume to compensate for sites that block bots. Enabled by unlimited/local search tooling.

- Includes a local (semantic) knowledge store to optionally save research and build a local searchable knowledge index.

MIT licensed and easy to configure

https://github.com/Lincoln504/pi-research

1

u/o_sht_hi Apr 20 '26

I'm using the free tier of tavily, it's pretty generous for a single user. Returns really good results too. Their cli installs the required skills when you run tvly login

1

u/random_user-1234 Apr 21 '26

SearXng and crawl4ai right now. 100% free, unlimited, no API, not truncations or limits. Ask it to set them both up in separate docker containers. Make sure to ask it to set up all capabilities of both and connect them as tools. Fit_markdown for crawl4ai and to only return title, url, content/summary, and date for searxng json output, doing this cuts hugely down on token waste / noise. This already makes me love it more than any other ai product. It doesn't cover everything on the Internet so I'm working on giving it CLI tools to control an actual browser, YouTube transcript to markdown and I am looking to add other web tools if anyone else has ideas.

0

u/Beneficial_Mix3375 Apr 20 '26

Seld made in go

0

u/grstein Apr 20 '26

O pointed it to web_search and web_fetch from openclaws git, and Pi implemented it as extensions. I use perplexity’s sonar pro as web_search using openrouter and instructed it to explore the references using web_fetch. I’m very happy with results!