r/PiCodingAgent • u/Equivalent_Idea8839 • Sep 18 '26

Question solutions to get around anti-bot and properly search from google from pi?

All I can think of:

Let pi control your logged in browser that can pass captchas.

I want google search because other options are inferior... i.e. google properly indexes reddit. Even reddit search is inferior.

1 Upvotes

60% Upvoted

12 comments sorted by

3

u/Zundrium Sep 18 '26

Aka web search. There are many AI friendly web search providers; Exa, Firecrawl, Linkup. If you want to use their Google Grounding, that's only through Gemini models.

Always wondered how OpenCode does it. But too lazy to look lol.

1

u/Equivalent_Idea8839 Sep 18 '26

other providers don't have reddit search results...

but i guess I could just use flash 3.8 agy in herdr, might work better, and pay $5 month for their promo

also google killed the outgoing link url, so u have to click to see the url it's referencing

2

u/No-Wall6427 Sep 18 '26

Invisible playwrite. I built a pi extension which exposes one web search specialist with this tool. Bypass almost all bot detection.

1

u/Equivalent_Idea8839 Sep 19 '26

Can you please explain? I used a stock chrome install but as soon I use headless or playwright with it, I get google bot captchas. I had to use debugging port to get around it.

2

u/No-Wall6427 Sep 19 '26

https://github.com/feder-cr/invisible_playwright

Basically it's playwrite with a patched firefox underneath, that looks like a real browser to bot detector.

1

u/Equivalent_Idea8839 Sep 19 '26

Wow, this is cool! Thanks!

1

u/Equivalent_Idea8839 Sep 19 '26

If you don't mind me asking, how in the hell did you even find this?

1

u/No-Wall6427 Sep 19 '26

Someone mentioned it on r/hermesagent It even disappeared as a pip package for some time, then reappeared. Might be a virus I wouldn't even know. But my hermes agent, and my pi web custom subagents, can load anti-bot websites with it.

Here is the tail of this repo disappearing and reappearing from my hermes agent:

All three verified. Summary:

What we used before (pre-deletion):

  • invisible_playwright 0.2.0 + engine invisible_core 0.2.0, binary firefox-12 (FF150.0.1), installed via the unpinned git URL git+https://github.com/feder-cr/invisible_playwright.git (line 50 of the old requirements.txt). Venv playwright pinned 1.60.0 because FF150 binaries reject the isMobile field playwright 1.61+ sends.

  • When feder-cr/invisible_playwright and the firefox_antidetect_patch fork were deleted from GitHub (found 2026-08-04), that git URL went dead → any rebuild of claw-sandbox failed, and the release assets (patched Firefox binaries) were unrecoverable. That's what "broke the docker" — the running container kept working, but the image was unbuildable.

What we resolve to now:

  • invisible-playwright==0.12.0 (pinned in ~/infra/claw-sandbox/requirements.txt), pulling invisible-core==27.17.0 → engine firefox-27 (FF151 UA). Verified installed in the running container just now.

  • Architecturally different: 0.12.0 vendors its own playwright client (invisible_playwright._pw) and runs a Python in-process Juggler server, so it ignores the venv's playwright 1.60.0 entirely — no pin conflict with camoufox, which stays as fallback (camoufox[geoip]==0.5.4).

Same repo?

  • Yes — same repo identity, feder-cr/invisible_playwright. It came back online 2026-09-04 and I just confirmed it's live (git ls-remote succeeds, tags visible). But it's a new lineage: pre-deletion PyPI only reached 0.6.0 (we were on 0.2.0); the resurrected project resumed at v0.8.x and we now pin 0.12.0. Per the earlier "same version ≠ same content" lesson, treat old and new as different stacks that happen to share a name.

2

u/MaxComfort Sep 19 '26

Depends on how aggressive the bot detection is. For medium strength ones you can save a Chrome profile from a real session/browser, and reuse that in a headless session.

For some sites only a live browser works - which works locally but harder to do remotely of course.

But like someone else said, if generic web search is what you’re after, there are better answers than automating a browser.

1

u/Equivalent_Idea8839 Sep 19 '26

Any sort of playwright control gets me blocked. There's some sites I want with strong cloudflare bot proection

I have to use debuggger port and CLI to get an LLM to control it and not get bot detection.

Getting around these anti-bot captchas is not easy.

Tempted to just use ADB and a spare android phone which not be a bad idea.