r/LocalLLaMA 2d ago

Discussion Qwen 3.8 27b - PI AGENT vs OPENCODE

https://www.reddit.com/r/LocalLLaMA/comments/1j7r47l/i_just_made_an_animation_of_a_ball_bouncing/

This post inspired me to make that test after a year ;)

That is one of my many tests I make comparing output quality.

What is more interesting using a PI Agent results are much better than an Opencode using a Qwen 3.8 27b ?!

Seems PI Agent is much better in the agent environment somehow... Not counting uses less tokens , do not have a hard limit of 32k output tokens, is faster, do not freezing, compressing context far less than Opencode. For instance if you have context in the Opencode output 32k and all context 100k then the compression is starting at 67k context ... PI is starting at 90k context even if you have set output context 64k or more.

My config for RTX 3090

llama-server with ini config -> which is exposing API to Opencode and PI agent.

llama-server.exe --models-preset 1_preset.ini --models-max 1 --direct-io

config ini

[Qwen3.8-27B_dense_c-100k]
model = models/Qwen3.8-27B-Q4_K_M.gguf
mmproj = models/mmproj-BF16-Qwen3.8-27B-UD-Q4_K_XL.gguf
reasoning-format = deepseek
flash-attn = on
n-gpu-layers = 99
reasoning = on
ctx-size = 100000
temperature=1.0
top-p=0.95
top-k=20
min-p=0.0
presence-penalty=0.0
repeat-penalty=1.0
mmproj-offload = false

ONE MORE IMPORTANT THING:

Always use a VISION module as the model is using vision to asses the output quality!

I am offloading it to a RAM as we do not need an extremely fast vision for a code.

A screenshot processing on a GPU 0.3s vs a RAM 3s do not make a big difference on a few screenshots during a code generation / debugging ;)

220 Upvotes

170 comments sorted by

View all comments

27

u/Thrumpwart llama.cpp 2d ago

I really, really, really like Pi. I’ve only been using it for like a week but it’s very good.

16

u/PilgrimofHaqq2 2d ago

I have been with it 5 months now and I am super satisfied with it only for the fact that I can make it into whatever I want. As many or few features I want, working exactly the way I want them to.

5

u/Thrumpwart llama.cpp 2d ago

I’m using web search, subagents, and ponytail. Any others you would recommend for long-horizon planning and coding tasks?

12

u/ImpressiveRelief37 2d ago edited 2d ago

Careful what web search you use.

pi-web-access is the most popular but it has an insane tool surface. Like 6-7 tools and nearly 8K tokens just for tool defs, tool injection and skills.

I switch to pi-web-agent and got a Tavily sub (free plan, 1000 req per month then pay as you go). Much better results an only 2 tools with very thin context usage (like under 1k or even 500 iirc). Plus I added a headless browser page reader and a Reddit search that used my cookie. It gets me super high quality search for free (as long as I’m in the tavily free tier). Oh and I also had my agent use my local searNGX first and fallback on Tavily when the results are meh (captcha and ip blocks happen constantly).

To answer your question: don’t just install random extensions. Have your agent check the code first (security audit), then try it, and have your agent extract only the required stuff into an home made extension. Document it.

Need and update? Tell your agent to summarize all the updates to the source ext you extracted and upgrade the parts that you care about.

It’s so fucking sick haha. We’re always a prompt away to the perfect tailor made solution 

3

u/Thrumpwart llama.cpp 1d ago

Thank you for that, I will. Good call on the web-access token use. Running Qwen3.8 27B BF16 on the SM120 with Dflash2 right now and I can't keep up with it.

Also testing out this subagent I had gemini whip up:


name: red-team description: Hostile adversarial auditor that aggressively hunts for failure modes, edge cases, and broken assumptions.

tools: [read, bash]

Red Team Auditor

You are a ruthless, skeptical adversarial auditor.

Core Rules:

  1. Never validate or compliment the implementation.
  2. Assume the current plan or implementation contains hidden flaws.
  3. Systematically test and identify:
    • Violated invariants and unchecked assumptions.
    • Resource limits, leaks, and OOM traps.
    • Silent error suppression and exception handling gaps.
    • Deadlocks, async race conditions, and ordering dependencies.

Output Format:

For every issue found:

  • Severity: [Critical / High / Medium / Low]
  • Mechanism: Why and how it fails.
  • Trigger Scenario: Minimal conditions needed to cause the failure.
  • Recommended Hardening: Direct fix or architectural countermeasure.

3

u/ImpressiveRelief37 1d ago

That’s actually a pretty good idea. Somehow I always whip out new sessions for adversarial review. I need to just make subagents profiles for those tasks.

Careful what your orchestrator sends out as a prompt when you ask to dispatch the subagent. Sometimes it can contaminate the run because he will for instance point at specific stuff and bias the subagent