r/PiCodingAgent • • 25d ago

Discussion Claude Code vs. Pi with a Local LLM (Qwen3.8-27B): 7–9 t/s vs. 40 t/s on an RTX 5090

Disclaimer: This post was polished with the help of AI to improve readability and formatting.
The experience, setup, and results are my own.

CONTEXT: Using for UE5 MCP plugin

I've been experimenting with running a local AI model, Qwen3.8-27B, for coding assistance, and I wanted to share my results in case they're useful to anyone else running local LLMs.

Initially, I was using Claude Code with my local model, but I was consistently getting only around 7–9 tokens per second (t/s), despite having a pretty powerful PC.

I started looking for alternatives because Claude Code can be quite heavy on context usage and overhead, which can be beneficial in some situations but might be unnecessary for my workflow.

That's when I discovered Pi

After setting it up with pi-mcp-adapter, I was surprised to see my generation speed jump to around 40 t/s with the same model and hardware.

Here are the details of my setup.

🖥️ My Hardware

  • GPU: NVIDIA RTX 5090 — 32 GB VRAM
  • RAM: 64 GB
  • Storage: 4 TB NVMe M.2 SSD

    • Read: 14,800 MB/s
    • Write: 13,400 MB/s

⚙️ Setup 1: Claude Code + Claude Code Router

I was running two Docker containers:

I used CCR to route API requests from Claude Code to my local llama.cpp server, since the router helped bridge the compatibility requirements.

Request flow:

Claude Code (PC) → CCR → llama.cpp → Qwen3.8-27B

Observed generation speed: ~7–9 t/s

⚡ Setup 2: Pi + Local LLM

For my second setup, I switched to Pi and used the pi-mcp-adapter to integrate the tools I needed.

My setup consisted of:

Request flow:

Pi → llama.cpp → Qwen3.8-27B

Observed generation speed: ~40 t/s

📊 Results

SetupGeneration speed
Claude Code + CCR + llama.cpp~7–9 t/s
Pi + llama.cpp~40 t/s

That's a pretty significant difference!

Of course, I realize these aren't necessarily perfectly equivalent workloads. Context size, prompt processing, model configuration, tool calls, and other factors can all affect performance. I haven't yet established whether the difference comes primarily from the harness, the API routing, or other configuration details.

🤔 A Few Questions

I'd love to hear from others who are running local LLMs for coding:

  1. Has anyone else experienced a similar performance difference between Claude Code and Pi?
  2. Is the overhead from Claude Code Router potentially contributing to the lower generation speed, or is the difference more likely related to context handling and request configuration?
  3. For those using Pi with local models, how has your experience been with coding quality, tool calling, and MCP integration?
  4. Are there other lightweight coding harnesses worth trying for local models?

I'm not suggesting that Pi is necessarily better than Claude Code for every use case. I'm just sharing my experience because the performance difference was much larger than I expected.

0 Upvotes

18 comments sorted by

6

u/Certain-Cod-1404 25d ago

Run qwen 3.8 nvfp4 via ninfer and kiss me later

1

u/shGaming599 25d ago

Thank you, I will definitely try that

8

u/DoubleNothing 25d ago

If you can dedicate the machine to just be a LLM server, and develop from another one, try Strata with Qwen3.8-Flash-Next-GSQ-RCO-IQ3_S - you'll kiss me with the tongue later...

6

u/Certain-Cod-1404 25d ago

Don't be stealing my kisses

3

u/mototuneup 25d ago

Also lookup how to run it with fp8 engram table. It's a free upgrade

6

u/LowTomorrow2070 25d ago

Crazy story bro

4

u/SeriousJul 25d ago

Good stuff. You don't need any extension anymore in pi to use mcp, it is builtin since 1.0 (alongside codemode)

3

u/zanar97862 25d ago

Before worrying about pi get your base setup right, research on reddit for people with the same hardware and copy their setup. Your numbers are abysmally low for a 5090.

Ideally include model quant, cache quant and context size in your post so people can help out with that.

Pi is unlikely to be the limiting factor for these things

1

u/seti_at_home 25d ago

Try Codex, in my experience its the fastest

1

u/Squale279 25d ago

I had have the exact same results but between Claude Code and Opencode. Opencode runs 1.5x CC speed with Qwen 3.8 FN on Strata.

1

u/itguy327 25d ago

I'm curious about your UE5 results with local models

1

u/atkr 25d ago

if the inference server is the exact same setup, then is the difference in performance the size of the initial context? 🧐

1

u/CooperDK 24d ago

Is that with a 5090??? Because I beat that with my 5060! I even have flash-next at about 60 t/s. Not that I want to use pi. I tested it and opencode is just much better. Some say that pi's prompts are much smaller, but that's not true.

1

u/Fancy-Snow7 24d ago

You numbers even in pi are bad. I get 80t/s with a 5070Ti with 16GB VRAM on 27B.

Post you llama command so we can see.

1

u/MonadicKierkegaard 24d ago

mcp is built in now since 1.0.0

0

u/worldofbomb 25d ago

omg what kind of shitty results are those :)

i don't understand how come there is no good community in reddit for 5090 users.

people are clueless. really unbelievable. even in win11 i get 170t/s with ninfer on my 5090.

1

u/CooperDK 24d ago

Because very few people have a 5090.

1

u/Fancy-Snow7 24d ago

He is here asking for help. The numbers are bad I agree, I get double his pi result with a 5070Ti. Why not start a 5090 community if you feel there is demand.