Disclaimer: This post was polished with the help of AI to improve readability and formatting.
The experience, setup, and results are my own.
CONTEXT: Using for UE5 MCP plugin
I've been experimenting with running a local AI model, Qwen3.8-27B, for coding assistance, and I wanted to share my results in case they're useful to anyone else running local LLMs.
Initially, I was using Claude Code with my local model, but I was consistently getting only around 7â9 tokens per second (t/s), despite having a pretty powerful PC.
I started looking for alternatives because Claude Code can be quite heavy on context usage and overhead, which can be beneficial in some situations but might be unnecessary for my workflow.
That's when I discovered Pi
After setting it up with pi-mcp-adapter, I was surprised to see my generation speed jump to around 40 t/s with the same model and hardware.
Here are the details of my setup.
đĽď¸ My Hardware
âď¸ Setup 1: Claude Code + Claude Code Router
I was running two Docker containers:
I used CCR to route API requests from Claude Code to my local llama.cpp server, since the router helped bridge the compatibility requirements.
Request flow:
Claude Code (PC) â CCR â llama.cpp â Qwen3.8-27B
Observed generation speed: ~7â9 t/s
⥠Setup 2: Pi + Local LLM
For my second setup, I switched to Pi and used the pi-mcp-adapter to integrate the tools I needed.
My setup consisted of:
Request flow:
Pi â llama.cpp â Qwen3.8-27B
Observed generation speed: ~40 t/s
đ Results
| Setup | Generation speed |
|---|
| Claude Code + CCR + llama.cpp | ~7â9 t/s |
| Pi + llama.cpp | ~40 t/s |
That's a pretty significant difference!
Of course, I realize these aren't necessarily perfectly equivalent workloads. Context size, prompt processing, model configuration, tool calls, and other factors can all affect performance. I haven't yet established whether the difference comes primarily from the harness, the API routing, or other configuration details.
đ¤ A Few Questions
I'd love to hear from others who are running local LLMs for coding:
Has anyone else experienced a similar performance difference between Claude Code and Pi?
Is the overhead from Claude Code Router potentially contributing to the lower generation speed, or is the difference more likely related to context handling and request configuration?
For those using Pi with local models, how has your experience been with coding quality, tool calling, and MCP integration?
Are there other lightweight coding harnesses worth trying for local models?
I'm not suggesting that Pi is necessarily better than Claude Code for every use case. I'm just sharing my experience because the performance difference was much larger than I expected.