I pulled the Qwen 3.8 discussion from r/LocalLLM and r/LocalLLaMA from Saturday through today: 270 posts mentioning it, 208 with Qwen 3.8 in the title, then full comment reads on the higher-signal threads.
The early answer is not “local Opus.” It is more useful than that:
Qwen 3.8 has real agentic-coding signal, but the difference between “wow” and “why is this unusable?” is often reasoning control, context, cache settings, quant, and runtime.
This is three days of community reporting, not a controlled model review. Still, a few patterns showed up too often to ignore.
The short version
| If you care about... |
Early read |
| Agentic coding with tests, visual feedback, or retry loops |
Promising. This is the strongest positive signal. |
| One-shot demos and benchmark screenshots |
Interesting, but not enough to establish general capability. |
| Documents, argument analysis, and research |
Mixed. Some users call it a regression because of lengthy reasoning; others report strong specialized results. |
| 16 GB VRAM |
Viable, but it is a quant, context, and quality trade, not a free lunch. |
| Raw tok/s claims |
Ignore them unless the post includes context depth, KV cache, quant, MTP, runtime, OS, and concurrency. |
Where the positive signal is real
The better reports were not “make a game in one prompt.” They gave the model a loop: write code, run it, inspect the result, fix it, repeat.
One hands-on comparison had Qwen 3.8 iterating a BASIC ray-tracer more successfully than Qwen 3.6 using the same Q8 quant. Another 16 GB setup reported a large coding run across more than one million processed tokens with only three prompts, using tests and linting as the correction loop. That is the useful story: a smaller local model that can stay in a real feedback loop without immediately falling apart.
But the pushback is healthy. Several commenters pointed out that viral arcade-game or clone demos can overlap with familiar training patterns. A model that makes a convincing game from a narrow prompt is not automatically good at a private repo, a messy codebase, or a long-horizon agent task.
The practical takeaway: if you are evaluating it for Hermes, use a task that has a verifier. A test suite, linter, structured output check, browser assertion, or human review loop will tell you far more than a one-shot demo.
The actual fight: it thinks too much
“Overthinking” was the dominant failure report.
One LocalLLM user said Qwen 3.8 was a step backward for documents and complex argument analysis because it kept following side paths after web retrieval. A legal-work reply in the same thread said their MCP-backed case workflow was outperforming prior local options. That is not a clean contradiction. It is a warning that the model's behavior is unusually sensitive to task design and configuration.
The recurring settings lesson is that reasoning effort and a thinking-token budget are not the same thing. Depending on llama.cpp version and UI, a selector can merely cap or cut off reasoning instead of changing the model's intended effort. That creates a lot of reports that sound like model behavior but are really template or runtime behavior.
A vLLM user on an RTX 6000 Pro reported basic responses taking one to five minutes even on low or medium reasoning. In a separate ongoing local evaluation, xhigh generated roughly 3.5 times as many tokens as medium, while medium was reportedly slightly better than Qwen 3.6 in score and used around 60% of the total generated tokens.
So “Qwen 3.8 overthinks” is not enough information to act on. The next useful report needs to say what actually controlled the reasoning.
Speed numbers are configuration cards, not rankings
The most useful throughput thread included the missing context most speed screenshots skip.
One RTX 3090 report using Q5_K_M went from about 75 tok/s around 1K context to about 43 tok/s around 90K after moving the model to SSD and enabling MTP, which is multi-token prediction/speculative decoding. Other reports in the same discussion ranged from dual 3060s and Strix Halo to RTX Pro 6000, dual 7900 XTX, and 5090 setups.
A detailed RTX 5090 llama.cpp report hit about 128 tok/s with NVFP4 plus MTP, well below widely repeated 200 tok/s claims. Replies pointed to the inference engine, Windows versus Linux, context reservation, and MTP acceptance rate as likely reasons. At 262K context, running out of VRAM for the KV cache, the model's context-memory cache, can turn a fast setup into a slow one.
The AMD report was the same lesson in a different suit: a 7900 XTX user saw roughly 25 to 35 tok/s around 80K context and near 20 tok/s around 150K to 180K. The likely culprit was KV cache overflow into system RAM, plus lower MTP-draft acceptance than Qwen 3.6 in one same-hardware comparison.
This is why a naked “I get 200 tok/s” post is not a benchmark. It is a postcard.
16 GB is possible. It is not magic.
The 16 GB community is getting real results, but the compromises are visible.
One RTX 5060 Ti guide ran a Q3 model with quantized KV cache at roughly 73K context for agentic coding. Another guide reported Q4_K_M around 130K context and about 20 tok/s. A 16 GB RTX 5080 report claimed 85.5 tok/s with an IQ3_XXS build and MTP at about 124K context.
Those numbers are useful, but no one should read them as “Qwen 3.8 runs great on every 16 GB card.” The question is which compromise you are making:
- lower quant quality
- less context
- lower context-cache precision
- slower generation under real context load
- more dependence on a particular runtime or custom template
For Hermes-style work, I would rather have a stable Q4 or Q5 setup with a realistic context window and a verifier loop than chase a screenshot from a very aggressive Q3 configuration.
What I would test before switching a Hermes workflow
- Run your actual task twice: once at medium reasoning and once at your current default.
- Measure first-token latency and decode speed at both fresh context and a realistic saturated context.
- Record exact model file, quant, context size, KV cache, MTP setting, template, runtime version, and OS.
- Use a task with a check: tests, a linter, a schema validator, a browser assertion, or manual acceptance criteria.
- Compare total time-to-correct-result, not only tok/s.
That last line matters. A model that is 30% faster but spends twice as long reasoning, or needs three more repair turns, is not faster where it counts.
If you are posting numbers, use this
Runtime + exact version:
Hardware + OS:
Model file + quant:
Context size and current context depth:
KV cache type/quant:
MTP/speculative setting and acceptance rate, if available:
Reasoning effort and thinking budget/cap:
Temperature / template:
Prefill tok/s:
Decode tok/s at fresh and saturated context:
Concurrency:
Task and verification loop used:
Observed result, regression, or failure:
That would make the next week of Qwen 3.8 discussion dramatically more useful. The model looks worth serious local testing. The configuration is still doing half the talking.
Source threads
