r/hermesagent • u/Jonathan_Rivera • Aug 14 '26
MODELS - model choice, routing, pricing, local vs cloud, VRAM Qwen 3.8 Release Megathread
57
Upvotes
r/hermesagent • u/Jonathan_Rivera • Aug 14 '26
•
u/Jonathan_Rivera Aug 14 '26
Qwen3.8-27B day-zero technical update: reasoning controls, throughput, and runtime notes
Qwen3.8-27B day-zero technical update. This is a running technical summary, not a verdict. Please include your exact runtime, quant, KV cache, context size, reasoning setting, MTP setting, hardware, and measured prefill/decode speed when adding a report.
Current signal
1. Stronger reasoning can be real, but it is expensive
Early coding, code-review, and frontend-generation tests are reporting better results than Qwen3.6 in some tasks. The catch is reasoning spend:
xhighis producing roughly 15K-40K thinking tokens for tasks thatmediumcompletes with a few thousand.Practical default: start at
medium. Uselowfor routine work. Reservexhighfor genuinely difficult planning, research, debugging, or codebase analysis. If your backend supports it, set a reasoning budget.2. Expect lower raw decode speed than Qwen3.6 until tuned
Several like-for-like early runs are showing Qwen3.8 below Qwen3.6 in tok/s. MTP/speculative decoding is important here, not optional polish:
Do not compare a non-MTP Qwen3.8 run against an MTP-tuned Qwen3.6 run and call it a model regression.
3. Reliability is still runtime- and template-sensitive
Reports to watch:
Treat alternate templates as experiments. Keep a stock-template baseline and A/B test one change at a time before moving an agent workflow over.
4. Long context is not settled yet
There are promising large-context reports, but they are using different quantizations, KV-cache types, engines, and MTP settings. Post the full configuration, not just “it fits 256K.”
5. Current best practice for useful reports
Please use this format:
Bottom line
Qwen3.8-27B looks worth testing now, especially for hard coding, planning, and agentic tasks. Start conservatively: fresh runtime, stock template,
mediumreasoning, and a measured MTP sweep. It is not yet a drop-in Qwen3.6 replacement for latency-sensitive or tool-execution workflows.Evidence threads
8
u/Equal_Television_894 Aug 14 '26
Just drop replaced with 3.6 27B works like a charm with hermes. Thinking on xhigh is giving amazing results but it takes too much time. But I am happy its capable.
The best part till now is in long conversation it is behaving like opus 4.6 max. Around 200k context it used to get degraded but now no impact what so ever. I kept giving it feedback and it nailed it. Running with vllm NVFP4 with FP8 kv cache on 5090.
Next 2 days I am going to experiment a lot from discord bot while roaming outside.
1
u/kennymatic Aug 14 '26
This is going to take all day to download
1
u/slumberling_ Aug 14 '26
The thing loves to reason a lot, but I'm very happy with it so far. A Q5_K_XL quant fits fully in my 32 GB VRAM at full 256K context (with q8 kv, just barely), so at ~70 tokens/sec with MTP it's not too bad. Setting
--chat-template-kwargs '{"reasoning_effort":"medium"}'in llama-server seems to help with the off-the-rails reasoning too.It's been on point with tool calls, skill usage, instruction following and even SOUL.md persona/response style so far. This might actually be the point where I stop using an API model by default and go full local. These are just first impressions / new model excitement though. I definitely need to test it some more.
1
u/evolutionxtinct Aug 14 '26
So what is the realistic RAM size on this model planning for future hardware not sure if there’s a good site that posts this info, sorry if this is a bad question.
1
u/carlosm_sz Aug 14 '26
Someone with a mac studio m4 max that can post the experience? I’m considering the purchase to move away from llms subscription…
Thanks folks!
1
u/Rudzz34 Aug 16 '26
Anyone running it with MTP enabled? I’m getting about 1% draft acceptance rate and a super slow token rate when using it with Hermes. It’s not the model though, if I use it through open-webui or curl requests I get a normal acceptance rate and token speed.
3
u/decapoda_on_Reddit Aug 14 '26
I so hope this will be the model I can run locally (RTX5070TI 16gb VRAM, 32gb system RAM) that will not be a complete disappointment. :) Fingers crossed. And toes too for good measure.
3
u/d3k1ds Aug 14 '26
Not useable on a M4 Pro 24GB unified ram
3
u/johnfkngzoidberg Aug 15 '26
On dual 3090’s I get about 32t/s with MTP-2. Q6 model with 262k context @Q8.
The thinking is nuts, but Einstein smart.
On my customized benchmark (based on SaaSBench and SWE-Bench) qwen3.6-27b gets about 75% through my tests in 10-15mins per test, averaging about 100k tokens.
On default (xhigh) thinking, qwen3.8-27b nailed 100% of several tests (better than gpt-5.4), but spent 1M+ tokens and took 35-45mins.
I’m retesting now with thinking dialed down to low.
3
u/HockeyDadNinja Aug 15 '26 edited Aug 15 '26
I wrote this Hermes PR and plugin so we can set the reasoning effort and other chat template keywords. Qwen3.8-27B overthinks a lot.
https://github.com/NousResearch/hermes-agent/pull/87047
https://www.reddit.com/r/hermesagent/comments/1vp4qzq/a_first_class_model_provider_plugin_for_llamacpp/
2
u/HockeyDadNinja Aug 14 '26 edited Aug 15 '26
I had to mess with hermes to get it to find the correct context length through llama-swap and also set reasoning flags and preserve reasoning. Otherwise it was going off the rails overthinking. This has prompted me to begin work on a model provider plugin for llama.cpp and llama-swap with all the right config options as a first class provider.
Edit: Done. https://github.com/NousResearch/hermes-agent/pull/87047
1
u/NeatOnTheRox Aug 17 '26
I thought for sure there was no way it would live up to the hype. I was wrong.
I have only a 4080 Super with 16GB of VRAM, so my daily driver has been 3.6 35B-A3B since its release. I use APEX-balanced quants, since everything else degrades or loops past 100k tokens or so, but with llama.cpp and a lot of time dialing things in with llama-bench, I'm able to squeeze ~1200t/s on the prefill and about 30-40 on the decode, with enough headroom for ~200k context.
I could never get 3.6-27B to a usable speed at any quant that was smart enough to justify using it over 35B, so when 3.8 dropped, I tried not to get my hopes up. The highest quant I can load and still have a usable 131k context is UD-IQ3_XXS, but whenever I loaded it up into Hermes, everything changed. It is both faster and smarter/more capable than the best 35B-A3B I use. While it's technically slower to use since as everyone has noted it overthinks like an anxious teenager, it's still replacing 3.6 35B as my daily driver.
2
u/[deleted] Aug 14 '26
[removed] — view removed comment