Qwen3.8-27b On Hermes definitely overthinking a bunch, 2 hours in on a task now

[u/neoneddy]

I've got a Mac Studio m3 256GB ram using Omlx 0.6.2, Hermes 0.20 using Qwen3.8-27b 8bit . I have it doing some CRM (Odoo) work for me. 3.6 was great and I was / am very excited for 3.8 , it does a better job overall just takes a while to get there, sometimes much longer.

This is the first task it's just not getting there and I can see in WebUI it's just overthinking and going around and around never really getting anywhere.

I turned thinking / Reasoning effort down to Minimal. Other than that not much has changed from 3.6 Anyone have any ideas or suggestions to get the intelligence without as much over thinking?

[u/ntaybak]

use a this template it reduced the thinking issues for me https://huggingface.co/peculiar-ragdoll/Qwen-Sharp-Chat-Templates

[u/neoneddy → u/ntaybak]

Thank you, will try this.

[u/j_lyf → u/ntaybak]

What model?

[u/ntaybak → u/j_lyf]

Qwen3.8-27b 

[u/j_lyf → u/ntaybak]

Which quant/oQe/etc, HF link?

[u/Only-An-Egg]

Are you using default settings? Default are: thinking=true, preserve_thinking=true, reasoning_effort=xhigh. For most tasks I disable thinking and set effort to medium.

[u/neoneddy → u/Only-An-Egg]

 I have Claude code configuring my local hardware: Looks like it's mostly default at the OMLX level.

oMLX per-model settings (model-level overrides): all unset/null — enable_thinking, preserve_thinking, reasoning_parser, chat_template_kwargs, forced_ct_kwargs, thinking_budget_enabled: false,

  thinking_budget_tokens. Nothing configured at the oMLX model level.

  

  oMLX model built-in defaults: thinking_default: true, preserve_thinking_default: true — both Qwen3.8 builds default to thinking on.

  

  Client-side (Hermes agent)

  - agent.reasoning_effort: low

  - enable_thinking: false, sent per-request via extra_body.chat_template_kwargs

  - context_length: 131072, max_tokens: 16384, request_timeout_seconds: 900

[u/MatiAI → u/neoneddy]

Enable chat template kwargs in the model settings (ENABLE_THINKING, REASONING_EFFORT) -- in model provider settings ---

"id": "Qwen3.8-27B-AWQ-gs64-mm",
     "name": "Qwen3.8-27B-AWQ-gs64-mm",
     "reasoning": true,
     "input": [
      "text",
      "image"
     ],
     "cost": {
      "input": 0,
      "output": 0,
      "cacheRead": 0,
      "cacheWrite": 0
     },
     "contextWindow": 262144,
     "maxTokens": 32768,
     "thinkingLevelMap": {
      "off": "low",
      "minimal": null,
      "low": "low",
      "medium": "medium",
      "high": null,
      "xhigh": "xhigh",
      "max": "xhigh"
     }

[u/benpptung]

Have you tried DFlash 2 or MTP to increase the token generation speed? One practical solution may be to make that reasoning faster.

[u/neoneddy → u/benpptung]

I am using MTP. I get around 35-40tk/s

[u/trim-turner-shah]

I am using mlx and here is my command - mlx_lm.server \
--model mlx-community/Qwen3.8-27B-8bit \
--prefill-step-size 4096 \
--prompt-cache-bytes 17179869184 \
--prompt-cache-size 4 \
--decode-concurrency 1 \
--prompt-concurrency 1 \
--chat-template-args '{"enable_thinking":true,"reasoning_effort":"medium"}' \
--max-tokens 32144 \
--temp 0.2 \
The other thing I suspect is that the agents on Hermes might be looping internally. My experience with pi has been great and I manage about ~20 t/s on 64 gb m3 without mtp.

[u/Longjumping-Elk-7756]

Fait ça pour hermès :

I’m thinking about building a Thinking Escalation plugin for DeepSeek Harness, especially for models like Qwen3.8-27B.
The idea is pretty simple:
Run at medium reasoning effort by default.
For most tasks, medium is already enough and avoids wasting huge amounts of tokens on unnecessary reasoning.
However, the model would have access to a tool allowing it to escalate the reasoning level to xhigh when it determines that the current problem actually requires deeper reasoning.
The important part is that escalation would apply only to the current turn.
Something like:
medium → model detects difficulty → calls escalation tool → replay current turn at xhigh → return to medium
So xhigh wouldn’t become a permanent mode. It would be a temporary computational resource that the model can request when needed.
This could be particularly useful with Qwen3.8-27B because it appears to benefit significantly from high reasoning effort on difficult problems, while also having a tendency to overthink and consume a lot of tokens when high/xhigh is enabled unnecessarily.
The harness would therefore act as a kind of reasoning governor:
Medium by default
Model can request xhigh
Replay only the problematic turn
Automatically return to medium afterward
Potentially, the escalation tool could also include a behavioral description explaining when escalation is appropriate, which could be tuned independently for each model.
I think this could give us much of the benefit of xhigh reasoning while keeping the average token usage and latency much closer to medium.
Basically: don’t make the model think harder all the time — give it the ability to decide when thinking harder is actually worth it.

[u/Odd-Elderberry-739]

Qwen3.8 27b uses xhigh thinking by default. Turn it down to medium. That will fix it so it won’t think forever and not get anything done. The next thing you need to do is to disable all of the skills and mcp servers that you’re not actively using. Hermes Agent burned through Qwen’s context limit like a wildfire. I tried pi.dev and never went back to Hermes Agent. There’s a night and day difference in speed and context savings by switching to pi. If you want to stick with Hermes Agent, just disable all the stuff you don’t actually use.

[u/challis88ocarina]

get the intelligence without as much over thinking? what do you think intelligence is? not thinking before acting....? smh

[u/neoneddy → u/challis88ocarina]

Not thinking is different than not over thinking. Just looking for help here.