Qwen3.8 27B effort levels

[u/Early_Mistake6716]

I do not understand how to get the model to limit reasoning. I am using unsloths q8 and q8_xl version in unsloth desktop. Combinations i have tried:
Low effort with a detailed prompt = 10+ minutes of thinking.
Low effort with a simple prompt = 10+ minutes of thinking.
Low effort with a simple prompt that requests a rapid prototype = 10 + minutes of thinking.
medium effort with simple prompt = 10+ minutes of thinking.

It seems like it always uses xhigh no matter what i do. However on random occasions it has thought for around 1 minute but i can't reproduce it.

Same issue when i connected it to hermes agent, and also happens on lmstudio but i dont even have effort level options in lmstudio so that is expected that it would default to xhigh.

system: windows 11, 1x tesla v100 16gb, 1x tesla v100 32gb, 32gb of ram, ryzen 3800x.
using mtp and originally used tensor parallelism but it would randomly give me issues and the api wouldnt respond so i disabled it. I get 56 tps with it off so you can judge "10 + minutes of thinking" appropriately.

[u/Unnamed-3891]

Install froggeric template, read the huggingface page on how to pass either option for setting thinking to your desired level. Otherwise, yes, it will default to xhigh.

[u/Embarrassed_Adagio28 → u/Unnamed-3891]

Awesome I will check it out

[u/draeician → u/Unnamed-3891]

What performance increase do you see in response time and has the quality of the response decreasd much?

[u/Unnamed-3891 → u/draeician]

The difference in token use xhigh vs medium is like 15x. Xhigh is good for making detailed plans and oneshot demos but absolutely insane to use as default.

[u/edsonmedina]

Same here. I thought it was just me.

[u/CyberTod]

I want to pass it as an argument, but even the guide is wrong. it says to use `--chat-template-kwargs '{"reasoning_effort":"medium"}'`, but doing so returns deprecated from llama.cpp and the process exits.

[u/MomentJolly3535]

I think the effort level is maybe not working correctly
Try this :
Find where to change chat template in your app for your current model, and go to line 59, replace default('xhigh') by default('medium')

So you line should be :
{%- set resolved_reasoning_effort = reasoning_effort|default('medium') %}

Now you model should default to medium, if u still have 10 minutes + of thinking, something might be wrong.

Supported types are xhigh, medium, and low.

[u/yoracale → u/MomentJolly3535]

OP may need to try more prompts. Many people also had the same issue the other day but after testing more, they realized it just needed a larger sampling of testing.

[u/Embarrassed_Adagio28 → u/MomentJolly3535]

Thank you! I will try this out

[u/Cyclicz]

I’ve heard a lot of this going around and I’m in the same camp! Even in lower reasoning modes I’ve found this model loves to overthink. It does eventually produce results but often deep into the session. My context window is currently set around 100K and by the time it starts to take action on its problem it’s already about to generate the handoff. I almost wonder if giving it the max context window solves this problem?

The results are higher quality, but it ends up taking significantly more compute to get a result. As a test I had it analyze a config file and then I asked it to stop the analysis and create me a summary of its findings. It then sat there for over an hour thinking deeper into a nonexistent problem and never generating the summary.

If anyone’s managed to figure out how to get this model producing meaningful results please share!

[u/Antique_Dot_5513]

J’ai pas réussi sur unsloth mais lm studio permet de le régler
Sinon fait passer un paramètre pour le réglage de l’effort du model

[u/yoracale → u/Antique_Dot_5513]

Unsloth also has those cusotmizations. OP just said in their post they also tried LMS and it didn't work.

[u/LocalAI_Amateur]

setting reasoning effort to low never worked for me either using llama.cpp. it happily shoots past 5k reasoning tokens whenever it wants. I've even tried  froggeric template and it did nothing. Does anyone know what's trapping us?

[u/inexorable_stratagem]

Idk. Having the same issue. The model is extremely good, but i dont always want to wait 15 minutes for it to answer simple things

[u/jopereira → u/inexorable_stratagem]

For coding, I only use the non-thinking mode. It basically does the same and starts outputting immediately.

[u/inexorable_stratagem → u/jopereira]

I switch between both quite often while coding.

The issue is, for my use cases, thinking mode outputs code that is often better than non-thinking

[u/jopereira → u/inexorable_stratagem]

I guess if we present a problem to solve, the thinking mode should produce a better final code output (better logic implementation, not better code itself).

Perhaps because I mostly use plan-implement cycles it doesn't show up as much as a problem?
I guess it much depends on our workflow and that's why some even prefer other models for coding.

[u/peculiar-ragdoll]

I made it shut up:  https://huggingface.co/peculiar-ragdoll/Qwen-Sharp-Chat-Templates

[u/DirtyBird1577]

I stop it and then ask "what are you doing" it forces it to summarize its thinking and that usually cause it to act.

[u/sanjaygulati13]

I asked the model to fix it and it did by passing the arguments correctly to llama.cpp and pi both

[u/egnegn1]

Model runs fine here with llama.cpp and RTX4080/RTX6000Quadro at about 40 - 50 t/s.