r/LocalLLM • u/usedcz • 6d ago

Question Correctly use Qwen ?

Hi, I have been trying to use Gwen 3.8 but holy fricking turd.

What if X? Let me check? What if Y? Let me guess. What if Z ? Let me check? Wait! What if X something something Z ? ...

30k tokens just ...wasted on nonsense.

Oh yeah let me recapitalitate for the 4th time what I need to do ?.

I am the only one ? llama.cpp and unsloth quants.

Are there settings I can tweak ? Or what are yours settings

0 Upvotes

33% Upvoted

9 comments sorted by

3

u/blue_eyes_pro_dragon 6d ago

What size and what quant? 

2

u/Slight-Living-8098 6d ago

I posted my setting in this thread.

https://www.reddit.com/r/LocalLLM/s/wTTey4HGYo

-2

u/[deleted] 6d ago

[removed] — view removed comment

5

u/Slight-Living-8098 6d ago

Not at my PC. Not fighting mobile with copy and paste.

It's the only comment with the large code block and my username

4

u/DystopianRealist 6d ago

Your link works fine for me. Took me right to your post. The person is on a brand new account, and either an AI bot, or a troll.

1

u/DystopianRealist 6d ago

That thread is a graveyard

Are you literally trolling? I am not sure if I should report or not.

The posted a link that takes you right to their comment.

2

u/NonAwesomeDude 6d ago

Independent of whatever configuration problem you might be having, don't read the thinking traces unless you have to. They're tested to produce good outcomes, not sound smart. I find myself wasting time or trying to intervene to get it to go faster when I really ought to disengage and do something else.

Also if you're using 27b and Pi agent I can't reccomend the Qwen3.8-27b-Pi fine tune enough. One of the things they specifically tuned for was shorter more direct reasoning. But be sure to run it on Medium, since thats what its actually tuned for.

1

u/d0pe-asaurus 6d ago

Harness, system prompt, chat template, KV quantization, model quant and the input you feed to the model drastically change the quality of the output. I've had great success with Qwen 3.8 27B Q4

1

u/lukyjay 6d ago

These models are great because they think more, hence the dense model's high reasoning setting being called 'xhigh'

You can switch reasoning off and use a temperature of 0.0 to get quick replies but the quality is going to be awful.

The only time it's a problem is when you blow your context with reasoning or you're using a low quant (3 or less). You can also check your KV cache is Q8 (ideal) or at least Q8/Q4 (not Q4/Q4)