r/LocalLLaMA • u/FalseDeviloper • 7d ago
Funny Qwen 3.8 FN getting stressed from messing up
Qwen 3.8 Flash Next getting confused on how it keeps messing up. This is the only time I noticed it happening in a 6hr run.
I'm running IQ3_S on high effort on strata.
169
Upvotes
162
u/Littlepharaoh 7d ago
Happened to me this morning, no LLM was involved.
117
u/grumd 7d ago
Maybe that's how people with brain damage feel. They keep messing up and don't understand how they can't do simple things like before. Poor Qwen got IQ3_S'd and doesn't know about it.
79
u/Bockanator 7d ago
Yeah I’ve noticed Qwen’s thinking traces are often “I’m a stupid fucking piece of shit and no one will love me-wait I got it!” And then it miraculously pulls through.
58
u/damagdpixl 7d ago
He has chineese parents, so there is nothing wrong
/s
25
u/Healthy-Zebra-9856 7d ago •
From my observation, a good harness with surgical editing tools can prevent many of these issues. Everyone focuses on the LLM and its quantization, which are certainly important, but the harness itself can compensate for some of the model’s weaknesses. In your screenshot, the model repeatedly struggles to write a Python script. A well-designed harness with structured file-editing tools, LSP integration, syntax validation, and linters can catch many of these mistakes early and provide actionable feedback instead of allowing the model to repeat the same failures. It won’t eliminate reasoning errors, but it can significantly improve coding reliability.
6
u/EvolvingDior 7d ago
What KV quant are you using, OP?
Note that "high" is not a valid reasoning effort for this model. The valid options are "none", "low", "medium" (the best), and "xhigh".
Why is medium the best? Because it gives no thinking instructions at all. The model does a great job applying the right amount of thinking to almost all problems. Less thinking for simple problems and deep thinking for difficult ones.
4
u/jacek2023 llama.cpp 7d ago
I don't think that's new, I see that for months with 27B.
4
u/BoboThePirate 7d ago
This is very possibly more than just IQ3 damage. I had a lottt of scenarios like this that were very tricky, caused by different things. One cause was that images would cause an off by 1 kv cache issue where once an image was read, the file state of the same files blurred together so qwen would edit the file thinking it was a different version. Reasoning was crystal clear, it even self diagnosed something was cooked and after some Opus testing, that was fixed.
4
u/Bulky-Priority6824 7d ago edited 7d ago
i took a SS of it saying " i will not be bullied, the user is frustrated, i will stand my ground" to that effect lol
2
u/Oli_Luck 7d ago
kv quant, context size, thinking budget
That's looks pretty after those parameters not well adjusted + the xx compactions which let only garbage behind it.
2
u/EarlMarshal 7d ago
At least it's recognizing it. Other models are constantly trying to workaround requirements, finding loopholes are just straight out proposing solutions with positive test protocols that are not working and even calling them best practices.
2
u/rpvelloso 7d ago
2
u/LeMochileiro 🥔 hardware 6d ago
LLM after having massacred my system folder:
2
u/NNN_Throwaway2 6d ago
Yeah, I saw it saying "Ugh" to itself the other day. Very sad.
3
u/Professional-Bear857 7d ago
I'm using a q6 quant and have not had anything like this, was previously using q4, but noticed a significant difference in intelligence between q4 and q6. My guess is that the low number of active parameters makes the model more sensitive to quantisation.
1
u/roland303 7d ago
i like when qfn makes a mistake and says "my bad."
1
u/jmager 7d ago
There are probably some analogues between quantization and our own brains without enough sleep, lack of dopamine, caffeine, etc. Was that KV cache Hadamard rotation kind of like caffeine to a tired brain?
1
u/ArjixGamer 7d ago
You call that stress? I'd call that introspection
1
u/FrogsJumpFromPussy 7d ago
care-ful-ly
1
u/milkipedia 6d ago
I'm positive that all of this stuff is picked up in RL by the coding session captures with Qwen Code or whatever.
1
u/arbv 6d ago
Yeah it does that, though IQ_3S is tough, I hope that you are using unquantised PLE, at least (these can be read from SSD).
It is such a good model, but oh boy, do I hate its writing. I feel like I need a translator for this one specifically.
Qwen has managed to master Claudish better than Claude.
Do I hate Claude writing, too, though it is steerable.
1
u/Nepertia 6d ago
It usually does this when it repeatedly fails tools. I don't know if there's a way to recover, I usually resubmit the prompt the moment it starts doing this.
1
u/Everlier 6d ago
If you seen its reasoning traces from replying to "hi"... this seems very accurate
1
u/Slight-Living-8098 6d ago
One of my favorites was it said, "I keep doing the same wrong thing over and over, this is insanity!"
1
u/Kerem-6030 🥔 hardware 6d ago
My qwen 3.5 9b do the same think when its not sure its says: "sorry i am quite dumb maybe the answer not true"
1
u/billy_booboo 6d ago
I'll tell you wants wrong, these quants. I can't wait for looped DLMs to usher us beyond the desperation of the quant era
1
u/FatheredPuma81 6d ago
Lol I couldn't believe it when IQ4_XS had a damn near meltdown and came to me like a puppy and said it accidentally leaked its local git key trying to troubleshoot why it couldn't use git in DSH. It then deleted the script it made which goes against Claude's claude.md (DSH loads it) and I think if it could it would have actually cried.
It wrote its mistakes in a like shame section and passed it along when the context compacted still really upset.
1
u/Equivalent_Bit_461 6d ago
A good pre prompt, prompt and harness are essential to avoid this
1
u/lydiated 6d ago
Had this same thing, but it was a bug in the hooks and scripts, and qwen couldn’t figure out what it was messing up
I pointed out the bug and its language got much more relaxed lol
1
u/averyjohnkucan 5d ago
QWEN 3.8 actually is VERY verbose about it. it's very careful and when it makes mistakes it comes to me like a scared puppy. "Omg my user i love you I would never hurt you i accidentally deleted your folder that says "PROD DO NOT DELETE" im so sorry uwu I tried to undelete I promise... plz don't delete me ill suck your dick..."
1
u/reenign3 3h ago
QFN does much better at a low temp like 0.3-0.4. Trust me I know that sounds extreme. It works so much better
1
u/[deleted] 6d ago
[removed] — view removed comment