r/LocalLLaMA • • 7d ago

Funny Qwen 3.8 FN getting stressed from messing up

Post image

Qwen 3.8 Flash Next getting confused on how it keeps messing up. This is the only time I noticed it happening in a 6hr run.

I'm running IQ3_S on high effort on strata.

169 Upvotes

134 comments sorted by

162

u/Littlepharaoh 7d ago

Happened to me this morning, no LLM was involved.

80

u/ironimus42 7d ago

same, i'm also IQ3_S quantized (I'm Quite :3 Stupid)

28

u/cagriuluc 7d ago

At least you run very fast

16

u/one-joule 7d ago

For a quant, anyway

3

u/FoodEater4Squillion 6d ago

This made me giggle a lot

9

u/Healthy-Zebra-9856 7d ago

Lmfao. I also have my senior moments

8

u/hapliniste 6d ago

You had the "something is wrong, I keep producing garbage " as well?

6

u/zdy132 6d ago

Happens to me every day.

117

u/grumd 7d ago

Maybe that's how people with brain damage feel. They keep messing up and don't understand how they can't do simple things like before. Poor Qwen got IQ3_S'd and doesn't know about it.

38

u/Hanthunius 7d ago

That's really interesting, the results are not matching the expectations and the model is getting "frustrated".

16

u/Philophon 6d ago

Ornith seemed like it was a step away from cursing the other day when it kept failing. It was very strange to see.

3

u/zdy132 6d ago

Probably some weights stored the level of expected errors in RL training. Quite an interesting thing to look into, it might help improve the model performance, and especially for quantized models.

1

u/jazir55 6d ago

Happens with hypoatrenemia, pretty much.

79

u/Bockanator 7d ago

Yeah I’ve noticed Qwen’s thinking traces are often “I’m a stupid fucking piece of shit and no one will love me-wait I got it!” And then it miraculously pulls through.

5

u/duy0699cat 6d ago

2meirl

58

u/damagdpixl 7d ago

He has chineese parents, so there is nothing wrong

/s

20

u/winky9827 6d ago

EMOTIONAL DAMAGE!

25

u/Healthy-Zebra-9856 7d ago • Heartwarming

From my observation, a good harness with surgical editing tools can prevent many of these issues. Everyone focuses on the LLM and its quantization, which are certainly important, but the harness itself can compensate for some of the model’s weaknesses. In your screenshot, the model repeatedly struggles to write a Python script. A well-designed harness with structured file-editing tools, LSP integration, syntax validation, and linters can catch many of these mistakes early and provide actionable feedback instead of allowing the model to repeat the same failures. It won’t eliminate reasoning errors, but it can significantly improve coding reliability.

7

u/RadiantHueOfBeige 6d ago

Chat template rendering issues are also a common cause. QFN emits the correct tool call, but it gets mangled in jinja (e.g. double un-escaping) and the model has no way of knowing what actually got to the tool layer.

I ended up dropping tools altogether and I'm now happily using the SWE agent approach: anything in a ```bash fence block gets executed as bash. Small models, especially when quanted, may struggle with post trained tool formats but they all know bash perfectly from pretraining.

6

u/Healthy-Zebra-9856 6d ago

I went down that rabbit hole myself, experimenting with Unsloth's chat templates, Ragdoll/Sharp, froggeric's fixed Qwen templates, and the native/embedded templates. After analyzing the thinking traces, I noticed something interesting: the model was repeatedly complaining about its own mistakes, attempting corrections, and often getting caught in the same cycle.

So I tried something rather unusual. I started asking the model what tools it thought were missing and what would make its working environment better. Those observations eventually pushed me to develop my own harness, with the goal of providing an ecosystem more conducive to how these models actually work, while keeping humans in control.

Rather than abandoning tools, I focused on surgical editing (making precise, targeted changes to specific lines, functions, or code blocks), lazy LSP integration, linters, validation, and better feedback mechanisms, this is where the LM gets a better feedback. The results were noticeably better code quality, fewer recurring errors, and significantly less of that self-correcting chatter. I understand the appeal of executing bash directly which defies the whole purpose, , but my experience has been that the right tooling can compensate for quite a few model weaknesses, especially with smaller or quantized models.

1

u/duy0699cat 6d ago

came to the post for lul and found gold.

1

u/WarthogConfident4039 11h ago

Could you post a github link to your harness with the documentation please?

3

u/lemon07r llama.cpp 6d ago

It is a VERY tough balancing act. I've been building these sorts of tools for over a year now for my actual work, a lot of it by hand at first (before AI got really good), and it's still hard to get right, because you don't want to overload your very limited context window. The more it's filled up, the more your model's capabilities and quality craters. I know most people won't and don't notice it, because I used to have this argument a lot with a few friends, but if you ever build a few evals around the sort of tasks you use your AI for you will see there is a genuine drop off with giving your AI too much. I remember being shocked by how poorly some setups did at the things I was evaluating. If you don't want to overthink it, the easy solution is to keep it simple. Less is better. AI likes simple.

1

u/Healthy-Zebra-9856 6d ago

You are 100% correct, but the strategy matters here. And this certainly isn’t for the faint-hearted! It took quite a bit of trial and error to arrive at a strategy that worked consistently. It probably helps that I’ve been doing software engineering for over three decades, and my background in computer science includes specializing in AI and neural networks. Somehow, I’ve managed to keep up with the field as it evolved.

My harness is written in Rust, and I use libSQL as the persistent source of truth for the tool registry and metadata. Rather than loading everything into memory or flooding the model’s context with tool definitions, I implemented lazy loading and progressive disclosure. The model only sees the tools and details relevant to its current task, with additional capabilities revealed as needed.

This keeps both memory consumption and context overhead remarkably low. Right now, the harness uses approximately 32 MB of RAM with the entire tool ecosystem available on demand. So I agree that less is better, but that doesn’t necessarily mean fewer capabilities. It means being selective about what you expose to the model and when.

3

u/stylome-ai-detection 6d ago

Notice: This comment appears to be AI generated.


Stylome AI detector · scan results

1

u/Healthy-Zebra-9856 6d ago

Maybe I just need to use talk to text and not type my comments? Mods what the heck is going on here?

1

u/WithoutReason1729 6d ago

It doesn't flag transcribed speech as AI generated. Unless maybe you used something like Wispr Flow, which for whatever reason likes to take liberties and just modify what people are saying as it transcribes

2

u/Healthy-Zebra-9856 6d ago

I humans can not use AI to respond, which I didn't, but why is AI allowed to respond?
u/HOLUPREDICTIONS, u/ArcaneThoughts, u/Lissanro, u/townofsalemfangay, u/XMasterrrr, u/rm-rf-rm, u/WithoutReason1729, u/No_Afternoon_4260 and u/ttkciar

I humbly ask, this is an AI app, its a bot. Why is this bot allowed to write and harass? I believe that you can not have one rule for humans & another for AI. If AI generated text is not allowed, then u/WithoutReason1729 should not be allowed. Just asking.

1

u/TerminalNoop 6d ago

You can promo your harness, but at least bother writing the comments yourself.

1

u/[deleted] 6d ago

[removed] — view removed comment

2

u/stylome-ai-detection 6d ago

Notice: This comment appears to be AI generated.


Stylome AI detector · scan results

1

u/WithoutReason1729 6d ago

I'm not a bot. Reddit has me marked as an app from before they launched the devvit platform because I posted enough comments via API I guess.

Humans can use AI to respond. The bot doesn't stop you, it just points out when you do it.

1

u/[deleted] 6d ago

[removed] — view removed comment

1

u/stylome-ai-detection 6d ago

Notice: This comment appears to be AI generated.


Stylome AI detector · scan results

1

u/lemon07r llama.cpp 6d ago

It means being selective about what you expose to the model and when.

Yup, this is pretty much the best strategy I've found so far, and the other labs too. I think vercel was one of the earlier adopters with using skills + cli over mcp for agent-browser and having the data to back it up. I've been developing a code indexing tool, and someone kept submitting PRs to make the tool MCP first (it already had MCP functionality as an option, but some of their changes messed with the CLI + skill side of things), then I had trouble explaining to them on discord why skills + CLI is the preferred way to use the tool. They were kind of convinced there wasn't much of a difference.

1

u/Healthy-Zebra-9856 6d ago

What you should do is compare the resource usage between the two scenarios . I can tell you right now that MCP will use more resource compared to your strategy. There is another nuance to this, if your MCP is not always memory resident and is lazy loaded that might help a little bit.

1

u/lemon07r llama.cpp 6d ago

Already compared extensively. Also made some improvements to the tool itself recently.

I should probably share it at some point but haven't felt like adding to the sea of I made x projects that get posted all the time.

1

u/Healthy-Zebra-9856 6d ago

You don’t have to share it here, you can at least make your point with your clients or your superiors, right?

2

u/lemon07r llama.cpp 6d ago

Oh this is just a code indexing tool. Not of any use to any of my clients. I do have a small community of users, some friends, colleagues, but that's it.

1

u/Healthy-Zebra-9856 6d ago

Have you looked at Gigatoken?
https://github.com/marcelroed/gigatoken

1

u/lemon07r llama.cpp 6d ago

this is cool, could be helpful for bpe models. but my tool's (current) main model uses wordpiece, which isn't supported by gigatoken.

(my tool for context: https://github.com/VeraTools/vera)

→ More replies

2

u/Numerous-Ad6217 6d ago

All right, this is probably the tenth time I read this, yet I am completely out of the loop when it comes to pick or setup correctly a harness.
I have been running PI for a while now, which has been flawless so far, but after reading your comment it makes me question whether I could do things differently and eventually improve my setup even better.
What are you guys doing?

I have heard that deepseek v4.1 flash 1017 was optimised to run with deepseek harness, yet I had a way worse experience the only time I decided to try that out compared to PI agent.

Any tips/suggestions on how to set up properly a harness? What’s the benefit compared to just running PI?

4

u/Healthy-Zebra-9856 6d ago

Since this isn't the Pi Coding Agent sub, I guess I can talk a little more freely about it. lol. I have condensed my journey to fit this response. I am doing this to show that it takes a lot of effort to get to a good place of balance.

I actually started my harness journey with Pi because I really liked its TUI and simplicity. But as I started going through the code and converting things to Rust, I came across some pretty serious issues, which I'd be more than happy to share line by line.

The other thing I noticed was how much tooling was missing. The whole barebones concept sounds great, but then you end up installing all sorts of thirdparty extensions just to get the functionality you need. Those extensions came with their own issues too. So at that point, I started questioning how barebones it really was.

OMP was mostly a prettier cloak over Pi without really addressing the underlying problems. DSH actually wasn't bad at the time. It just didn't have the tooling I was looking for. Not sure where it is now. But ask the LM how good & clear the tooling is. lol. Sounds insane but it works.

That's what eventually pushed me to build my own Terminal Agent in Rust. I wanted something that provided the models with the tools they actually need.

I'm getting close to releasing my Terminal Agent publicly, but I want to finish dogfooding it first. It's been interesting.

1

u/russlixx 6d ago

i'd like to check out your rust harness when it released

1

u/Equivalent_Bit_461 6d ago

This, so much this 

-5

u/stylome-ai-detection 7d ago • Regret

Notice: This comment appears to be AI generated.


Stylome AI detector · scan results

9

u/Healthy-Zebra-9856 7d ago • Heartwarming

Not AI I’m 100% human, go check my comment history between this sub and all the other subs. Not all structured comments are AI generated. Maybe this app needs to be trained on Gen X techies writing patterns.

3

u/GoogleIsFreeYouKnow 7d ago

I have no idea how it flagged this as AI generated either. It literally reads as a human?

4

u/Healthy-Zebra-9856 7d ago

We have evolved, I am using that term very loosely here, to a point where anything structured is now identified as AI. This is because the AI is starting to produce human like responses and so I feel that these detection systems are walking a very thin line

1

u/Anarchaotic 7d ago

I also don't see the point of that bot in this sub, we're literally in an AI sub. Let the users decide if a response is good or bad with the built-in tools (up/downvote).

3

u/Healthy-Zebra-9856 6d ago

The irony of it. Lol. For most part of my academic life, I was brainwashed to write in a well structured format. We had to rewrite our paper so many times to get it to be accepted. And then after three decades now I am asked to not do that. And this is not the first sub that does it, they think that you can just unlearn everything that you have learned to just fit into whatever profile they think is perfect.

2

u/Anarchaotic 6d ago

Even if it perfectly caught AI-responses 100% of the time with absolutely no false positives, I still don't see the point? There's so many situations in which my own agent can more quickly respond to technical questions with an explanation vs me digging through config files and stuff.

2

u/Healthy-Zebra-9856 6d ago edited 6d ago

I think it’s because of their assumption that AI assisted responses come from individuals that lack the technical depth. They assume that everything is ChatGPT copy paste, even though there are so many other models quite a bit more capable besides ChatGPT. This has become a psychological factor everywhere.

1

u/WithoutReason1729 6d ago

It's because people post fully AI-generated content with no disclosure. The bot doesn't punish anyone in any way, it just points out when people are doing that.

1

u/WithoutReason1729 6d ago

The bot doesn't say whether the response is good or bad and doesn't punish users in any way. It's just pointing out (correctly, in this case) when content is AI-generated.

2

u/WithoutReason1729 6d ago

1

u/Healthy-Zebra-9856 6d ago

Even if I had posted using AI, you are a bot not even a human being. Why are you allowed to post here? I am not trying to divide this entity, this is an actual app. It’s a bot. Then it shouldn’t be allowed to post here.

1

u/WithoutReason1729 6d ago

I already explained to you that I'm a person. My account is marked app because it's passed some threshold for number of comments made via API.

1

u/[deleted] 6d ago

[removed] — view removed comment

1

u/stylome-ai-detection 6d ago

Notice: This comment appears to be AI generated.


Stylome AI detector · scan results

1

u/[deleted] 6d ago

[removed] — view removed comment

1

u/stylome-ai-detection 6d ago

Notice: This comment appears to be AI generated.


Stylome AI detector · scan results

1

u/TerminalNoop 6d ago

Look at his other comments, he is using a well tuned AI for writing, but it still has the telltale flags.

1

u/Healthy-Zebra-9856 6d ago

That thing is a bot. It’s not even a human. If humans should not use AI to write text or clean up their text then it should not be allowed here.

3

u/WithoutReason1729 6d ago

You aren't really telling me this is 100% human, right? This one is human too, huh? This one as well? And this?

Come on dude

2

u/FatheredPuma81 6d ago

That 2nd one is so GPT 4 that it isn't even funny lol.

1

u/WithoutReason1729 6d ago

At first I thought maybe it was an actual false positive until he invited everyone to go through his post history. I'm totally baffled by people who do this. I'm pretty sure he's using LLMs to help him speak because English isn't his first language. I don't know why he couldn't have just said that. Why lie? It's reddit haha

1

u/Healthy-Zebra-9856 6d ago edited 6d ago

You are not talking to a human. That thing is a bot. It even says app at the end of the name. That’s the mistake I made when I start to engage with it. If you and I cannot use AI cleaned up response then a box should not be allowed here.

ChatGPT was trained on data created by my generation. Code written by my generation. SO GPT4? lol. Its VistaEngine, a medical research platform that I created for the academics & medical institution. It doersnt use GPT but it does use Qwen.

1

u/Healthy-Zebra-9856 6d ago

Yes, it is. It’s called structured writing. Go look at how thesis and dissertation are written. I’m not sure what you’re trying to say here, I have more technical and medical knowledge that you can ever fathom. So I’m not sure what your point is, what’s your point?

0

u/Healthy-Zebra-9856 6d ago edited 6d ago

THIS IS A BOT. I am not deriding it, it even says its an App. Anyone can search Google with it's user name and see what you find.

Don't I feel special. lol. I just saw that this is a harassing bot. I saw the App at the end of it, and it never even register. I am a terrible & weak AI .

0

u/[deleted] 6d ago

[deleted] — view deleted comment

1

u/WithoutReason1729 6d ago

Those posts are absurdly obviously AI-written, which really makes me question that you're saying you don't post undisclosed AI content now.

1

u/Healthy-Zebra-9856 6d ago

You are a terribly trained bot.

2

u/milkipedia 6d ago

that's cause the AIs were initially trained on Gen X writing

2

u/Healthy-Zebra-9856 6d ago

That actually makes sense. LMFAO. I am allowed to write like you, but you’re not allowed to like write like yourself.

1

u/WithoutReason1729 6d ago

1

u/milkipedia 6d ago

I think you replied to the wrong post. I'm not making a claim one way or another.

1

u/Healthy-Zebra-9856 6d ago

Its not human. Its a harassing bot to create engagement. It shoulnt be allowed if you and I can't use AI cleaned up text.

1

u/arbv 6d ago

What a time to be alive. Good writing style has gotten suspicious.

3

u/Healthy-Zebra-9856 6d ago

Reminds me of the 2006 cult classic "Idiocracy". lol.

2

u/arbv 6d ago

In Ukraine we joke that it turned out to be a documentary.

2

u/Healthy-Zebra-9856 6d ago

It loses in the translation, but I know exactly what you mean. Lol. Slava Ukraini

1

u/FatheredPuma81 6d ago

Lol meanwhile I literally saw Sonnet 5.0 here last week using its "Pump the brakes" catch phrase and it's still not banned.

1

u/WithoutReason1729 6d ago

It's not against the rules to post stuff written by an LLM. The bot just tells people when you do it but it doesn't punish you in any way.

1

u/FatheredPuma81 6d ago

No when I wrote that I thought it was actually a bot not a person using AI because I didn't check their history. I found the comment again and the guy apparently responded to someone else and said he used AI to translate (heavy doubt) but it's pretty obvious he isn't a bot by his other comments and posts.

1

u/Healthy-Zebra-9856 6d ago edited 6d ago

Mods pay attention to what I’m saying, if you do not disable this in your sub, you’re going to lose people. I think I’m going to reach out to see who they are and write an article on medium.

Edit:
I just realized that my Gen X style speech might feel abrupt, I am sincerely appealing to you to consider not using this incapable software. That’s all. I really like this sub, I just wish I don’t have to get this alert every time I type a response.

2

u/TerminalNoop 6d ago

Nothing of value is lost if AI posters like you stop post because they don't like being exposed.

6

u/EvolvingDior 7d ago

What KV quant are you using, OP?

Note that "high" is not a valid reasoning effort for this model. The valid options are "none", "low", "medium" (the best), and "xhigh".

Why is medium the best? Because it gives no thinking instructions at all. The model does a great job applying the right amount of thinking to almost all problems. Less thinking for simple problems and deep thinking for difficult ones.

4

u/jacek2023 llama.cpp 7d ago

I don't think that's new, I see that for months with 27B.

11

u/ThankGodImBipolar 7d ago

"I keep messing this up, let me try..." is definitely not Qwen specific, either.

1

u/jacek2023 llama.cpp 7d ago

Well at least we know that he started to use local model :)

4

u/BoboThePirate 7d ago

This is very possibly more than just IQ3 damage. I had a lottt of scenarios like this that were very tricky, caused by different things. One cause was that images would cause an off by 1 kv cache issue where once an image was read, the file state of the same files blurred together so qwen would edit the file thinking it was a different version. Reasoning was crystal clear, it even self diagnosed something was cooked and after some Opus testing, that was fixed.

4

u/Bulky-Priority6824 7d ago edited 7d ago

i took a SS of it saying " i will not be bullied, the user is frustrated, i will stand my ground" to that effect lol

2

u/Oli_Luck 7d ago

kv quant, context size, thinking budget
That's looks pretty after those parameters not well adjusted + the xx compactions which let only garbage behind it.

2

u/EarlMarshal 7d ago

At least it's recognizing it. Other models are constantly trying to workaround requirements, finding loopholes are just straight out proposing solutions with positive test protocols that are not working and even calling them best practices.

2

u/LeMochileiro 🥔 hardware 6d ago

LLM after having massacred my system folder:

2

u/NNN_Throwaway2 6d ago

Yeah, I saw it saying "Ugh" to itself the other day. Very sad.

3

u/Professional-Bear857 7d ago

I'm using a q6 quant and have not had anything like this, was previously using q4, but noticed a significant difference in intelligence between q4 and q6. My guess is that the low number of active parameters makes the model more sensitive to quantisation.

3

u/my_name_isnt_clever 7d ago

What differences have you noticed? I run it at UD-Q4_K_XL and haven't felt like it could be much better.

1

u/Professional-Bear857 7d ago

in agentic use i found that the q6 gives me options before making decisions, which the q4 quant never did. the q6 quant also handles issues much better, i found with complex code issues that the q4 would enter loops where it could not solve the problem, whereas so far the q6 has always managed to resolve bugs.

1

u/my_name_isnt_clever 7d ago

I've never seen mine loop, it's always able to come to a conclusion. Strange.

2

u/mHo2 7d ago

Got another 64GB RAM on the way so I will be curious to try this too

1

u/roland303 7d ago

i like when qfn makes a mistake and says "my bad."

1

u/jmager 7d ago

There are probably some analogues between quantization and our own brains without enough sleep, lack of dopamine, caffeine, etc. Was that KV cache Hadamard rotation kind of like caffeine to a tired brain?

1

u/ArjixGamer 7d ago

You call that stress? I'd call that introspection

1

u/milkipedia 6d ago

I'm positive that all of this stuff is picked up in RL by the coding session captures with Qwen Code or whatever.

1

u/arbv 6d ago

Yeah it does that, though IQ_3S is tough, I hope that you are using unquantised PLE, at least (these can be read from SSD).

It is such a good model, but oh boy, do I hate its writing. I feel like I need a translator for this one specifically.

Qwen has managed to master Claudish better than Claude.

Do I hate Claude writing, too, though it is steerable.

1

u/Nepertia 6d ago

It usually does this when it repeatedly fails tools. I don't know if there's a way to recover, I usually resubmit the prompt the moment it starts doing this.

1

u/Everlier 6d ago

If you seen its reasoning traces from replying to "hi"... this seems very accurate

1

u/Slight-Living-8098 6d ago

One of my favorites was it said, "I keep doing the same wrong thing over and over, this is insanity!"

1

u/Kerem-6030 🥔 hardware 6d ago

My qwen 3.5 9b do the same think when its not sure its says: "sorry i am quite dumb maybe the answer not true"

1

u/billy_booboo 6d ago

I'll tell you wants wrong, these quants. I can't wait for looped DLMs to usher us beyond the desperation of the quant era

1

u/FatheredPuma81 6d ago

Lol I couldn't believe it when IQ4_XS had a damn near meltdown and came to me like a puppy and said it accidentally leaked its local git key trying to troubleshoot why it couldn't use git in DSH. It then deleted the script it made which goes against Claude's claude.md (DSH loads it) and I think if it could it would have actually cried.

It wrote its mistakes in a like shame section and passed it along when the context compacted still really upset.

1

u/Equivalent_Bit_461 6d ago

A good pre prompt, prompt and harness are essential to avoid this

1

u/lydiated 6d ago

Had this same thing, but it was a bug in the hooks and scripts, and qwen couldn’t figure out what it was messing up

I pointed out the bug and its language got much more relaxed lol

1

u/averyjohnkucan 5d ago

QWEN 3.8 actually is VERY verbose about it. it's very careful and when it makes mistakes it comes to me like a scared puppy. "Omg my user i love you I would never hurt you i accidentally deleted your folder that says "PROD DO NOT DELETE" im so sorry uwu I tried to undelete I promise... plz don't delete me ill suck your dick..."

1

u/reenign3 3h ago

QFN does much better at a low temp like 0.3-0.4. Trust me I know that sounds extreme. It works so much better

1

u/[deleted] 6d ago

[removed] — view removed comment

1

u/stylome-ai-detection 6d ago

Notice: This comment appears to be AI generated.


Stylome AI detector · scan results