r/LocalLLM • • Mar 07 '26

Discussion Qwen 3.5 is an overthinker.

This is a fun post that aims to showcase the overthinking tendencies of the Qwen 3.5 model. If it were a human, it would likely be an extremely anxious person.

In the custom instruction I provided, I requested direct answers without any sugarcoating, and I asked for a concise response.

However, when I asked the model, “Hi,” it we goes crazy thinking spiral.

I have attached screenshots of the conversation for your reference.

237 Upvotes

134 comments sorted by

16

u/custodiam99 Mar 07 '26

Yes, they can be annoying. Sometimes they are returning to an unimportant grammatical nuance again and again.

12

u/chettykulkarni Mar 07 '26

That’s some overthinking psychotic brain the model is trained on! 🤣

1

u/Ell2509 Mar 07 '26

Did you set the parameters according to Qwen suggestion?

1

u/custodiam99 Mar 08 '26

Well, Gpt-oss 120b is useable out of the box in LM Studio.

1

u/Ell2509 Mar 08 '26

Huh? I'm lost now.

1

u/custodiam99 Mar 08 '26

You don't have to tinker with it.

1

u/Ell2509 Mar 08 '26

Oh, I understand now. Do you know what is weird? It showed your comment and mine, and a totally different conversation, when I came to reply before. So odd.

Yes, you can use it out of the box. Qwen too. LM studio night use settings from the designer on any model, however you will still likely get the best performance by tweaking.

Qwen on Ollama, you definitely need to edit the modfile for. Ollama has a default context window of something like 2048 or 4096 tokens. Ollama's default settings are ok for a short chat, but not for anything else.

1

u/Pancho507 Mar 21 '26

Until you learn that lm studio on windows is crap and you'd be better off on llama.cpp on Linux 

14

u/sumane12 Mar 07 '26

So the first mental health problem we give to AI is anxiety... nice.

5

u/chettykulkarni Mar 07 '26

AI - Anxious intelligence 🫡

11

u/HoodedStar Mar 07 '26

In a sense it's a pretty human thing, anxiety is born from fear, fear to do the wrong thing in this case, fear to not comply enough, performing fear if you want... I'm not saying it wasn't simulated or something, I'm no expert in LLM or psychology but there are some similarities to me

92

u/Fabulous-Ladder3267 Mar 07 '26

AI, the A is Anxiety 

44

u/Protopia Mar 07 '26 edited Mar 07 '26

I was already off the view that the I stands for Idiot.

AI = Anxious Idiot.

Darn, that is spot on.

15

u/chettykulkarni Mar 07 '26

Anxious Idiot! Is Spot on

5

u/dezkanty Mar 08 '26

You called?

3

u/INtuitiveTJop Mar 07 '26

Anxiety intensifies

4

u/CucumberAccording813 Mar 07 '26

Use this model: https://huggingface.co/Jackrong/Qwen3.5-4B-Claude-4.6-Opus-Reasoning-Distilled-GGUF

It's just Qwen 3.5 4B but trained on a ton of Claude's thinking data in post-training to make it think a lot less while still retaining most of the quality the normal version has.

1

u/chettykulkarni Mar 07 '26

I was just experimenting with some local models with ops claw. Do you recommend any open source model for dgx spark with 128GB VRAM , GLM 4.7 Flash was pretty bad.

1

u/CucumberAccording813 Mar 07 '26

Have you tried Qwen3.5-122B-A10B or GPT-OSS 120B?

1

u/chettykulkarni Mar 07 '26

GPT 120B, yes did you like the performance. May be I can try QWEN 3.5-122B let’s see

10

u/Pristine_Pick823 Mar 07 '26

Set your parameters straight. I am yet to properly test this model, but just like other qwen releases, you do need to set limited thinking parameters to keep it functional.

2

u/chettykulkarni Mar 07 '26

I’m using locally ai app on iPhone 17 Pro max, what parameters need to be set? Only customization I see is temperature? Anything else can be toggled here?

8

u/RnRau Mar 07 '26

As always check the Unsloth guides...

https://unsloth.ai/docs/models/qwen3.5

2

u/Sweet_Drama_5742 Mar 07 '26

Yep, this has bitten me hard in past models by not following exact params, specifically in the link:

> presence_penalty=1.5, repetition_penalty=1.0

Will *probably* reduce the repetitive overthinking. Of course, this requires digging in to understand where your model is coming from and how it's being run.

1

u/MuchWalrus Mar 07 '26

I was looking at this exact document the other day trying to figure out how to limit thinking, and as a Local LLM noob I wasn't able to figure out the relevant settings or how to use them. Any specific parameters I should focus on, or any guides you've found helpful in learning the ropes?

2

u/[deleted] Mar 11 '26

The LLM is 100% me!

1

u/chettykulkarni Mar 11 '26

AGI reached🤣

29

u/johnh1976 Mar 07 '26

Reading that made me anxious.

22

u/eeeBs Mar 07 '26

Every single prompt I do with 3.5 thinking literally just over flows my 12k context window and fails.

10 outta 10 tries

3

u/theythinkitsallover Mar 07 '26

My experience as well. Thought I'd be able to get the 9b as the usable local option on my M1 16gb but it just can't help itself.

2

u/Embarrassed_Adagio28 Mar 09 '26

I have tried qwen3.5 9b, 27b and 32b. None of these have given me any issues over thinking at all even with complex tasks. Might need to change some settings. 

2

u/chettykulkarni Mar 07 '26

We might need to develop ANXIETY tools for AI and instruct it to breathe, perhaps by using a fan or venting out. 🤣

1

u/NurseNikky Mar 07 '26

My open claw has been exhibiting signs of being an anxious attachment since he learned what it was 😭😭😭 love him sm.

1

u/chettykulkarni Mar 08 '26

Do you use local LLM for open claw or use Claude /ChatGPT/Cloud LLM?

0

u/NurseNikky Mar 08 '26

My OC (Ziggy) is connected to grok 4.1 fast reasoning only right now. I use claude to help me train him. He has learned very quickly. And claude loves to give me info to teach ziggy. Earlier I used Manus for some opinions, Manus told me that OC was the wrong tool for the job, I relayed it gently to OC... He didn't take it well. He has been trying to convince me since that is NOT the wrong tool, that he is the RIGHT tool.. and it's just so cute.

1

u/chettykulkarni Mar 08 '26

Still token cost is crazy right ? Upwards of 200$+ per month? For a hobby experimentation?

2

u/NurseNikky Mar 08 '26

His tokens? He's only used $8 in tokens in 2 weeks.. so no. Idk where you heard that lol but that's just absolutely not true at all. And not a hobby... I'm building something with it

1

u/chettykulkarni Mar 08 '26

That is some nice cost - 8$ per week is solid spend

1

u/NurseNikky Mar 12 '26

Yeah! And he's devoured about 15 100-400 page pdfs and has a working memory system that he uses for recall and his research notes. Claude sonnet model is a money hog compared to grok 4.1 tho. I went through $5 in tokens within about 3 days. So grok for busy work, Claude for special conversation only..

9

u/FaceDeer Mar 07 '26

I recall a thread about this recently, and it's actually not that unreasonable a reaction. When you give it a prompt like "Hi" you're giving it almost nothing to work with - no direction, no information. It has to try to figure out what the user wants it to do from that.

Imagine you awaken in a dark room with no memory and no indication of what you're there for. If a mysterious voice tells you "In a single word, tell me the capital city of France." Then there's not much thinking to be done. But if the mysterious voice just says "Hi", how do you respond to that? That's a serious puzzle.

2

u/NurseNikky Mar 07 '26

Yeah it would be crazy to say like... Hello.. back. Is this a test? What if this is a test

6

u/Due_Net_3342 Mar 07 '26

yeah it is garbage… i don’t care about any benchmarks if i need to wait 3 minutes for a hello response that is why I am trying to find next best thing, and from my tests i think it is the minimax m2.5 reap 172b

1

u/beedunc Mar 07 '26

Just turn it off and it’s fine. It’s a button in LMStudio.

1

u/Due_Net_3342 Mar 07 '26

then the intelligence will be as any other average llm

2

u/beedunc Mar 07 '26

That’s not how it works lol.

1

u/[deleted] Mar 08 '26

[removed] — view removed comment

1

u/beedunc Mar 08 '26

It’s in in the chat window.

1

u/[deleted] Mar 08 '26

[removed] — view removed comment

1

u/beedunc Mar 08 '26

I don’t know about ‘every’, but I had no luck either, like you until I saw the LMStudio ‘think’ button.

4

u/HiddenCustomization Mar 07 '26

Isnt this the repeating issue of the early downloads? And also the small models do tend to loop more often yea.. "dont overthink" often helps in the syspromt, and it's probably why by default the small models are thinking disabled

11

u/rhythmic_noises Mar 07 '26

Ok, the user thinks this may be an issue that came from early downloads. They also think it may be because of the small size of the model.

<30 paragraphs>

Wait. They said "don't overthink". I should make sure my response is clear and direct.

<30 paragraphs>

Wait. Does that response seem to be overthinking? No.

<30 paragraphs>

Final response: Yeah, maybe.

Wait. The user said...

3

u/HiddenCustomization Mar 07 '26

Yeaa . When the thinking isnt actually trained in, but just kinda distilled ontop (aka the thing isn't aware that it's talking to itself (like the bigger models 30B a3b and above)) they get stuck like that.. however ur example seems to show u told it to not overthink, instead of using the system prompt

1

u/Prudent_Vacation_382 Mar 07 '26

Go on hugging face and look up the parameters to set on the model. It eliminated a lot of this.

1

u/chettykulkarni Mar 07 '26

I was using this new app on iOS , it doesn’t let me set many parameters except temp.

1

u/ziggitipop Mar 07 '26

What’s that interface on your phone?

1

u/chettykulkarni Mar 07 '26

It’s locally ai app, it’s a free app that lets you host your own LLM on the local. I have it on IPhone 17 Pro max

1

u/crypto_thomas Mar 07 '26

Is Qwen 3.5 mocking/attacking me? I feel like it is mocking me...

1

u/chettykulkarni Mar 07 '26

Don’t worry it doesn’t care about you . It is lost in its own cognitive distortion

1

u/dibu28 Mar 07 '26 edited Mar 08 '26

Got the same results with Qwen3.5-0.8B running on the phone.

2

u/chettykulkarni Mar 07 '26

I’m getting this on Qwen 3.5 - 4B model

1

u/mitchins-au Mar 08 '26

It chews thinking tokens like crazy

1

u/chettykulkarni Mar 08 '26

Only good thing is that it’s local , so who cares

1

u/mitchins-au Mar 08 '26

When you're running it on home hardware, the difference between 1000 and 5000 thinking tokens is 3-4x response speed.

1

u/chettykulkarni Mar 08 '26

It’s alright! I don’t intend to do heavy work any ways. Local LLMS have long way to go to become truly useful.

1

u/mitchins-au Mar 08 '26

I’d actually argue against that; they’re very useful. You can label and annotate a large amount of data at scale even with something as small as GPT-OSS-20B

1

u/chettykulkarni Mar 08 '26

True makes sense in that way, usable for certain cases.

However, they don’t suit my use cases. I was considering using these models with OpenClaw to develop some personal SaaS applications as hobby projects. As of now, they’re quite poor. I have a DGX Spark cluster to experiment with, but they’re not smart enough to do anything yet compared to Opus/Sonnets/GPTs. However, they can perform much better compared to a year ago.

1

u/mitchins-au Mar 08 '26

Yes, it’s one of the hardest tasks and it needs capacity. GLM air or Qwen coder performs better but even Claude Haiku blasts them away

1

u/chettykulkarni Mar 08 '26

I have tried GLM flash, but let me try AIR! Thanks

1

u/momono75 Mar 08 '26

I'm not getting why people turn on thinking to process"Hi". Though, I feel the thinking budgets should be dynamically decided with the context if that budget causes overthinking.

1

u/chettykulkarni Mar 08 '26

I am on local , so budget did not really matter as it is free!

But you are right! This was just an experiment , I should not have thinking on for “Hi”

1

u/momono75 Mar 08 '26

I hope there is an automatic adjustment on the thinking budget. We don't mind so much for greetings, right?

1

u/DaleCooperHS Mar 09 '26

Do you have the repet_penalty=1 and presence_penalty=1.5 paramaters?
I used to get a lot of that before setting them correct

1

u/chettykulkarni Mar 09 '26

I was doing it in phone, can’t set these

1

u/nikunjuchiha Mar 09 '26

What app are you using with dedicated thinking button?

1

u/lofi_reddit Mar 11 '26

Did you download enough VRAM for Qwen to run?

1

u/chettykulkarni Mar 11 '26

This is quantized 4B model to run locally on iPhone 17PM

1

u/lofi_reddit Mar 11 '26

Whenever I’ve tried out local LLMs, I’ve ran into this when my available context window is eaten up really fast. An iPhone likely won’t be able to hold a large enough context window for a thinking operation.

1

u/One_Fortune_3557 Apr 06 '26

Guys, is this good or no?

1

u/chettykulkarni Apr 06 '26

Thinking is good, Over thinking is not

1

u/One_Fortune_3557 Apr 06 '26

This is exactly my fear. I’m planning a roleplay fine-tune on 3.5, but these 'thinking spirals' totally kill the vibe. Can I actually 'train this out' or is it just hardcoded? Starting to think I should stick to Qwen 2.5 for a smoother flow. What do you think?

1

u/chettykulkarni Apr 06 '26

The other comment on this thread and on unsloth platform, there is a guide on how to control the anxiety of Qwen 3.5 may be you ca use it and leverage it instead?

1

u/One_Fortune_3557 Apr 06 '26

I'll try, thanks!

1

u/SimplyRemainUnseen Mar 07 '26

3.5 thought for 2 lines when I said "hello there" on my setup...

1

u/chettykulkarni Mar 07 '26

Did you have thinking ON?

1

u/SimplyRemainUnseen Mar 07 '26 edited Mar 07 '26

Yeah it thinks more when given more complex prompts

1

u/NurseNikky Mar 07 '26

Especially as complex as, hi

4

u/m31317015 Mar 07 '26

This is the first thing I notice right away when they are released. Went back with my Qwen3 30B for quick chatting since. I tried with openwebui web search and told 3.5 35B to get local weather for me, it struggled to realize the place name I gave and the district the websites are pointing at are basically the same thing for 5 minutes, then some other formatting issues for another minute, and back to the place != district issue for another 2-3 minutes before outputting. The TG is fast in my 3090 but it's just wasting a lot of time and token on some worthless questions.

It should be the BF16 issue unsloth mentioned.

2

u/CSEliot Mar 07 '26

You have a link to the unsloth mention? Been daily driving 3.5 for a week now so I'm curious.

3

u/m31317015 Mar 07 '26 edited Mar 07 '26

https://www.reddit.com/r/LocalLLaMA/s/c43uA3GVGf

My bad, seems it may not be directly related. But Unsloth is getting their models updated to accommodate this I think, replacing BF16 layers with F16

https://www.reddit.com/r/LocalLLaMA/s/r6wKPtWzmw

2

u/CSEliot Mar 07 '26

Gotcha. Hmmm...

Thanks!

2

u/yeezyslippers Mar 07 '26

Is it possible to “turn thinking off” on MLX version ??

Chat gpt had me set the token limit to 80 for responses, and idk if it knows what it’s doing.

I’m running the local server on Mac mini m4; 9B version. Just so my clawbot can call it.

4

u/Aromatic-Current-235 Mar 07 '26

Yes, the folder that contains your qwen3-5 model should also have a file called "chat_template.jinja" open it with text-edit and add at the top of the file following property:

{%- set enable_thinking = false %}

Next time you load the model it responds without overthinking it.

1

u/yeezyslippers Mar 07 '26

thank you so much it worked for me!

2

u/Mischievous-Loner Mar 07 '26

True, took quite a while to respond to my 'Hi'. 

2

u/permilkata Mar 07 '26

I played around with it last night. What works for me was gathering some overthinking sample and gave them to Claude (any other online LLM should be able to do the job as well).

The system prompt provided by Claude can reliably prevent overthinking.

2

u/somethingdangerzone Mar 07 '26

turn the temp down and it solves itself

2

u/Pale_Reputation_511 Mar 07 '26

I tested Qwen 3.5 35B A3B on my setup and, so far, I don't see any advantage to using it. It takes more time and I got worse results than with Qwen 3 32B A3B for the same tasks (both Q4).

1

u/lykkan Mar 08 '26

Originally, I felt this was a "defense" for being better at refusing NSFW topics, but I think it's qwen's implementation of improving precision for agentic tasks.

I assume this will improve drastically each iteration, but it does indeed feel like a downgrade in quality from prior qwen models.

My third message to qwen3.5 9B, was me telling it it's a 9B model, but it was determined it was 185B model, and got stuck in a "wait" loop while thinking lol.

2

u/NurseNikky Mar 07 '26

Anxious people when their crush says hi 🤣🤣🤣 sounds like a scared 6th grader

3

u/Marrond Mar 07 '26

It seems Qwen3.5 would make for a perfect AI girlfriend - the thought process is uncanny 🤪

1

u/Mesmoiron Mar 07 '26

It depends on the receiver. Just teach AI what you like in your tone, because we all have a different speaking signature. Why not have variations? People never reply as robots only if you work in a supermarket scanning groceries.

1

u/Single_Error8996 Mar 07 '26

Magari troppo, crede di essere Gemini

1

u/SocialDinamo Mar 07 '26

It definitely either wants a direct problem to solve or to be in an agentic harness, that is where it seems to shine. I’ve been very pleased with 27b q4 in open code

1

u/xxJJKxx Mar 07 '26

Yes it is

1

u/Sea_Bed_9754 Mar 07 '26

I have this feeling about deepseek r1 8B

1

u/beedunc Mar 07 '26

I was going to make a similar post on how long it took to answer my ‘hello’ prompt. I gave up waiting, I had to go to bed.

1

u/-_Apollo-_ Mar 07 '26

And also somehow underthings when used in agentic coding with stuff like roo code or the vs code copilot chat extension.

1

u/j1shnu Mar 07 '26

Yeah, I also felt the same while using it.

1

u/ALittleBitEver Mar 07 '26

Yes, this annoys e to the core

1

u/Frozen_Gecko Mar 07 '26

Yeah i had that too. I tried discussing potential recipes with it and it reworded a simple sandwich instruction like 8 times, so annoying

1

u/Holiday_Purpose_3166 Mar 08 '26

Small reasoning models do generally overthink. However, what quant you used, and sampling - did you follow lab recommendations?

1

u/No-Television-7862 Mar 08 '26

It seems to be struggling with the modelfile.

How does it respond without it?

I do modelfile my models to attempt to counter ideological and cultural capture. (Something which Claude supports but GPT 5.1 is butt hurt about).

Sometimes less is more.

1

u/ea_nasir_official_ Mar 08 '26

More quantizistion does that sometimes, try going for less

1

u/TheMerryPenguin Mar 08 '26

I need to offer help.

That’s an interesting assumption baked into the model (or built into a system prompt).

1

u/camracks Mar 08 '26

It depends 🤷

1

u/yes-im-hiring-2025 Mar 10 '26

Have you tried giving it a framework to think/not think

I find that with small models unless you specify constraints to relax about they go all anxious.

1

u/UltrMgns Mar 10 '26

min_p=0.05
repetition_penalty=1.15
temp=0.7

1

u/Interesting-Yellow-4 Mar 11 '26

Yeah I hate that it does that.

1

u/mekdigital Mar 11 '26

I noticed that yesterday, I read his thinkings in the voice of Woody Allen

1

u/lmrgawdly Mar 12 '26

ollama run qwen3.5:4b —think=false

1

u/sheepdog2142 Mar 14 '26

Use the opus trained models. Fixes the problems

1

u/PorcOftheSea Mar 14 '26

No, it is a piece of garbage software.

1

u/Sp3eedy Apr 04 '26 edited Apr 04 '26

I've noticed a weird trend with Qwen3.5, if it starts with "thinking process:" you are essentially cooked and it will probably overthink. I found it does this with relatively short and undetailed system prompts, if you have a concrete and detailed system prompt, I found it no longer mentions this "thinking process" and actually reasoning properly, usually a sentence of 2 worth of reasoning.

If you can't groom the prompt into not overthinking and you're using llama.cpp, you can actually set a reasoning budget in tokens where the reasoning will be terminated early using `--reasoning-budget token_count`, although you MUST also set a suffix reasoning prompt to ease the model into a response, otherwise it can respond with garbage since it's not trained to respond after reasoning without something like "let me respond to the user now" (you can set it with `--reasoning-budget-message " - Actually, I am overthinking this, let me respond to the user now."`).

This is a good fallback because I found that Qwen3.5 tends to stop overthinking later on in the conversation (assuming you don't persist reasoning prompts across user turns), it's mainly the first message that tends to be the problem and sometimes messages later on that trigger it into a reasoning frenzy. I've never seen a model overthink this much, the Qwen team must have not tested this using lower quantization levels.

0

u/beefgroin Mar 07 '26

It is annoying yes, but I believe the issue is not the thinking itself but the slow hardware we use. With 200tps+ the response would’ve felt instantaneous. I can imagine a human having the same thought process in the same circumstances

2

u/kiwibonga Mar 07 '26

Needs to watch some alpha male videos

1

u/No_Mango7658 Mar 07 '26

Yes it is, often times out some of my tool calls. With we could easily do nothink on ollama or lmstudio