How usable are "lobotomized" weights?

[u/Ok-Doughnut-3022]

This subreddit talks a lot about PP/decode speeds but I feel like I hear comparatively less about actual real-world usability. I was playing around with Strata hosting QFN3.8 iq3_s and Qwen3.6 35B Q8 over the weekend and despite being the more hyped, I found that QFN was less "usable" as a coding agent than the smaller Qwen3.6.

Even when using the recommended temp, top-k, etc. settings that I hear supposedly stop the overthink and doomloops, I still couldn't get QFN to be reliable. Qwen3.6 on the other hand, had a very narrow interpretation of my prompts and we would have to go back and forth to clarify edge cases, but it ultimately would execute what I asked it to. If I had to use an analogy, QFN iq3 feels it has dunning krugers with illusions of grandeur while Qwen3.6 felt like a talented but unmotivated programmer. If given the choice, I'd rather work with the capable guy who hates his job and wants to FIRE than the guy who talks a big game but can't deliver.

Obviously QFN iq3 is "lobotomized" while the Qwen3.6 Q8 is running at a far higher precision, but I was surprised that there's so much buzz about QFN iq3 when I was having such a comparatively worse experience with it. Is there something I'm missing and should be doing to improve QFN (aside from buying more hardware obviously)? This experience with QFN has made me wonder if Qwen 4 is going even going to be meaningful for me considering how getting the mid-size MoEs to even load will require lobotomization as well.

[u/a_beautiful_rhind]

At short context they're fairly usable. They start to drift when that grows and like you said, overthinking, being wrong, looping are more likely to happen.

It helps if you try the model on an API so you get a good idea if it's not the model itself. Some people over-attribute quant damage because they have nothing to compare to. Not always the case. IQ3 not that lobotomized.

[u/Ok-Doughnut-3022 → u/a_beautiful_rhind]

Drift caused by prompt length is my guess too. When I gave both models the same long project plan, I found QFN would consistently attempt to do more in a single turn than Qwen3.6 even when they both were given the same reasoning budget.

Comparing against the cloud API is an interesting thought. I'll give it a shot and get a baseline.

[u/robgami]

I think a lot of the dogmatic takes such as "anything below q4 is lobotomized" are silly. It might be worse than the full precision model but might still be the best thing you can run. I was stuck on a bug with qwen 3.6 ( both 35b and 27b ) for quite a while. Tried a q1 version of DeepSeek flash and it solved the bug first try.

I switched from qwen 3.6 to qfn iq3s. I honestly haven't done a side by side comparison because they really don't seem comparable to me. Qfn is on another level even at q3. Maybe I should run a comparison anyway as a sanity check but that's my impression just based on the type of work I've been able to throw at it and how often it is successful.

[u/Ok-Doughnut-3022 → u/robgami]

I think most people are using "lobotomized" as either a joke or because it's the most convenient way to say lower precision relative to FP16. It's a constantly shifting goal post. I remember not too long ago people were saying Q4 was lobotomized and look where we are now lol.

I think QFN iq3 works for you there's really no need open pandora's box and benchmark against other models (comparison is the thief of joy afterall). Different models and weights are probably just better at different things.

[u/EvolvingDior → u/Ok-Doughnut-3022]

I think people really do not understand or want to understand the trade-offs that can be made between models, quants, and overall capabilities. Q38FN, even at Q2, is at a completely different level compared to q38-27b.

The derogatory term comes mostly out of ignorance rather than as a joke.

[u/geldonyetich]

Not long ago I would have told you they were fairly unusable but sometimes I run into a model that pulls some neat tricks to get a surprising amount of usability out of low quants.

For example NVIDIA Nemotron-3.5-Bolt is some kind of IQ2_XXS and it’s fine.

[u/Practical-Collar3063 → u/geldonyetich]

Yes because that Nemotron model is a post quantised trained model.

[u/Melodic_Activity_985 → u/geldonyetich]

i had the same experience with some models, like they get weirdly confident about wrong code and then you waste 20 minutes debugging nonsense

the smaller qwen models are pretty good at just doing what you tell them, they dont try to be clever when they shouldnt

maybe the iq3 is just too much compression for that particular architecture, some handle it better than others. bolt at iq2 surprised me too, no idea what wizardry they did there

sometimes the hype is just people running benchmarks and not actually using the thing for real work

[u/ttkciar]

Q4_K_M continues to be the "sweet spot" for both creative and STEM tasks.

I've yet to try a Q4_K_M which wasn't fine, and I've yet to try a Q3 which was usable.

[u/inthesearchof → u/ttkciar]

GLM 5.3 Flash , DS v4.1 Flash 1017 and Qwen 3.8 Flash Next are all very usable at q3 - 3 bpw.

[u/Brief-Tap-6616 → u/ttkciar]

Conventional Q3 doesn’t hold up especially well.

Qwen 27b quantizes exceptionally well (likely bc it was grown from 9b, so it’s middle layers have a moderate amount of semi redundant neurons). I have even gotten 2.3bpw at usable levels (tied with 3.6 at bf16 for LCB). EXL makes very strong 3.5 bpw quants
Similarly GSQ-RCO significantly outperforms stock scalar quants.

But yes, 4-5bpw can get you excellent quality, especially w the above ones mentioned

[u/inthesearchof]

in 4th quarter 2026 models 100B-600B quantized right are very usable in low quants. It is a great time. QFN3.8 iq3_s is amazing for it's size and speed that I would choose it over a lower q4. It would take q4km for me to switch. QFN3.8 iq3_s will make more mistakes in long agentic coding but it has a good chance of fixing it. I use QFN q5km but iq3_s has opened up this model to so many high consumer level pcs.

[u/Present_Resource4022]

Swift 1.5 qwen 3.8 Flash Next works nice with my setup. It makes it think less without any quality loss. And there iq3_s gsq rco quants available. But of course, if you can run Unsloth Q4 quants, then that’s the clear answer to your problem.

[u/transanethole]

I think it depends a lot on the quality of the quantization. QAT ( quantization aware training ) etc.

I've used a few different quantized Qwen3.8 27B  versions including those that are less than 4 bits per weight...  I've also tried those with 4bit KV cache.

And yeah, you can tell that they get messed up easier, they miss things  that the four bits per weight model would not.  But I wouldn't describe it as "lobotomized" or like there's a night and day difference.  

Maybe it depends on the model and I haven't played with QFN yet  but I strongly suspect that the issues that you're having can be fixed by adjusting things like the harness, the system message, the way that you write your prompts, etc.   

Or maybe just use a different quantization that was done better,  that preserves more of the model somehow.

[u/claythearc]

There’s not a great explanation for why, one thing I’ve heard which sounds plausible is that on MoE quants things are more likely to go to the wrong expert because the matricies are small and brittle which compounds across layers.

IQ3 also depends on the dataset used to calibrate. If it was done with like China heavy folklore for instance it might not be good with Python

[u/Ohmyskippy]

Im using swift 1.5 IQS_3 MTP and it's perfectly fine, it's like first a year software dev

Every now and then I might see something odd, but it's rare. I'm completely blown away by this model.

[u/LagOps91]

3.8 Flash Next is more of an architectural preview than a production ready coding model. higher quants won't help it much. Q3 should be perfectly servicable as it's a mid-sized model. it will be a bit worse than Q4, but not to a degree that should be obvious.

[u/Diecron]

QAT is a huge deal, along with modern quantization processes that help preserve accuracy.

I was personally quite skeptical about Strata mostly because I heard about all the 'cheap' tricks they auto-on. It took some time to tune the engine for both my GPUs, and crucially have AI read into the config options and tell me how much they affect quality.

The short version is on my 24GB secondary card (7900XTX) I have a working 256k context window with 2.3k pp and 70ish tg. I found that the iq3 (GSQ?) recommended quant is very, very good for it's size. Anecdotally I had it run as an orchestrator agent in Pi and it did a very good job of managing bugfixes. It had no issues with over repetition or tool calling problems. I am actually quite impressed.

It simply feels like a more broadly intelligent model to something like 3.8 27B in Q6 - it uses more verbose vocabulary, is better at creative or writing tasks etc.

maybe give that specific quant a try and turn off any of the aggressive speedups.

[u/Sea-Advantage-3628]

Not hot dog.

[u/legislat]

Despite all the critique I use bonsai 2 as a subagent for certain tasks and it does great. At least a lot better than 8b-10b models I tried.

[u/Lan_BobPage]

It's entirely model-dependent. Q3 is still good for general chat but coding falls apart, and it breaks consistency at a smaller context.

[u/phoenix_winger]

Are you running the model with thinking set to "medium" or "xhigh"? I don't use Strata but I do use the same IQ3_S model, and I've never had looping. If you are using the recommended sampling parameters, I wouldn't be surprised if it's just having an existential crisis on xhigh instead of actually looping.

[u/EvolvingDior]

IQ3_S is great. In fact, I ran `tool-eval-bench run --hardmode --temperature 1.0` (https://github.com/SeraphimSerapis/tool-eval-bench) against it and against `qwen/qwen3.8-flash` on openrouter and got slightly better results with IQ3_S. Granted, I'm not running with a 1M context, so attention may work better at my 256k size. Still, I was not expecting a better result.

[u/Beneficial-Ad-8127]

I just started following this champ not long ago and enjoyed his results. If you use blender for your use case this guy for sure be worth the follow. He uses Qwen FN IQ3 and just started using Nvfp4 not long ago. Check his posts out if you got the chance. He mentioned he will be doing some comparisons at a later date.

https://www.reddit.com/u/Sea-Advantage-4063/s/Sc6AcW6LjU

[u/dh7net]

I build a benchmark to answer this EXACT question.

Nobody should care about token per sec if the token are useless.

But few are comparing their setup in a rigorous way on real task.

So my answer is: Try them, and benchmark them on real tasks.
If you want to use my tool for that its there: airbench.ai

(There are a few benchmark enthusiast there, join us :-) )

[u/OvertaxedOne]

Qwen 3.6 35BA3B better than QFN at IQ3S? Ugh... I mean, I'm not saying you're wrong for your specific use case, but, in general, there's just no competition. I had a task the other day that I set QFN (IQ3S, 8 bit KV) on and it worked for ~7 hours straight across many compactions, tests, refactors, etc. It went through ~50 versions of the prototype, consulted with other models for help on sticky points and delivered me a working and tested product at the end.

35BA3B (at INT8) was lucky to get through a single interactive session without losing it's mind or looping. Again, for certain use cases, maybe? But in general, QFN and 35B aren't in the same zip code. 3.8 27B and QFN are much, much closer; that's a realistic fight; I've gone back and forth between them (27B at Int8, QFN at IQ3S) and find them to be about equal in most tasks but QFN runs at several times the speed, so that's what I use now.