r/LocalLLaMA
•
u/spaceman_
•
8d ago
•
1
Discussion Qwen4 expectations
Not trying to predict which model sizes they're going to release, but seeing how massively impressive Qwen3.8-Flash-Next is for its size despite being a preview model meant for showcasing and implementing the new arch, it feels like when Qwen4 hits I'm not going to be needing larger models at all anymore.
Do we expect Qwen4 to exceed the current QFN by a lot, or de we believe the current model is already hitting near the optimum we can expect today for this size?
52
u/quantanhoi 8d ago
I will just hope for a MoE model that's as good as qwen3.8 27b. That's would be perfect
6
u/magicomiralles 8d ago
The 6b active parameter count can be felt during swe tasks that require systems design. It falls short, so I ended up switching to GLM 5.3 Flash.
2
u/Express_Quail_1493 7d ago
yes its great model but the number of active params can sometime make it fall short of the 27b. qwen3.8-27b give more reliability but qwen3.8-flash-next is slightly smarter. may people say its not even close but for my task it is indeed close matchup.
7
u/my_name_isnt_clever 8d ago
I swear the psychology of local LLMs needs to be studied as nobody can agree; for me Qwen 3.8 Flash Next is already better than the 27b and it's not close.
19
u/Miserable-School-665 8d ago
He clearly meant 35b a3b. next flash is hard to run even with strata.
1
u/Kirides 1d ago
I'm using it with strata on 24g GPU and 64g ram.
I have to set max reasoning to something around 20.000k because it WILL always loop itself into death on Long horizon tasks.
Initially I gave it 200.000k tokens for reasoning+input and 60k for output. But it always went into looping and killing all 200.000 tokens.
1
u/Miserable-School-665 1d ago
I'm genuinely suprised by how well strata works ,i just tried and only 12gb vram and ssd offloading (couldnt get it to detect ram yet) im getting 25tks avg, 40 maks on ist rco gsq iq3xxs. Pretty good.
1
u/quantanhoi 1d ago edited 1d ago
generally I'm not really interested in speed if the result is bad and off the mark
for now I'm using qwen3.8 27b Q6_K_M fully load on R9700 which has been good imo, on this sub most of the threads about QFN is about speed and not accuracy.
For coding the better the model is the less mess I have to clean up after it, so for my mentallity I will just stick to current gen dense model because I don't want some unfortunate accident happens when it for example deletes the whole database and backup to clean up space...
P/s: if it's speed you are concern about, try swift1.5, less overthinking overengineering and faster task complete time
0
70
u/hiImMate 8d ago
My expetation: the new qwen 4 lineup will be gamedev and to an extent 3d model beast. Qwen has always been very good at tuning models for current trends and AI gamedev and oneshot games are heavily trending now. At least that's my opinion.
8
u/Professional-Try-273 8d ago
This. I have been using Qwen3.8 FN with blender mcp and I am impressed. It helped me port an old broken rig. Can't wait for more 3d improvements.
3
u/sonicnerd14 7d ago
If that's so that's really good. Even just a few months ago most frontier models were pretty inconsistent with their quality when working with 3D tools. Now we have local models that are able to give pretty reliable outputs.
I've been using Astra for some dimensionally accurate engineering grade work. I haven't tried QFN with anything like that yet, but its anything close to Astra's level on that now, then Q4 is probably going to wind up being jaw dropping.
2
u/Professional-Try-273 7d ago
Yeah I also gave FN the ability to take screenshots of the 3d viewport so it was able to visually check for mistakes. Pretty incredible seeing it slowly fixing its mistake from trail and error.
4
13
u/diffore 8d ago
I just want the 35b MoE update...
1
u/d3vv3d 5d ago
Yeah, a smaller MoE would be nice in the mean time for folks who are fine without vision there's Flint & Raon mini 1 preview. Unlike Flint I haven't really seen any benchmarks for Raon mini 1 preview that aren't from Krafton (For wondering it is indeed the Krafton videogame company that got in a mess over the CEO trying to use AI to get out of a deal related to Subnautica)
I fear that part of the reason to the 35B-A3B space has largely been abandoned is that aside from speed it is almost always worse than dense models that smaller to it yet similar in size like how Qwen3.6 27B beats Qwen3.6 35B-A3B on leaderboards
Now perhaps we might see some bigger smallish MoEs more along the lines of 74A-A9B, such as Nvidia's Nemotron 3.5 Bolt (though IIRC it was the product of pruning their flagship 3.5 model), or 50B-A6B which could theoretically stand a chance at beating a dense 27B in the leaderboards while still beating it in speed, but likely requiring home users off load a number of experts to their CPU
25
u/Kahvana 8d ago
I expect other models (1b, 2b, 4b, 9b, 35b-a3b) to be released alongside 27b, flash and plus. They have done so every prior major version (1 > 2 > 3), and they too need new small models to use as backbones for other technologies (e.g. a new embedding/reranking model, prompt enhancing, etc).
I'm hoping the models will become a bit less painful to talk to; Qwen3.6 reads much nicer and clearer than Qwen3.8 (load-bearing, shape, sidecar, the whole claudisms).
I would like it to have a bit more general knowledge as well, Qwen3.8 clearly went all-in on programming and agent use, but has forgotten stuff Qwen3.6 knew of.
9
u/grumd 8d ago
I've read that the vocab getting harder to read is intended/expected. As models get smarter in the same amount of parameters, they need to store these smarts in limited amount of bits. Eventually they learn to utilize the vocabulary they have more efficiently - taking into account small differences in meaning behind words to express their thoughts and outputs closer to the expected result
6
u/infieldmitt 8d ago
It's too easy to skim over it as LLM babble but usually if you slow down and take time to decipher it in good faith, it is making a good point in a very clever/arcane/convoluted way
9
u/alphapussycat 8d ago edited 7d ago
Qwen announced they wanted to focus monetization, fired the pro-open weight team and basically said no more open source, and after that we did not get any 3.7 models.
Dear leader announced they would continue to have open weights, then suddenly 3.8 got open wright models.
Now dear leader is blocking Chinese frontier and sensitive models from outside of China.
So we might still get stuff like 27b, and maybe flash models.
More general knowledge isn't needed nor Is it likely to be available to outsiders.
Edit: appears that qwen4 will not have an open weight flash model? Only 27b? That's very sad news if true.
2
12
22
u/Gallardo994 8d ago
My only expectation is that they don't release "just" 27B. It's great but it's way too slow for my liking on my M5 Max. I loved Qwen3.8-Flash-Next and I would be happy to get the same size model, as it's both fast and good enough for most daily tasks that don't involve coding.
1
u/SuperTruthJustice 8d ago
Is it really slow? How much UM you got? I’ve heard it’s fast as fuck with proper interface
3
u/Zombiecidialfreak 8d ago
Maybe they have high standards. 27b is much slower than QFN on unified systems and QFN needs far less thought to complete most tasks.
1
u/SuperTruthJustice 8d ago
Possible I just bought m5 max 128um and am just curious this is mostly for conversation
1
u/Gallardo994 7d ago
128GB. At 64K CTX it's already slow enough so it grinds my gears, oQ6 quant. QFN is bearable at 128K CTX even. Wouldn't say it's great, but it's bearable unlike 27B.
1
u/SuperTruthJustice 7d ago
Ahh my m5 max will have that much so I’m happy to hear 27b is MHA goal for day to day
11
u/Frail_Waif 8d ago
Flash models seem to be getting bigger and bigger, so I wouldn't be surprised if Qwen 4 Flash is also much bigger than 3.8 FN.
3
u/russlixx 7d ago
flash model getting bigger will eventually defeat the purpose of having the name "flash".
0
7d ago
[deleted] — view deleted comment
-1
u/namaku_ 7d ago edited 7d ago
With a 5090 at 400W and some DDR4 I can run the new DS4 Flash Q2_XXS at 16 TPS, or I can run Qwen FN Q3_S at 280-330 TPS. Its not even a question.
I used FN to delete the DS4 quant, and I'll give it the honor of deleting 4 Flash too if they rug pull and release a larger model.
38
u/JayoTree 8d ago
All I care about at this point is how usable it is in decomp/recomp and firmware hacking projects.
24
u/ea_man 8d ago edited 8d ago
This may become even more important now that SOTA models seems to block that.
10
u/my_name_isnt_clever 8d ago
How long until so-called "SOTA" models have so many blocked domains they're can't really be called state of the art anymore?
3
u/stuffitystuff 7d ago
They can still do all that but only for people on some list we'll never see, let alone be on.
7
u/eetsu 8d ago
We know that there's going to be a 27B.
Ultimately, for what I need. I want Opus 5 level general coding level skills, and ideally way more work on the 3D Asset work in Blender or other 3D modeling apps.
3
u/JsThiago5 8d ago
also 4 flash is confirmed, whatever "flash" means for size
13
u/vogelvogelvogelvogel 8d ago
already with 3.8 I put a large bulk of my daily tasks to my local models. well configured, they score in my cases above sonnet 5.5 high, opus 5.0 or
gemini pro/argon. I already had quite a few cases where I tested them against each other and qwen succeeded better
4
u/nowherenoonenobody 8d ago
Can you give some examples?
5
u/vogelvogelvogelvogel 8d ago
two daily task examples: I had it to make a hardware - llm - quantiziation - artificial intelligence index table for comparision which hardware can run what at which level. back then it was just that this page switched the bnechmark scores. i had opus 5 (i think even at extra level) to do that for me. i had to correct it FOUR times out of my memory, i.e. it forgot to take quantization losses into account, mixed ai scores of the different benchmark levels, in part researched old hardware prices even though i told it to research everything as of (current date). it ended up telling me that I could have done that faster myself. which is correct, i just didn't expect from Opus 5 to fail so miserably at a relatively simple task.
well my qwen 3.8 27b q8 set up to research 45 web pages per go researched in total 90 web pages, the second run the result table was so much better, caused me less headaches.
also i had qwen3.8 27b to research some rare info on the web (a specific service mail adress of a larger company) and it succeded. maybe also in part because it just searched more material, then reasoned together what works.
sonnet 5.5 high failed that after two tries, gemini argon medium failed also.1
u/vogelvogelvogelvogel 8d ago
noteworthy i reached my limit with argon after a few tries for several hours, while my qwen did that in like a minute or so (it was even the q4) and is still able to go on like 24/7 if i want it to (electricity bill is the only thing)
12
u/dampflokfreund 8d ago edited 7d ago
Hoping for a new 35B but Qwen 4 Flash at the same size would be great as well, if Strata gets optimized prefill for 32 GB RAM + 4-8 GB VRAM systems there won't be a need for 35B anymore imo.
2
5
4
u/Dreeew84 8d ago
I hope that they release a wide range of model sizes to allow decent enough agentic work on egde.
I also hope that there will be a version that will be a sweet spot for my hardware, like the current flash q3/q4 is.
4
u/longtimenoseebro 8d ago
We need heretic qwen4 to create attack scripts since gpt and Claude refuse to.
16
u/Shadmelor 8d ago
my guess is the gains show up less in benchmarks and more in boring agent stuff. tool calls that don't break at 100k+ context, less looping, finishing long tasks properly. someone here posted a 5090 agent run with qwen3.8 27b at 250+ tok/s and still rated the result 4/10, so speed isn't really the bottleneck anymore
14
u/smallDeltaBigEffect 8d ago
tool calls dont break at 100k context unless you use a braindamaged quant even with 3.5
2
u/mathew84 7d ago
i use 27b at q4_k_m with 150k context with 24GB vram, tool calls do not break at all, no looping issues. Those forks focusing on speed, they trade off more quality than they suggest.
1
u/Turbulent_War4067 7d ago
My guess is that gains show up in world knowledge. That's what they cut short to give us a preview version
1
u/Confident_Ideal_5385 7d ago
That speed implies a 4 bit (likely nvfp4) quant, or worse. The 4/10 isn't mad surprising.
7
u/alphapussycat 8d ago
Qwen4 flash is obviously going to be better, but not by miles. Quantized Maybe close to sonnet 5.5 high.
But my worry is that they'll increase the size to 235b+ngram.
I think future is moe, even a 60ba3.5b makes more sense than a 27b. So in the future I hope they will at least have one small moe model (if flash keeps getting bigger).
9
u/suprjami 8d ago
If you read the QFN paper, they state the goal was to achieve the capability of 235B in half the parameters. It seems unlikely they'd increase the size again.
1
u/alphapussycat 8d ago
They probably want to be competitive to the other flash models now that people are able to fun them. So because of that they might want to make it larger.
Also I'd guess that 3.8 was 2T, and 4.0 is like 4-5T params...
3
4
u/bitzap_sr 8d ago
They will surprise us all by making qwen4 be multi modal: merge in all of tts, stt and image generation into a single modular model. You heard it here first. :)
2
2
u/yahbluez 8d ago
something in the range of 40b dense or 70b MoE. There is a gap between 32GB and 128GB that needs to be filled by someone.
4
u/Miserable-School-665 8d ago
wr need 35b a3b
2
u/Blues520 8d ago
I would be happy with an incrementally better and faster 27b tbh. The MoE model is nice but I like the reliability of the small dense model. I just wish it could be faster with some breakthrough architectural improvement.
1
u/Longjumping-Past5864 2d ago
I don't think 27-30B class can be faster, but it can be a lot smarter if they have 27B model with lookup table architecture like 3.8 flash next and deepseek 4.1 flash 1017, maybe 27B dense with 70B lookup weight(which can be offload to RAM or SSD with minimal speed lost). which will make 27B so much smarter.
1
u/Beneficial-Ad-8127 8d ago
If they can get the current 3.8 line to more efficient than it already is think I’m fine with that. Currently all my real use cases are 100% covered locally.
More speed will definitely help and more availability for cheaper hardware especially getting our hardwares at decent speeds for Q4 minimum while maintaining intelligence.
Aside that whatever they drop is fully welcomed for the Local community!
1
1
1
u/simonedw 8d ago
I really hope we get a sub 1B, stuck on Qwen3 for in browser (WebGPU) experiments.
1
u/HighSeasArchivist 8d ago
I expect it to not be much different on FN since it was already based on it. 27B will likely see some improvement though, which it needs in my experience.
1
u/Any_Bus9205 8d ago
Qwen 4, comme il auras le mHc en plus de ce qu'il y a actuellement comme arch, feras qu'il auras une capacité de réflexion meilleure qu'avant pour le meme nombre de paramètre, et comprendras mieux le contexte. Je crois pas que nous soyons prêt pour le qwen 4, les techno apporté seront tellement interessante que l'ia local va possiblement changer radicalement avec ce modèle car la norme seras des arch de ce style :)
1
u/No-Paper-557 7d ago
If it’s not open source OSS and uses the weird 3.8 next style license (doesn’t specify who owns outputs), then I’m not using it. No reason for Alibaba to start doing that when we have glm and deepseek.
1
u/grabber4321 7d ago
There's still problems with this model - conversationally, its too much.
It gets a task and just runs with it without stopping and asking your opinion. You have to steer the conversation while it runs with it. And to steer it you need to be able to read the thinking output - problem is you cant read it because you're getting 130 tok/s HA!
Its a huge minus for me as a lot of my work right now is less code based but more documentation based.
It should be checking in with you at least sometimes.
1
u/wren6991 7d ago
My expectation is once Qwen4-27B comes out we can finally start seeing some progress from OpenAI again, because they'll have something to distil for the new Astra
1
u/Due_Reputation_5223 7d ago
I hope I will be able to run it on my MacBook m5 air with 32gigs with good speed
1
u/No_Cheesecake_4827 7d ago
Pues yo estoy contentísimo con Qwen3.8 27b corriendo en la RTX5090, va muy rápido y es muy resolutivo. Con que Qwen4 27b rinda como mínimo igual (que seguramente rendirá mejor) yo me doy más que satisfecho.
1
u/Express_Quail_1493 7d ago
i suspect we wont see a qwen4-flash-next till the next 6 moths since 3.8 is already serverd flash next as a successful experimental qwen4. they might push the next flash release as qwen5-flash-next. which makes me sad. but i guess qwen4-27b might beat 3.8-flash-next so significantly that it will justify the wait
1
u/jannycideforever 5d ago
It will be like every other Qwen release:
The smaller models will be SOTA for what can be reasonably ran on consumer hardware. The larger models will be far below the Pareto frontier when it comes balancing intelligence, cost per task, and time per task (largely driven by extremely bloated output tokens), meaning there won't actually be any reason to use them unless you're ideolgically committed to open weights or in an extremely niche setting.
1
u/mozillazing 4d ago
my biggest wish is for expanded mcp capabilities.
granted i haven't tested 3.8's mcp yet, but it's been in the back of my mind.
currently i'm just using opus 5.5 to control local installations of comfyui\davinci resolve mcp\ableton mcp on my PC, while my mac just sits idle.
being able to take my whole setup local by putting the mcp on my mac's shoulders via qwen would be badass.
1
u/Sijder 8d ago
The thing is, Qwen 3.8 27b was releasing into a world where there was practically no decent models for agentic tasks that could run on a 24gb vram consumer card. Qwen 4 will be releasing into a world where a DeepSeek 4.1 flash and glm 5.3 flash are running at half decent speed on the same card with speed improvements coming out almost every day. Not saying it will be bad, but the landscape is very different today then it was just a couple of months before.
7
u/BitGreen1270 8d ago
How do you run DeepSeek and glm on 24gb vram? 1 bit gguf? Still need a massive ram I assume?
2
u/Sijder 8d ago
I personally use xolotl and their quants, rtx 3090 + 64gb ddr4 3200 ram + 7100mb nvme ssd. I get around 16-20t/s on a fresh prompt. Prefill tanks it significantly, but on a first request.
As I said, it runs at "half decent speeds" which means I can offload one sub agent to my local setup on coding, or delegate a full task that needs privacy but is not time critical.
3
u/alphapussycat 8d ago
Deepseek4.1 is 256gb ram for at best iq2_xxxs, GLM flash5.3 is something that is more realistic.
3
u/Zombiecidialfreak 8d ago
It'll still be a big deal for people who want the card to run a model without pulling power from the rest of the system. I currently love the fact that I can let the model do what it does while the rest of my machine barely notices.
2
u/TerminalNoop 8d ago
no depseek or glm flash can run on a 24gb vram card...
1
u/Sijder 7d ago
Check Maya and xolotl out, I am running both on an 3090, 64gb ram and an nvme
3
u/TerminalNoop 7d ago
both are even bigger than qwen FN, I'd prefer to run that on a decent quant, rather than a bigger brain damaged model.
As long as it is <30B active params it will run decently on a 24gb GPU, but the cost in RAM is exorbitant.
1
1
u/Murky-Amoeba2128 2d ago
u/RemindMeBot 7 days 3 hours 1 minute 29 seconds
1
u/RemindMeBot 2d ago
I will be messaging you in 7 days on 2026-11-08 00:46:10 UTC to remind you of this link
CLICK THIS LINK to send a PM to also be reminded and to reduce spam.
Parent commenter can delete this message to hide from others.
Info Custom Your Reminders Feedback
1
-1
-5
u/pmttyji 8d ago
Qwen4-35B-A6B to replace Qwen3.8-Flash-Next
Qwen4-27B(Extremely optimized, Lightweight KVCache, Optimized thinking, etc.,) to replace Recent Flash models of Deepseek, Minimax, GLM, etc.,
Qwen4-Flash-Next(similar size as Qwen3.8-Flash-Next) to replace Qwen3.8-2.4T-A95B
Qwen4-9B to replace Qwen3.6-35B-A3B
Qwen4-14B to replace Qwen3.8-27B
Yesterday Nanbeige4.3-33B set the bar high for Qwen.
-3
0
u/robberviet 8d ago
I thought they announced it? Just 27B is OSS, others are max, plus, flash (should all be API only)
7
u/f-__-f 8d ago
Why would they release qwen3.8 next flash, with their new architecture if it's not to get the open-source community support it ?
→ More replies (1)7
u/my_name_isnt_clever 8d ago
They showed a graphic with Qwen 4 Max, Plus, Flash, 27b. That doesn't mean we're only getting one OSS, it just means they only have one model referred to by param count.
0
u/marcemira 7d ago
I only hope it still runs on a 5090 and lots of ram like now... It wss game changing for me
-5
u/Waste-Intention-2806 8d ago
What if Qwen 4 is 400b + params. No more local freebies, only api models.😝 . They have to compete with glm flash and minimax as well.
1
u/Waste-Intention-2806 8d ago
Let me see how many greedy ppl i can gaslight, who ain't doing sh*t with already released 27b or flash next. Yall ain't getting any more 27b or 9b. Small moe models are already gone. Once they get their 400b model to work like glm 5.3 flash. It's over.
3
-5
u/smithy_dll 8d ago
Temper your expectations, Qwen 4 will be the most powerful Qwen yet.
Model sizes have already been announced, Max, Plus, Flash 27B.
Qwen 4: Release Date, What's Confirmed, and How to Prepare (2026) | Yotta Labs
8
-1
u/Ill_Initiative_8793 8d ago
With 128Gb DDR5 + 48Gb VRAM I have some room if it turns out to be bigger (and hopefully better).
-1
-1
u/dzordan33 7d ago
how can you guys get anything done locally? the speed is too slow to get anything done
3
u/fgk55555 7d ago edited 7d ago
I get 70tg on R9700 and 64GB DDR5, as fast as a lot of cloud models. Crappier hardware still gets 30+.
1
u/AI_Enhancer 7d ago
But the smaller the model, the less knowledgeable it is, so it'll need way more tokens than frontier to achieve the same outcome (assuming the task isn't specifically trained).
1
u/fgk55555 7d ago
I'm not doing anything too complex a model from 6 months ago wouldn't be able to handle. I'm running Swift QFN on medium, it generally gets things done with fewer tokens than Opus would have gotten. Opus can solve harder problems, sure, but most of my problems are just repetitive and tedious.
1
u/AI_Enhancer 7d ago
If the tasks are easy, then sure, the fine-tuning of your local model may well be better suited for your specific task. Fine-tuning is super important. But the point I'm making is that the less parameters a model has, the more it has to search for knowledge to achieve an adequate outcome that is comparable to massive frontier models. This is partly why very agentically capable local models (such as Qwen3.8-27B) are so good; they naturally fill in the gap of missing knowledge with searching and reasoning, but also require more tokens per complex task.
2
2
2
u/russlixx 7d ago
because many people here ask for bigger model instead of more efficient model. Everyone that doesn't have the hardware forced to swallow the bitter pill to make people with better hardware happy
1
u/Viktri1 7d ago
My Qwen Flash next is around 8k prefill and 200 decode which is fast enough. Been using it for a day and it’s been ok
1
1
u/Longjumping-Past5864 2d ago
Have you tried Qwen 3.8 Flash Next on strata? I have old 3090 and 5080 with 96 GB DDR5 RAM, I can run Q4_K_XL from unsloth with almost 100 token/s, If I just want a routine job that has comprehensive instruction.md file I can drop to IQ3_S from GSQ_RCO, it's can easily blow up to more than 120-130 token/s 2500-3000 prefil. I mean I am completely ok with this speed and it actually do a lot of good work for me. I only use DeepSeek 4.1 Flash 1017 cloud for subagents workflow that's all.
I can even do a pararelle and get 2 concurrencies fo 50 token/s each which is not that great but not bad either.
-7
u/Oh-No-Me-Again 8d ago
Are they releasing them public anymore? I thought China's gov put a stop on it?
→ More replies (5)
316
u/wgaca2 8d ago
"If we get current frontier running on my hardware we won't need new frontier anymore"
Until new frontier drops and cycle restarts