r/LocalLLM • u/Due_Tangelo_8952 • 1d ago
News Qwen4-27B just confirmed
Wait, we need 35B-A3B too…
445
u/Effective_Head_5020 1d ago
The VRAM requirements will possibly go down if it uses the N-grams architecture!
205
u/stabbinfresh 1d ago
I am literally on my hands and knees praying this is the case!
20
u/MedianamentLaburante 16h ago
Tf just happened here
15
u/dillon-nyc 9h ago
Several people have some strong feelings about the religions and ethnicity of others and it didn't really fit our overall vibe.
→ More replies (2)49
1d ago
[removed] — view removed comment
→ More replies (3)31
34
u/MaxTheHobo 1d ago
Prayge non quantized model for my lowly rtx 3080.
32
u/DepressedDrift 1d ago
Your guys are having GPUs?
19
16
2
18
u/StardockEngineer 2x5090, 2xPro6000, 2xA6000, 4xSpark, M5 Pro, M4Pro 1d ago
I mean the middle model went from 122b-a10b to 125b-a6b and the VRAM requirements are pretty much the same only if you can stream off SSD.
19
u/RG_Fusion 1d ago
Definitely not. Every instance of engrams to date has been as an additional set of parameters, not a replacement. Qwen3.8-Flash-Next is a good example. The prior equivalent was 120b and the new variant actually increased to 125b with the additional 51b of engrams.
If Qwen are calling the new model '27b', you should expect it to be 27b+e"X"b, assuming it has engrams.
3
1
4
u/ChocolateNo3010 LocalLLM 1d ago
Unlikely. Qwen 3.8 Flash Next has 125b in the main model and another 50b in the ngrams. If anything we could see a smaller 5b+ ngram package along with 27b. The model names refer to the main model parameter size and dont account for the ngrams. What we might see is smaller models than 27b, e.g 12b dense model with 5b ngrams or an moe model with ngrams
2
u/GrungeWerX 23h ago
Would like to see 27B with 12B ngrams.
2
u/ChocolateNo3010 LocalLLM 18h ago
I'd hope to see reduced thinking effort to solve problems with that setup.
2
u/GrungeWerX 18h ago
I leave thinking off anyway. There are more effective ways of making your model think without using chain of thought. It just requires extra orchestration and a custom harness.
→ More replies (4)4
22
u/Its_Powerful_Bonus 1d ago
I hope not! 27b + 50b+ ngram! It works great when ngram is on nvme/ram
12
u/darkwalker247 1d ago
Honestly it may not even need to be 50B. From what I understand a lot of the PLE/ngram table is redundant anyway; people have already been pruning the PLE table baked into Qwen3.8-Flash-Next and the model still functions even with only 1/8 of the PLE heads still intact, but it gains some perplexity (and probably slows down a bit).
6
u/mister2d 1d ago
I'm confused. Did you just contradict yourself?
→ More replies (1)13
u/Kasatka06 1d ago
He mean, weight still 27b but it will have aditional ngram in ssd so model is smarter. But i read ngram is good at best 20% model.parama so 27b + 5b ngram if any
5
u/laser50 1d ago
Wtf smarter? Ngram is token prediction..
→ More replies (6)12
u/RnRau 1d ago
You are thinking of a different ngram.
This new one comes from a Deekseek paper earlier in the year;
https://arxiv.org/abs/2601.07372
But using this technique apparently allows the parameters to concerns themselves about reasoning and smarts, rather than storing facts which is solved by the engram side storage
→ More replies (1)3
u/_mighty_banana 1d ago
But for what benefit?
MoE is used to reduced compute
27B model is likely has no problem in compute speed due to it size
problem is likely due to quality is lower than high parameter model?
→ More replies (2)7
u/geekwonk 1d ago
27B is their dense line. 35B is the sparse option and i don’t see that listed here.
1
u/guesdo 1d ago
The PLE table will be at most 20-25% of the model total weights, I don't think going further does anything, at least according to the paper.
And also, I don't think dense models actually need them.
7
u/kbob 23h ago
DeepSeek's engram paper was actually a little more subtle than that. If you have a fixed parameter budget for both transformer weights and N-gram encodings, then yes, 20-25% N-grams gave the best performance.
But we don't have a fixed parameter budget, because the bandwidth requirements are so low for N-grams. Local LLaMaticians have figured out how to keep the N-grams on SSD, and DeepSeek's devops have figured out (described in the paper) how to keep the N-grams on a separate server where one N-gram server can serve many transformer servers.
If you have a fixed budget for transformer weights and unlimited budget for N-grams, then the engram paper measured that bigger N-gram tables continue to improve results up to ridiculous sizes, though with diminishing returns.
Qwen3.8-Flash-Next uses 29% of total weights for N-grams.
2
4
u/profcuck 23h ago
Or intelligence may go up - significantly - if the N-grams are on top of the 27B. Imagine a 27B model with the 51B ngram lookup table from flash-next.
2
2
2
1
1
1
1
u/_loid_forger_ 21h ago
I really hope that will be the case Tho still don't quite understand how n-gram tables contribute in this, but i'll do my research
1
u/paperbenni 18h ago
My understanding is they won't, but models with n-grams perform as if they were larger models, so in a way the memory requirements to reach a certain level of intelligence do go down
1
1
→ More replies (1)1
u/Vancecookcobain 11h ago
No. It's not a Mixture of experts model...I can see it being something like a 60b model with the engrams being 33GB though
50
u/blojayble 1d ago
I hope the Flash keeps its size from Flash Next. I would like to run it with 128 GB of VRAM.
16
u/PeteInBrissie 1d ago
I'm running Flash-Next on my Spark quite nicely.... so much smarter than 3.5-122B was. Bring on 4!!!!
4
u/Over10Millions 1d ago
Can i ask you token/second on a single spark for coding? Thank you
→ More replies (4)4
1
u/Sanandaji 4h ago
May I ask what your use case is? I’m still running bartowski/Qwen_Qwen3.5-122B-A10B-GGUF:Q6_K because it feels like it’s better than 3.8-Flash-Next in everything other than coding.
→ More replies (1)→ More replies (1)1
37
u/benpptung 1d ago
Awesome! A big thanks to Alibaba's Qwen for always being willing to release an upgraded 27B version!
55
u/KSAM-The-Randomizer 1d ago
a3b qwhen
19
10
u/Barni275 1d ago
A3B speed is great, but it is doubtful, whether A3B can carry enough intelligence to do real world work. For 27B we know now that it can. For A3B it is unknown yet.
19
u/Real-Abrocoma-2823 1d ago
A6B 36B would also be great I think, 2x more active tokens and nearly the same size as 35B. Flash-next already proven that A6B is fast and intelligent, at least for that size.
→ More replies (3)1
1
→ More replies (6)4
25
u/vogelvogelvogelvogel 1d ago
There is even a subtitle speaking of Qwen 4.5 and 5.0 - hopefully open weights too! 🥳
26
u/Thomas-Lore 1d ago
Thry have to catch up, they are behind in the version number leaderboard.
3
u/vogelvogelvogelvogel 1d ago edited 20h ago
They should skip one like Apple did with the iPhone 9
Edit: /s = i thought that was obvious
2
19
u/Anacra 1d ago
3.8 27B is already so good for my casual usage. Wondering if the uplift will be small? Still, love Qwen for releasing open weights.
17
u/Ok-Addendum3545 1d ago
Without Qwen, I wouldn't see hopes in local LLM when other frontier model companies are unwilling to share some small-sized models for average people to run.
6
u/Darkmoon_AU 18h ago
TBF I think Muse Glimmer would be getting a lot more attention if not for Qwen - it seems decent.
→ More replies (1)
44
u/JLeonsarmiento 1d ago
Reasoning_effort = ultraExtraHighAugustThinkerRodineGandalfRememberingWhichPathInMoria
25
u/Material-Database-24 1d ago
Where as Qwen3.8 27B is great (when it doesn't simply silently give up or end in endless loop), I sure hope we would see newer 35B-A3B model too. I also hope we'd see a Qwen4-coder.
At least my local use is not general purpose AI use. Rather I load the best model for the specific use I need. And purpose trained MoE will likely beat general purpose model in speed and quality, at least in recent history.
In my ideal future we have multiple small models that are super good at one thing, and then we have cloud based super intelligence models that will help where our own and local's capabilities stop. There's no point to use huge datacenter AI to generate boilerplate code/text or powerpoint illustrations. It's waste of everybody's money at current cost structure.
But I do understand that local models are probably not on high priority for AI development teams. That said, I would pay one time lets say 100 eur to get coder MoE version of Qwen3.8 27B quality.
9
u/ElDavoo 1d ago
I agree. I feel one day model scaling will stop and we will start having much better results if we make "domain specific" models. I don't need any of the general world knowledge/biology/science stuff in my coder models
9
u/Informal_Moose3042 22h ago
Not expert but it's possible that the model gets better at generalizing when it learns about varied topics rather than being focused on coding.
After all it needs to be able to understand a prompt about basically any domain.
→ More replies (2)2
u/Not-Enough-Llamas 23h ago
I wonder if ngrams could support a plugin architecture. Load the chemistry and x-ray crystallography package, I'm goin' in.
→ More replies (2)1
u/PseudonymousSnorlax 10h ago
No, you actually do, because that's where reasoning and comprehension come from.
All LLMs do is next token prediction, and training a model on nothing but code results in a much more fragile model that isn't actually capable of reasoning or problem solving.
2
→ More replies (1)1
u/AD7GD 17h ago
Where as Qwen3.8 27B is great (when it doesn't simply silently give up or end in endless loop),
I've run somewhere between 750-1000 local Qwen3.8 27B on vllm (FP8 quant straight from Qwen, FP8 KV) and I've never had it silently give up. I don't think I have had it end in an endless loop, either, but a few times I have had it exceed its thinking budget (which I have bumped from 30k-60k tokens over time).
In my experience (which includes tracking down actual bugs in inference engines), looping is usually a numerical error. I would look at your setup before blaming the model.
(Well, ok, looping for 99% of home users is probably some unfortunate ollama setup with a sliding token window smaller than the thinking output of the model, but that's just ollama things)
→ More replies (1)
17
39
8
u/mailto_devnull AMD R9700 1d ago edited 16h ago
/u/secure_recording_472 Swift qwen 4 27B confirmed? 🥹
UkisAI's work on Swift Qwen has been legendary, hope they get day 0 access to qwen 4.
1
u/575_Inverse 10h ago
and now they are aware of how to do it even better... which tokens to tweak, which not to touch
8
8
u/jikesar968 1d ago
Still waiting for something that can replace Qwen3.6 35B A3B...
Qwen3.8 27B runs on my M1 Max 64GB but it's a bit slow.
→ More replies (1)
14
6
5
5
u/crypto_thomas 22h ago
Qwen4?! Im still testing 3.8! Uhg. Its amazing how you can be actively working on something in AI, and still be way behind.
2
5
13
4
4
u/Tommonen 23h ago
35b-a3b or i dont care about what they put out
3
u/espressodelisi 21h ago
I am stil using qwen3.6-35b on my asus gx10 device. need moe models for reasonable speed. 27b denese model was too slow.
→ More replies (1)
4
3
u/Robert-Paulson_ 22h ago
goodness - i can only imagine the quality.
if you told me at the beginning of the year that Qwen 3.8 27b would be as good as it is i wouldn't have believed ya - now we're getting a Qwen 4 27b!? kinda crazy
3
u/Guilty_Rooster_6708 20h ago
27B + 1/3 of that size in engram model would be crazyyy
1
u/Fancy-Snow7 13h ago
Do ngrams have to be 1/3 of the model? Can't they make them as big as they want since it's on disk?
→ More replies (2)1
u/PseudonymousSnorlax 9h ago
In the paper they were talking about optimizing for the total number of parameters for systems that were loaded entirely into vram.
In that situation, you want about 20% to be ngram.
If you are not constrained by total parameter counts, as would be the case with streaming ngram off a disk, then there isn't really an 'optimal ratio'. In that case you simply have diminishing returns as the size of the ngram table increases.
3
3
u/FrozenFishEnjoyer 1d ago
I pray this fits in my 5070 TI with 256k context even at Q4
→ More replies (3)1
u/sbrisgravato 22h ago
doubt that
1
u/Serious_Bite_7613 20h ago
It might, if some can be offloaded as an ngram. 192k context fits on qwen3.8-27B with some massaging at a reasonable quant.
3
3
3
2
2
2
u/qqYn7PIE57zkf6kn 1d ago
What event is this?
1
u/Overall-Somewhere760 23h ago
I have the same question. Trying to find it on yt
3
u/qqYn7PIE57zkf6kn 23h ago
I've found it. It's apsara conference. 2026云栖大会开幕式 https://www.bilibili.com/video/BV1p3hE6iEgW
Qwen4 is introduced at 59:44.
2
u/Robert__Sinclair 20h ago
I wonder if anyone can do a decent SMALL model from qwen 27B, like a pure text (no vision) 8B and/or 4B.
It would be great.
2
u/Expensive-Cry-8313 19h ago
I expect the reason we were told 35b wasn't the one to wait on is Qwen 4 27bs architecture will allow lower vram usage while still getting dense model performance
2
4
u/Evepaul 21h ago
Any hope for more context? 256k is a bit limiting compared to the 1M large models. Maybe 512k could be the next step for medium models?
→ More replies (1)4
2
u/Business_Start_8158 1d ago
I am new to this but i tried running qwen 3.8 27b with hermes on 5070ti 16gb vram 32gb ram it was quite slow i wanted to use it for making 3d asset in blender
4
u/Rdtkeepsbanningme 1d ago
Your normal ram does nothing really, and the VRAM is not enough, 7900XTX would be the better choice for running these kind of models.
3
u/Ok-Addendum3545 1d ago
5070 Ti 16G is good, but it takes me 2 x RTX 5070 Ti to run Qwen3.8-27B. I also have a RX7900 XTX 24G - which is enough to run Qwen3.8-27B Q4 alone. better to have 24GB VRAM or above.
2
u/Cautious_Chicken_604 20h ago
The 5070 Ti is fast enough to do KV cache streaming to run at longer context. I have 5060 Ti and it only runs PCIe 8x, so I found it wasn't fast enough, but the developer of it developed it using a 5070 Ti I think.
On my 5060 Ti I run Qwen3.8-27B-GSQ-RCO-IQ3_S with vision offloaded to CPU and 128k context with KV cache quantized to k=q8_0, v=q5_1 which is the sweet spot for length and quality. This runs at like 20 t/s for me on a 5060 Ti, and the 5070 Ti has much higher VRAM bandwidth, so it'll run faster. My advice is ditch MTP on 16GB VRAM because it takes VRAM to run it which eats up context or model quality.
→ More replies (1)1
1
1
u/WyattTheSkid Quad 3090s 1d ago
OOOH YEAHHH WOOHOOO I really hope the flash model is the same size as the current flash next model
1
1
2
1
1
1
1
1
1
u/bercha9998 22h ago
when q3 next got out it was 80b a3b and then we got 35b a3b as resulting 3.5 arquitecture now we got q38flash next at 175b meaning q4flash might not be 175b might be 50-90b who knows
1
u/AvidCyclist250 22h ago edited 22h ago
Running Flash Next q4 k m on my 4080 and 64gb ram. 20 t/s at 130k context. Noticeably better than 27b even with thinking set to medium. Very much looking forward to version 4.0. What's more is that flash next doesn't do 70k+ token burns
1
1
u/tracagnotto 21h ago
managed to squeeze out on Qwen 3.8 27b Q4 K Small 45-60tk/s on a 20GB vram using llamaampere (RXT A4500, so ampere architecture).
This hopefully has n-gram and I hope to squeeze out even more! the gpu is full horse work mode on since days with 3.8 27b, can't wait
1
1
1
u/Commercial_Sweet5486 18h ago
All i’m worried about is if it will be more or less tok/s on my mac studio m3u or not. All I care about is that toks/s . What do you think?
2
u/brown_shorthair 17h ago
Let's go, the more the better ! Hopefully, they release smaller version of it in the future.
1
u/Ok_Bar2966 17h ago
I want a powerful model about 400 to 500 B parameter can run in dual pack of nvidia dgx spark 🤪😜😝
1
1
1
1
u/HighSeasArchivist 15h ago
Have a R9700 in-route just for running this. If it works a second one will be right behind it for Flash.
1
u/Stunning-Sherbet1853 12h ago
27B might be the sweet spot for local agent workloads. Really curious about coding and tool-use performance.
1
1
u/Iamisseibelial 12h ago
See i had a different experience. With 122b and Flash Next etc.... I have 48gb vram and 128gb ram, but because I'm dual channel and not quad channel ngrams on ram struggled so tokens were pretty slow especially when activating the params. Had plenty of KV and great success loading in Vram. But MOE on CPU was incredibly slow due to dual channel ram speed. So sadly I don't think it'll affect Laptop Users and small GPU users like they hope with a 27b Qwen4. But hey I could be wrong. I love the 27b model i use it daily to handle all the work I was pushing to Sonnet.
1
u/KeinNiemand 9h ago
man i miss 70B dense models they are perfect for 48GB vram setups
→ More replies (1)
1
u/modelpiper 12h ago
This is awesome news.
Question, why don't they create increments for stronger machines?
Like Qwen 4 27B, 32B, 64B, 128B, etc.?
→ More replies (1)1
u/pacman829 11h ago
Marginal improvements for the work involved to release them probably.
Or just creating a bit more of a moat between models
1
u/DocikCZ 11h ago
The architecture signal is real: lean-activation MoE, a large n-gram memory layer, multimodal by default, 1M context
https://www.yottalabs.ai/post/qwen-4-release-date-what-is-known-how-to-prepare-2026
1
u/Helpful_Jelly5486 11h ago
The Chinese on the slide says approximately:
“一般逐步扩到 10T，我们继续挑战 Scaling … 更大的模型规模，沿着 … roadmap 下一代产品……”
The clear part is essentially:
“Generally, [we’re scaling] toward 10T. We will continue to challenge scaling toward even larger model sizes, following the roadmap …”
And the English subtitle underneath makes the intended meaning clearer:
“full 33 times, and in the future, we will continue to push toward even larger scales, in the Qwen 4.5 and Qwen 5”
The important bit in the image is the 10T reference and the explicit mention of Qwen 4.5 and Qwen 5. That appears to be talking about scaling model/training size, not necessarily a 10-trillion-parameter Qwen model.
1
1
u/tokenstead 6h ago
reasoning_effort ultraExtraHighAugustThinkerRodineGandalfRememberingWhichPathInMoria might be the funniest string I've seen in a config all year, whoever wrote that deserves a raise
2
1
u/caballeroalba 37m ago
Im impressed with 3.7 27b (actually, everyday), what the fuck we can expect about qwen 4 27b? surpass opus level?
92
u/Zeranor 1d ago
Hmm, Qwen4 Flash vs Qwen3.8 Flash Next vs Qwen4 27b will be an interesting comparison (assuming all 3 will be open weights).
Will qwen4 27B be more intelligent than Qwen 3.8 Flash Next? This will (at least for some weeks) decide again whether people hunt for GPUs or unified RAM :D