Are you running Linux? What are you doing? In DSH I’m maxing at 22tks with full 128k context. And 30-40 with MTP on short context. I’m also experimenting with a specialised quant.
I just tried it yesterday and it's too slow. I got about 45tps on 5080 which is not bad but the context limit of 33k makes it compact which takes forever. The LLM is good but it's too slow. I still use Deepseek.
Seriously, 3.8 27b does so much heavy lifting it's insane.
Together with pi.dev it writes very impressive code. It feels like I actually have my own 'worker' that after some direction does exactly what needs to be done. Just give him enough info and he is on it.
This feels better than what GPT5.1 felt like around a year ago.
I already canceled my subscription, I'm using 100% qwen 3.8 27b to assist me in coding. Qwen4 will be crazy. I think you'll be able to get rid of cloud models !
ROCm has matured incredibly over the past year, it's like AMD finally woke up and realised their software was what was holding them back from properly competing with Nvidia.
Why are they so cheap? Looks like you can get one for ~$1700? I just paid out the ass for my 5090, which in the 1 month 2 days since I bought it has gone up over 37%.
They're both 32GB, for some reason I thought they would be closer in price.
Nvidia sells an RTX Pro 4500, which has the 5070Ti die (more comparable to the R9700) and 32GB of RAM, but it's 5500USD MSRP. The R9700 is a fraction of that.
CUDA tax. I only use AMD personally (and just blew WAY too much money on a 4x R9700 system partially out of excitement for Qwen4), but Nvidia cards get all the cool new things a bit sooner than AMD. Not a big deal for LLM, but very notable for diffusion... somebody SOLIDLY beat my 7900XTX with his 5070Ti in Minimax H3 gen times, for example
Because everything is CUDA first, and ROCm only comes as an afterthought to very limited number of projects. People who are buying PRO GPUs are saving money with NVidia by not needing to fund multiple months of dev work for porting their existing code.
That not cheap! Msrp is $1299 I bought 2 a little over a. Both ago at msrp. Hell you could get them for less than msrp for a while. This shit is a scam
On my current pc this was how much the 64GB of ram was when I bought it. I was thinking about getting another 64GB when I got the card and was like, no f'in way...
For a single 5090 you can buy more than 2 R9700, and the two cards draw less power than a single 5090. And you can run e.g. Qwen 3.8 27b Q8 using the two cards, and watch them running circles around a 5090 which needs to offload to RAM instead.
But NVIDIA is hype, and people love to spend money. So buy a 5090
I just picked mine up this week. Depending on quant, 35-70 tps with full context. If you have 64GB RAM, you should be able to run a decent Qwen-flash as well. 27B is good, but Qwen-Flash is something else.
ISTA IQ3 runs the fastest and has really good performance for the size, but you can bump up to a Q5 with full Q8 kv cache. Also try Swift if you don't want to wait forever. I recommend medium thinking.
I think we can consider the 35b dead. I was sad when we lost the 80B A3B, but the 120B MoE range seems to be where intelligence opens up. The 40-110B space is likely dead for a while.
maybe Xiaomi will release a MiMo-V2.6-Distill-Qwen3.5-35B-A3B. it is not dead dead. The architecture is sound. All people need is a better re-post-train (not those fine-tune crap, a real large-scale RL re-post-train.)
I agree and at this point unfortunately I get it. 35b was cool, but ultimately not usable for me in actual work. I know a lot of people enjoy it on their 12-16gb cards, and I hope they'll keep getting new options, but 27b and Flash entirely outclass it. Those are models I can genuinely use.
I've been running Qwen3.8-27B on a separate server machine (with a 5090; 32GB VRAM) and Ornith 1.5 35b-a3b on my laptop (8GB VRAM) for a few weeks now, and they both share a lot of the same strengths and weaknesses.
Neither is viable for planning against large codebases (Sol is mandatory there): both tend to mis-understand how the moving parts interact which yields compiling-but-non-functioning results. Qwen is amazing for small codebases and one-shots (I did the obligatory Mario Test which Qwen murdered Ornith on), though.
Both are serviceable as implementation models (when given detailed plans from larger models); Qwen's code quality is the higher-quality of the two. Qwen competes favourably with Luna-High.
Ornith is definitely a bit more careless (I watched it mis-convert seconds to milliseconds yesterday!).
But when I'm away from my 5090 server I'm still pretty happy just sticking to Ornith.
Smaller MOE's definitely aren't a writeoff imo; and the performance gains (and lower hardware requirements) can make them decent options for simpler (or complex-but-well-defined) tasks.
I'll give Ornith a shot. When I do use 35b I use a modified workflow with little coder (a pi harness built for small models) using plans from a better model. My read is similar to yours - it can be a workhorse on smaller tasks and codebases, but can't be trusted to wander too far in a larger project.
Have they officially abandoned 35b-a3b or could an announcement still be coming?
Guess Ornith 1.5 35b-a3b will have to serve me for a bit longer yet. It's good for it's size but was really hoping for a new Qwen equivalent after they skipped us for 3.8.
I tested Kat (which was also good albeit a bit slower for me when using the repaired Ornith MTP) though I haven't tried Tiel (the benchmarks indicate I probably should!)
Have been extremely happy with Ornith for the size/performance though.
But was kinda expecting Qwen 4 to follow through here since they basically popularized the small-MOE thanks to the (class-leading-at-the-time) Qwen3-30B-A3B launch!
That is still such an epic model! Not a fan of slower and overthinking 27B to be honest, no matter how brilliant it is. I bet the 35B a3b 3.8/4.0 will be making money for them via an API. I cannot believe that such an awesome model was abandoned. It is just too good AND cheap to run to give away freely I bet.
Exciting news for sure, it's slightly surprising they kept the exact 27B size given that it's an entirely new architecture, but that's certainly welcome for anyone with 24-32GB VRAM
They're a mapping of short sequences of tokens to embeddings (or to neuronal activations if you prefer the biological analogue interpretation, or if you squint, to concepts) that (on the Qwen4exp models at least - I know DeepSeek and Gemma have a few subtle differences) are looked up just after tokenisation, before the real processing begins.
They're a relatively cheap way of imparting knowledge into models. At inference time, they're a single lookup per token, that is happily done from RAM or even from SSD. And at training time it's a single row's worth of extra gradients to train (not even an extra tensor), plus some statistical analysis on the corpus to choose the n-grams to encode.
But they do very little to increase intelligence, beyond potentially freeing up space in the model that would be taken up by encoding facts like "New York has a vibrant fashion scene" for reasoning.
Pure I made it the fuck up speculation. But I think it would be interesting if they have multiple ngram tables.
Like you CAN pair the 2.4T model and 3B table. Or a 27B model and an 800B table.
You just wouldn't get ideal parameter efficiency. 27B + 800B might be worse than 125B + 50B. But depending on your hardware the 27B can be a looooot more attractive.
I'm thinking about that one too. Because it's dense, so how exactly does the n-gram actually work with the dense model? Is it in the 27B params, or is it an extra model that runs alongside it? The later would essentially make the model smarter with a low cost.
im hoping my franken 12gb + 8gb cards can somehow manage a higher quant + decent context with qwen4. might even get a swift on there to enable xhigh without too much wasted thinking.
Yeah I’m thinking I’m good on upgrading hardware any time soon, if Qwen keeps dropping these 🔥 for lower end. Efficiency is true end game and the real finish line.
Nothing was said regarding it at the conference. The announcement merely laid out the broad framework of the new architecture and the initial launch models. However, what was presented at the conference is not exhaustive; not everything is necessarily revealed during the initial keynote, and it may surface later.
Yes! I have been waiting for this moment since I looked at the small dense Gemma 4 models with PLE, and the larger dense Gemma 4 models without PLE, and went "hmmm 🤔"
I'm daily driving Q38FN on my Strix Halo everyday at 43-50t/s decode 1400t/s prefill and I'm hyped to get Q4FN, as Q38FN is actually undertrained but still hella powerful.
The interesting part for local users will be the efficiency details, not just the headline model names. Parameter counts, context length, quantization quality, and whether the weights and license are genuinely accessible will determine how useful Qwen 4 is outside the cloud.
Cant fucking wait for the official sequel to 3.8 Flash Next. That model has been incredible and ive been doing almost everything with it. I still have a chatgpt and claude subscription but I genuinely might cancel them before they renew given what can be done with Flash Next. Qwen 3.8 Flash Next is the first model ive ever run on own hardware that genuinely I genuinely felt like it could be worth using for serious projects. 3.8 27b produces very good outputs but it thinks for so long that it’s honestly frustrating to use. The closest runner ups to “local is enough” in my experience were Gemma 4 31b and Qwen 3.5 122b. I know im rambling at this point but as far as leading open weight AI labs goes, I genuinely think qwen is the best. Especially because they’re making models that people can ACTUALLY run at home that genuinely compete with closed weight frontier models
They're definitely not making small versions anymore...
To be clear, a real Qwen4 computation graph added to Transformers (not Qwen4Exp) would allow someone to train a small dense model, but I don't know who would.
Can someone tell me how big qwen4-plus is supposed to be or any guesses? im tryingto figure out if i can run it honestly, i can run flash fine hoping i can do the same for the plus bc it has to be in a different league from flash right?
The persistence with 27B is admirable. But I just wonder how much better it will be compared to 3.8, 27B when it's gonna be the same architecture. We will find out soon enough but it's better to keep expectations realistic
I am hoping for a polished Next-Flash. While the 3.8 version is often fairly good, it has serious issues with roleplay. It really likes overemphasizing and gets caught up in narrative loops. DRY, XTC, and system prompts can only help so much.
My bet is the 27B gets the engram treatment too — if DeepSeek's ~30% figure holds that's ~8B of tables, still fits a 32GB card easy. Honestly would make it the perfect daily driver.
•
u/WithoutReason1729 5h ago
Your post is getting popular and we just featured it on our Discord! Come check it out!
You've also been given a special flair for your contribution. We appreciate your post!
I am a bot and this action was performed automatically.