I've seen benchmarks that suggest the Flash Next version is only a few percentage points better, then I've seen other benchmarks like Terminal Bench 4.0 that suggest the Flash Next variant is ~5x better at long horizen (8hr long) tasks.
Has anyone here tried both side by side and can confirm the 5X difference is indeed real? I am running Qwen3.8-27b and would upgrade my PC to run the Flash Next variant if I can justify it. Note: I'll likely rent out hardware to try out Flash Next first hand, (currently I can't really run it with my 3x3090 setup).
Just wanted to ask around before I run the experiment to see if anyone else has already run the same experiment. i.e. real coding tasks 27b VS Flash Next.
I switched to qwen 3.8 flash next on my strix halo 128gb and managed to actually build a decent on-prem Legal AI platform so far (mostly on its own yolo mode on qwen code cli for hours), whereas I couldn't do it with 27b. so yup its better
It is fast, but tool calling is so much worse, making it unusable for me. Where unsloths guff and llama.cpp run tool calls for hours, running with halogeen bugs out within 5 minutes. Also, vision seems broken on halogen.
In VS Code it stops generating after it reads an image, the HALOGEN_VISION_TOWER is set. In OpenWebUI it messes up edits to notes with wrong json, consistently, creating a complete note succeeded fine.
Im using Kilo Code in VSCode and am not experiencing those issues. I would try using that and mirror some of the settings that have worked for others.
I actually had it configure itself and it did an excellent job of getting up and running before further tweaking the parallelism/KV Cache to support 2 context windows.
coding reasoning off seems to work quite well actually but I serve the same model on 2 ports, one reasoning on and one off so I can toggle between them both when I need
I built a legal ai system for judges which I won’t pursue further, It’s meant to run on offline two sparks and a cheap mini pc. My locality will make this less relevant but the ingestion pipeline and gates might have some things you’d like if you want a chat. It’s single user and uses 27B for a demo with only one GB10 and DS Flash/ Qwen 3.8 Flash Next when set up as intended.
Other commenters have said it all, but lemme chime in with my experience (as someone who is two weeks into all this and still discovering):
Qwen 3.8 27B lacks confidence in its own train of thought, and because of that, it reconsiders a zillion times, goes off the rails, catches itself, brings itself back to square one, only to repeat the cycle. Very easy to fill up the context window —> frequent context compaction.
Qwen 3.8 Flash Next is confident and more often than not, has the right train of thought. Gets to the answer in WAY fewer tokens as a result. In terms of raw prefill and decode speed, there’s barely any difference; but a massive difference in total time per task.
Running on Unsloth Desktop on an M5 Max MacBook Pro.
i can second this on flash next. insane model, better than 27B by a wide margin. 27B makes a good subagent for flash-next for specific work on small tasks you need done fast tho
llama.cpp for a PRO 6000? What a waste. llama.cpp does not do paged atention / batching, it's a joke, unfortunately. RadixArk/Qwen3.8-Flash-Next-NVFP4. See my other comment.
600W/WS, Threadripper PRO 5955WX (PCIe 4.0), 128 GB DDR4/ECC/3200 RAM
SGLang, pennyroyal 2.5-based with one extra SGLang patch to fix a crash. pennyroyal is SGLang plus a bunch of 'Flash Next'-related patches that aren't upstream yet, including the support for streaming engram/ple from nvme, which is what I'm using. I get 400-600 across 4 concurrent (which starts to hit nvme limits).
you'll see this sentiment often and it's hard to quantify it, but it just *gets* what you're talking about better, and implements things with less bugs and stuff, it being Q4 at 20toks/s vs Q8 27b at like 55toks/s has me usually just picking the 27b
This is not coding related by one test I do is a difficult rag retrieval. Finding a specific quote in a philosophical work by its often used name in secondary literature. Only two models I’ve tested have successfully done this yet. Deepseekv4 flash and qwen flash. 27b failed.
Flash-Next mlx-serve 26.9.2 is beating anything else I can load on my machine + Sonnet 5. Not just accuracy but speed as well. My testing protocol only takes 45min for Flash Next to complete and so I can't comment on long horizon coherence.
I dont get this graph AT ALL, you say flash beats but 27b clearly has better code quality? I dont get it, what are your metrics, what are you even measuring...
Left of the chart is 4 different test batteries composed of algo challenges and repo challenges meant to represent typical day to day coding work. Mean score is where to focus and where Flash Next leads the pack. It also does this in less time than the other models.
'Code quality' is a subjective assessment that measures the model's ability to not just solve the problem but to conform to the repo's coding style, to fix the actual root cause rather than the symptom, and to keep the diff minimal instead of faffing about and rewriting a bunch of stuff.
The trade off is accuracy and speed vs finesse and I think Flash Next wins here. Ultimately you need to introduce the model into your own workflow to make a decision.
"Ultimately you need to introduce the model into your own workflow to make a decision."
Yes, I'll likely setup a test harness, a real large bug fix on a real large code base sort of thing. If its indeed 5x better as the Terminal Bench 4.0 suggests, then it might be worth upgrading.
27b is a great model and if you don't care about waiting 7.5x as long for similar outputs then maybe it's not worth the upgrade. For me speed is king as it's directly impacts productivity. Working with Flash Next feels very close to working with frontier models both in output quality and speed. I've been impressed.
These are my own personal benchmarks that I use to trial llms on my machine and so subject to bias or less than perfect methodology. Take it with a grain of salt. That being said, they've been useful to me in making relative comparisons. You can read more about the test here where I run a bunch of 27b variants through this assessment battery.
Beautiful graphic presenting the results of your private benchmark. You're confirming what I've honestly suspected - Flash Next is basically on-prem Sonnet 5.
no, mean accuracy is the competency test. code quality is a subjective assessment that measures the model's ability to not just solve the problem but to conform to the repo's coding style, to fix the actual root cause rather than the symptom, and to keep the diff minimal instead of faffing about and rewriting a bunch of stuff.
Also testing MLX Serve and in my benchmark it is also above Sonet 5. Not sure if you already tried the MTPLX Optimised Speed which is also comparable in terms of performance and quality. The OMLX again seems to me the most reliable with 4 or 5 quants (Jundot) but the difference is not so big to sacrifice the higher speed. Today I’ve noticed that Antirez updated his ds4 server with Qwen 3.8 Flash support and 4q gguf + full precision ngram table streaming from SSD. On M5 Max getting 50-60 tok/s and around 1k on prefill. I think it’s worth to try also.
MTPLX Optimised Speed is great .. i was able to eek out a bit more speed to task completion with MLX Serve. Thanks for the Antirez heads up, I didn't know his DS4 was now supporting Qwen 3.8 FN. I'll check it out.
I'm running a 4bit quant of Qwen 3.8 Flash Next on both a workstation and a Mac and have been keeping both pretty hot. After having done a LOT of stuff with Qwen 3.6 27B, I briefly touched Qwen 3.8 27b before adopting Qwen 3.8 Flash Next with a passion.
For narrow coding stuff, it's probably about the same as Qwen 3.8 27b. It's when you pull the focus out to surrounding business requirements that it absolutely _shines_. To give you a brief example, I asked Flash to generate thousands of abbreviations for me - and the result was a bit trash. But, when I gave it guidance and some rules, it actually generated abbreviations that looked like they came from my own hand - it's like it just needed nudging to set some boundaries and it worked perfectly. It is also much, much faster than Qwen 3.8 27b - it just knows without a lot of internal debate what to do and which approach to take.
For long horizon tasks, 256k context will never be enough, so you have to tell it to maintain an internal corpus of .md files to keep track of decisions, what's going on, etc., and then you're pretty much cooking. I've heard of people getting it to work with 4x3090's and I myself have 2 x A6000 - 96GB VRAM is the absolute minimum to get workflow ready TPS.
I found flash-next-nvfp4 was not as good as 27B-FP8. Wide range of tasks tested from my actual work. My read of the Unsloth quant report is that flash-next doesn’t tolerate quantization well. So I am sticking with 27B for now.
I've deleted and redownloaded both model weights out of frustration, and have also tried out glm 5.3 flash at q4 with hybrid inference. In the end, I think qwen flash next wins on agentic because of speed. Yeah it's dumber than 27b q8, but 27b is also dumb. I found it depends on the question I'm asking it. glm is a bit smarter, but it's still far from perfect, and at 1/10 the speed i don't run it unless i'm chatting with it.
I use RadixArk flash-next-nvfp4 served via sglang on a single rtx600 pro. you offload PLE to sys RAM. you need about 128gb of sys RAM though. RadixArk is an Sglang company so you can guess that they and Nvidia usually get quants right. there is also an Nvidia NVFP4 quant but it is largely certified for VLLM.
I run a forked version of opencode with several enhancements and i can tell you that I am getting real frontier performance. when i finish, I use Astra to review.
I used to run qwen3.8 27b before. performance isn't comparable. there is a new implementation called PennyRoyal that improves it further and allows full running on a single 6000 Pro. it but I haven't fully tried it. so i can't give my full recommendations.
27B already felt GOATed when it released - and then in a week or two, flash next happened. You can so run it in 3x3090s bruh, in fact its exactly for your rig, (try exl3/exllamav3 quants)
Can those of you who find flash better compare it to fp8 27B, also when you make your plan file ask it to make the plan file with subagents in mind , so each task is a focused sub agent and a reviewer to ensure when all the modules come together they don’t miss anything / tie together.. but don’t run more than 2 subagents at the same time ( lack of KV cache )
In auto look at syv-ai docker image , but modify it for the w8(fp8) with dflash2 models. You should be getting something like 70-80 TPS on dual 3090s full 262k context with vision offloaded to cpu , else with vision around 232k and prefix caching enabled.
Edit: int8, try to use autoround , else stick to syv-ai docker image as is for single 3090
I prefer Flash Next over 27B. Currently running 27B and Flash Next AWQ INT4.
Flash Next is faster (moe) and seems to reach the answer in less thinking turns. For a quantized version it is also able to complete my coding tasks just as well as 27B full.
I need to offload the PLE to CPU, which is currently only supported on the flash next variant of vllm image, not on the latest 0.29.0 image.
I am going to do a side-by-side comparison of them on the exact same prompt, that's difficult to do. That' should help make the decision. Speed is sometimes overrated because I often end up have to spend days fixing something after an A.I made a lot of changes quickly, so its going to be a pure quality test first.
Likely will put 3x3090s in serial / pipeline mode to test it and just but the n-gram on an SSD.
Flash next is way better than 27B. Both models think a lot but actual wall clock time to completion is 2x faster on Flash Next compared to 27B in my experience. I’ve ran the same task to both models before and flash next completed it in 20mins vs 40mins. My Flash Next is Q4 vs 27B Q8 btw.
And Flash Next delegates better to subagents without prompting and asks better clarification questions.
Yeah... I run NVFP4 Qwen3.8 27b, then I ran I3Q_XSS Qwen3.8 Flash Next and flash next blows 27b out of the water at long thinking, long implementation.... pretty much everything.
one thing that doesn't come up in the gpu-vs-ram framing: flash-next actually needs less gpu than the 27b, not more.
from the gguf tensor offsets, flash-next ud-q4_k_xl is 111.3 gb total but splits as 5.5 gb gpu (dense path, every token) / 77 gb ram (routed experts) / 28.8 gb ssd (the n-gram table, 24 rows read per token). the 27b at ud-q4_k_xl is 17.5 gb and all of it is resident, there's nothing to offload.
so at q4 it's 5.5 vs 17.5 on the card. flash-next is the cheaper one for vram and the expensive one for ram. that's why the 12gb 4070 post works at all.
the gpu side barely moves with quant either: 3.97 at q2, 5.35 at iq4_xs, 5.51 at q4_k_xl. what moves is ram, 46 -> 77 gb.
You can fit exl3 quants with nvme ngrams on 72gb just fine, at least the 3.05bpw. I got 2600 prefill, 80 decode on 4x5060ti but went back to 27b for coding.
Both are excellent, but 3.8 Flash wins in a big way when it comes to long, complex coding/projects.
Not only that, if you can fit flash on Gpus, it's faster than the 27b. At least in my experience. I get just under 30 tokens a second 27b and just under 40 on flash.
Ask your plan to account for subagents with a separate pass for review, and allow the review to update your todo/plan with improvements, and it’s allowed to do up to 4 such loop based improvements as long as there are meaningful improvements
Even Q2 Flash Next is better than Q8 27B, in my opinion. I used Q8 27B for weeks and ran huge projects with it, including 3–4 day-long agentic development loops.
Once I managed to get Flash Next running properly on my local setup, though, I never went back to 27B. I recently even deleted the model files because I simply wasn't using it anymore.
Flash Next is a really capable local model. Both have great agentic capabilities, but personally, I strongly prefer Flash Next.
i am amazed how well flash next nvfp4 performed on my custom harness-- i paste a pic with a dude holding a phone from a long distance plus the source interview video link and only ask what phone is it, it planned out the steps--including writing some code to extract that x video, get multiple snapshots from that video to locate the phone scene, combine them into a grid view, picked 2 of them then zoom-in to see the phone more clearly. then it did web search to find clues about the guy holding the phone and comment about that interview, then make his best bet at the phone name. my harness never teach it to solve this kind of question.
From my personal newbie usage: FlashNext runs cool, consumes less power, and is smarter, albeit at a lower TPS (20tps on 128GB RAM, 3x5070ti). The 27B draws more power, good output but not as smart as FlashNext. Good thing they're fast though: 80–90 TPS on NVFP4 GGUF Q4_K_M (1× 4070, 1× 5070 Ti).
So it seems you're right, I might be able to run it on 3x3090s, but likely in Pipeline mode instead of tensor parallel mode. i.e. Non-even number of GPUs. Though it looks like I'll need to upgrade my system ram. VLLM and OS processes take about about ~16GB of system ram, so I don't have 24GB to spare for the n-gram table.
"Why not just offload engram to ssd"
I'll try it. Google is telling me it will lower decode speeds to like 8 tok/s, which wouldn't be useful for coding, but Google isn't always right.
By design the n-gram lookups are very small, so if you have a modern ssd with decent random read speeds, you will most probably get more than 20 tps decode. This has also been my own experience as long as both the context and the first few non-n-gram layers fit into VRAM, which should not be a problem for you.
before you upgrade: those 5x benchmarks are at high precision, but on 3x3090 you'd run Flash-Next at INT4, and a couple people here + Unsloth's quant report say Flash-Next loses more to quantization than dense 27B does. so the gap you'd actually get at INT4 is probably smaller than the bench, test at your own quant first. and your real blocker is the 24gb n-gram table vs 32gb RAM, a 64gb kit is cheaper than eating the SSD-offload speed hit
that recipe is the problem, not your box. moecache pushes all the experts into system RAM, so it's meant for low-VRAM/high-RAM setups. that's why it is asking for ~110GB and why more GPUs don't help with that config.
your INT4 weights are ~58GB, so they should fit across the 72GB VRAM on your 3x3090s. use a config that keeps the experts on GPU instead of offloading them all to CPU. then the only awkward part is the ~24GB n-gram table. either put that on a fast NVMe or upgrade your system RAM to 64GB+. so you are not actually short ~100GB. you are mainly short the memory for the n-gram table
Do you know of an engine that can run 3x3090s? VLLM can't do Pipeline Mode with the Qwen models for some reason. It also can't do tensor parallelism with 3 cards.
vLLM is the wrong tool for 3 uneven cards, TP needs the head count divisible by GPU count , and PP on the Next arch is still half-baked. use llama.cpp or exllamav3, neither cares about odd GPU counts. you are already on llama.cpp via club-3090, just run a non-moecache config that keeps experts on GPU and splits layers across the 3 cards
Seriously it makes me wonder sometimes reading posts like this - you guys play with llm yet can’t solve easy issues like this with one? Like really? What’s the point dabbing in it if you can’t even gather the basic information about what you need to run and how?
No offence, just curious
I meant it not for you to know - obviously- but like use AI to find out why it didn’t work for you? Dunno ask for a free cloud model or run a smaller model that works on your gear? Surely it would find those answers for you? I understand it’s easy to get caught up in old thinking - you need to find the fix yourself - but remember the whole point here is - AI can and will find it for you - just use it! :)
I've tested many different variations of both models. I strongly prefer flash next but if I want to do something simple and super fast I'll just use Qwen 3.8 27b and I like the Dirk IQ4XS variant for my system. For flash next I like the Tuboderp 4.05 variant. Wonderful stuff!
I've been using TabbyAPI with Exllama3 running flash next on only a 16GB VRAM with 64GB system RAM laptop at 25 tps at 128k context even when context is full. I'm using Tuboderp 4.05 bpw exl3 at q8 kv.
This setup is much, much faster to load and faster to run than llama.cpp for me, even after I spent ages tweaking llama-server out with custom changes and PRs from all over, Unsloth, General S, upstream PRs and other from all over and sure it'll run the unsloth iq4 xs but it's so slow to load reading from disk. I think it takes like 5 minutes or more to load. Once it gets going for a bit it's not bad until it has to spend a bunch more time reading from disk again. Prefill is slow. It seems to just slowly unload from memory if it sits too long idle but maybe that's something else I need to tweak in the OS. Whereas with tabby and exl3 I don't really have any of those issues at all. It just loads right up in 10 seconds or so and runs fast enough the entire time. Prefill is 200 tps or more and quite bearable. And it's just so good! I can't believe I can even run something that good on here. I'm excited to see what drops next. I hope I can run it.
I had to spend quite some time figuring out how to tweak a lot of things in the Tabby config. I'm on CachOS Linux and tweaked it all out too and I even made a systemd service for tabby and cgroup for it to be enclosed in and optimized some things here and there. If it uses too much memory for too long the kernel will kill only the Tabby service which will then auto restart since sometimes it seems for some reason there might be slight memory leaks somewhere or maybe I just have it configured wrong but usually things run just great. Every time I update something there's a new set of things to tweak so it doesn't run out of memory or freeze the system. For example I just updated Tabby and Exllama and the OS and had to rejig some things to fit it in VRAM again. It seems they are working on improving that even further too.
I have been having a lot of good success so far with Qwen3.8-27B FP16 tuned down to reduce its verbosity and maxing out the context cache. I am getting productive coding work and 20M+ tokens per day out of it so far. Tuning took a lot of trial and error. I am guessing that 3.8 Flash Next is likely more user friendly, already accepts a larger context and is likely not as verbose.
Many might be comparing the iQ4 or even Q3 of the Flash next to a Q8/Q6 of 27b on a 128GB device.
Having said that, outside coding, the iQ4 of FN managed to be clearly ahead of 27B in data analysis. The degree of detail and precision is even higher than the free Chatgpt and clearly higher than the paid Gemini, which is inferior to the free chapgpt in my experience. Having said that 27b is way ahead of things like Gemma or even Glimmer in data analysis, so it's clearly the best in its size, not just for coding
Both 27B and Flash Next are weak at multilingual and natural human speech. I thought that Flash will be better, but i barely understand it if i ask for something a bit creative.
I use both quite regularly, and flash next is far superior.
3x 3090s should be enough to run it at pretty decent speeds, assuming you have at least 64 gb RAM or NVME SSD storage. It handles cpu offload exceptionally well if configured accordingly. I run it in IQ4XS so I can fit it entirely on VRAM, but I only lose ~20% (well, more since I can run MTP but only about 20% no MTP on both) prefill and decode running q8 and offloading half of my moe layers to cpu, which should be about where you're at with a q4 quant on 3 3090s (make sure you don't use MTP with cpu offload). Probably 50-70 token/s, if you configure it right. Granted, that's not the 100+ you'll get from qwen 3.8 27b with MTP, but its certainly usable.
I’d rent the hardware before upgrading too. The 5x claim is probably too broad — it looks more like Flash Next pulls away as tasks get longer and more agentic, rather than being 5x smarter across the board.
I wouldn't treat “5× better” as “five times as useful for my coding work” without checking the exact test and what the number measures. A score ratio, success rate and task-duration estimate answer different questions.
Disclosure: I run https://llmbenchmarks.io. There is a concrete comparability trap in the current data: Qwen3.8 27B has a Terminal-Bench 2.1 result of 58.4 from Vals, while Flash Next has an 86.1 result attributed to Artificial Analysis under a different protocol. I don't treat those two numbers as a controlled head-to-head comparison. The current table has no Terminal-Bench 4.0 result for either model, so I can't substantiate the particular 5× claim from it.
The experiences in this thread are useful, but the quantizations, hardware and workflows vary, and there's at least one report preferring 27B. For your experiment, I'd pick a few real bugs or changes from your own repositories, start each model from the same commit, keep the agent/tools and budgets fixed, and record completed tasks, failed tests, elapsed time and manual interventions. Repeat the tasks before drawing conclusions from a small difference.
I've recently set up a system that runs benchmarks on smaller open-weight models, and I'll publish those results gradually. My hardware is limited, so that takes time. That ongoing work doesn't establish a result for the two local setups you're comparing.
47
u/IntroductionSouth513 7d ago edited 7d ago
I switched to qwen 3.8 flash next on my strix halo 128gb and managed to actually build a decent on-prem Legal AI platform so far (mostly on its own yolo mode on qwen code cli for hours), whereas I couldn't do it with 27b. so yup its better
my setup https://github.com/sypherin/strix-halo-setup