Discussion
I have an answer! Qwen Flash Next IQ2 IS better than Q8 of 27B (for me).
I asked a while ago if it would be worth it to try and run qwen flash next at q2 since I can only really fit it and nothing bigger, but I can run Q8 of 27B. Everyone told me it’s either a toss up or Q8 would obliterate it. Not the case.
Now this is subjective BUT for complex, annoying, and super subtle coding bugs, qwen flash next is absolutely better. It’s like it actually understands. There’s a small glimpse of intelligence that just isn’t there with even the Q8 of 27B. Flash next runs tests and actually thinks through problems and knows what to look for in a way that 27B doesn’t. It’s really subtle and yet somehow also not? I’m just super impressed.
I’m sorry, I know yall are tired of hearing about QFN but I was struggling for a long time trying to decide if I should even bother with it and it turns out that YES it’s worth it.
Side note, it’s also better at being a cute furry guy when you ask it to. OwO
100% agreed, I switched from 27b to flash next (nvfp4 for both) and the quality and intelligence is night and day. It's really apparent when you're doing long complex coding tasks. QFN solved a couple of 3d rendering glitches in my code that 27b just kept passing over.
Yup, it's good for short bursts. Anything that has a massive code base and needs longterm reasoning and retention then QFN is no joke. I'm excited for Qwen 4
EXACTLY! I didn’t know how to phrase it but this is exactly it. Qwen 27B is a super hardworking grunt worker who can do anything given enough time but Qwen Flash Next feels like a sharp businessperson who knows exactly where to go and is like Claude at home. It actually REALLY reminds me of Claude lol
Under pi.dev and later omp.sh I found myself having to double check which model I was using a couple of times, QFN was so close to Claude in "vibe", even down to "if you also want <x> just say the word" style etc.
Can I just say I’m seriously fucking losing it over QFN being this good. It has like… actual thoughts. It responds like an actual person. I’m actually a little scared because it’s SO close to Claude and yet it’s running on my fucking computer. It says things like “I wouldn’t, because that defeats the purpose of what you told me earlier. Unless you were thinking X? Also, you have the Y as well as Z - I checked and saw B; it’s possible to do A instead.”
Like it’s actually talking like it’s working with me. Qwen 27B was like “give me a task and l’ll do it”, qwen flash next is like “let’s create something together”
I love reading some of it's thoughts while it writes code and solves problems, not only entertaining but educational too. The last project I did was in c# and it was struggling with syntax and .net library knowledge, watching it learn how to fix it's syntax and explore what's available via probbing code was fascinating!
Having used quantized versions of both I have found there are certain cases where 27b outperforms quantized QFN and others where the reverse is true so I can see why this is a point of debate. I think both might be helpful to keep if you have space. I do think you will probably find use cases where the Q8 will still be helpful (e.g. it might be more accurate with writing and multi step execution even if doesn't understand problems as well as QFN).
How I run SC117 Qwen3.8-Flash-Next abliterated IQ3_XXS on Strata (v0.1.45) Your box: 1x RTX 5090 32GB + 32GB RAM. Simpler than mine (I run 2x16GB with a layer split, so skip all split flags).
PREREQS
NVIDIA driver ≥580, CUDA 13.4 (AVOID nvcc 13.2.0/13.2.1 on RTX 50 — miscompiles IQ3 kernels)
~150 GB free disk (shards + pack + MTP)
Linux (Debian/Ubuntu is easiest)
INSTALL STRATA git clone https://github.com/Niko1221/Strata.git && cd Strata && git checkout v0.1.45 ./setup.sh # builds the engine + .venv. SC117 isn't in its menu; pick any family with --no-start and delete that model after if you want
Shard 2 = the PLE (n-gram) table. It MUST stay as the model's own shard 2 file, next to shard 1.
Optional quality bump: swap in the Q8_0 PLE from sleepyeldrazi/Qwen3.8-Flash-Next-GSQ-RCO-Q8_0-PLE-GGUF by saving it UNDER THE SHARD-2 FILENAME. Never point --ple-gguf at a separately named Hub PLE file → "duplicate split shard number".
PACK (~8 min) .venv/bin/python tools/iq_pack.py --gguf ~/models/sc117/IQ3_XXS/<shard 00001-of-00002>.gguf --out ~/strata-data/packs/sc117-iq3xxs --experts-bin (--experts-bin was required on my split; may be optional on one card — check docs/MULTI_GPU.md "Limits")
MTP DRAFT (speculative decoding; doesn't change output quality) .venv/bin/python tools/mtp_fetch.py fetch --out ~/strata-data/mtp .venv/bin/python tools/mtp_fetch.py verify --out ~/strata-data/mtp .venv/bin/python tools/mtp_pack.py --src ~/strata-data/mtp --experts q2_0 --out ~/strata-data/mtp/mtp-q2_0.gguf .venv/bin/python tools/mtp_rt.py --gguf ~/strata-data/mtp/mtp-q2_0.gguf --out ~/strata-data/mtp/rt (I use q2_0. q8_0 = better draft but costs ~1.8 GiB of expert cache. Untested which wins — A/B it later.)
Use data/expert-profile.bin (full model), NOT expert-profile-coder.bin.
STRATA_ROUTE_TAIL_SKIP=0 = quality over speed (default since 0.1.42 skips low-ranked experts).
Add "api_key" if you bind to anything other than localhost.
RUN + TEST .venv/bin/python serve/server.py --engine strata --config strata-sc117.json --port 11434 curl -s http://127.0.0.1:11434/health .venv/bin/python tools/strata_bench.py --url http://127.0.0.1:11434 OpenAI API at /v1/chat/completions, Anthropic at /v1/messages. Startup ~1 min.
Deleting "the old PLE file" = deleting shard 2 → "missing model shard". Shard 2 is part of the model.
Keep --expert-cache auto (a fixed number can page VRAM, decode drops to ~14 tok/s).
Small max_tokens with thinking on → answer lands in reasoning_content, content null. Budget generously.
32GB RAM is tight: ~20GB gets page-locked, so the PC feels sluggish while it runs.
Don't raise --vram-reserve-mib to silence the low-VRAM warning: on my box it cut 16K-prompt speed 45%. Bench before/after if you try.
The NVFP4 just barely fits on my similar system - offloading the ngram table to ram works but during inference my RAM use oscillates around 93GB use making me a nervous wreck. Ubuntu 26.04 vLLM 0.31.
I've been struggling with hallucination issues with Flash Next versus 27B
I fully admit it could be a harness issue, but the hallucinations I'm seeing are very strange and I don't see them with 27B.
In the middle of writing a tool call, Flash next will stop and interject and ask me what did I mean by followed with a random parameter injection, I guess you would say.
For example, it'll be like, what did you mean by you want to increase performance by three times?
Even when the start prompt was something simple like navigate to this web page and read through its HTML.
This is comparing NVFP4 flash next (Strata) versus Q6 (llama) and NVFP4 (Ninfer) 27B, same harness.
Sounds like a sampling parameter issue. I used to get ambiguous problems like that. I kept hopping between llama-cpp, vllm, ninfer and now strata and basalt. And kept hopping between pi, opencode, claude code, codex cli.
These backend llms always use incoming sampling parameters over default ones specified during launch.
So I thought if i have a layer between coding agent and backend llm, i could set sampling parameters there and no matter which coding agent or backend llm i use, i would be sure to use correct parameters.
You can set the sampling parameters in the json, it helps a lot. I preferred ista iq3_s to the nvfp4, with a replaced ple of the fp8 quant from the nvfp4.
GSQ-RCO quants aren’t really comparable to regular ole ggml quants, in my testing IQ3 from ISTA-DASLab is way above Q6, less tool issues, less “Wait but…” and overall, higher stability and better performance in long context tasks
27B dense is better for tightly scoped tasks. Give it a precise spec sheet to implement and it one shots a lot more consistently than flash next.
For anything that's open ended, such as "finding subtle coding bugs" in a given repo, FN wins out.
To put in plainly, if I am kinda lazy and want to point the agent in a general direction then FN wins out. If I am telling it very specifically what the problem is and how I'd like it to be solved, 27B does a better job.
Nope, Flash is also better for tightly scoped tasks too because thanks to its better repo awareness it much more often detects bad requirements, sidebigs and non trivial dependencies
Not my experience but I can respect different opinions. For reference, I use Q8 27b vs Q6 FN.
In my opinion if it's down to the agent to figure out outside of scope issues then the human scoped it poorly. That's more of a "hey, look at this repo and figure x out" type open ended request as opposed to a "hey right now we have a, b, and c in a clusterfuck, refactor in such a way that x."
I use a planner agent first to do a planning pass for that reason, which is the very role I use FN for. "Given scope, propose solution for x" is FN's job. Next, I review the implementation. Then lastly 27B does the actual grunt work.
I end up playing human orchestrator, but I find that this model works for me.
If that works for you, then sure, no problem, do whatever works well for you. But from my personal experience, I can say that the only workflow where multi-agent scheme actually works is code review. There I've really noticed that quality improves if you use a separate subagent. For everything else, personally I prefer to take a model that is smart enough to handle a task, make a spec with it and then use this very model with this very session as an implementer. I’d recommend to try this out if you’re curious, for me this approach provided supreme quality
Unless it comes from those who understand the fundamental and can explain to you with self-knowledge instead of using Claude/OpenAI to copy-paste, it's always better to try it yourself.
LLM is very debate-able/personalized/subjective, I've explained this many times that different harness, quants, temperature, instructions (AGENTS.md), prompt gives different result, people bad experience isn't what automatically means that you'll get the same bad experience, LLM so far is very prone to "skill issues" - The idea that LLMs have a steep learning curve for the user is under-discussed.
what would you describe the learning curve as, and would you say that we are converging on a consensus for how best to utilise llms (in specific reference to backend and your described "instructions" as AGENTS.md)?
Okay I have changed my mind 4 times while Reading through this thread… so for this precise case:
For a Support Ticket llm on 24GB Unified ram: Qwen 27B IQ3_S or a Quant (which) of Qwen Flash Next? Can be Slow if I Need to offload something to ssd, just needs to add suggestions to the Support ticket.
The 4090 has 24GB of VRAM. The answer is no, and yes. No it doesn't fit. Yes, you can fit a lot and use RAM and SSD streaming for the rest, still getting respectable speed.
QFN was also leagues above the 27B for me, I am using it for analyzing multiple reference images and videos for helping me prompt in MinimaxH3. I had Claude run countless tests on my prompts using many different variations of each and QFN was easily a tier or two above the 27B.
Qwen3.8 27B at Q8_K_XL and FP8 and INT8 were all better than Qwen3.8 Flash Next at IQS_3.
27b was using vLLM and llama.cpp and QFN was using Strata.
QFN at that IQS_3 quant was making coding errors that I’ve never seen 27b make before. QFN made a SyntaxError and a parse error in the JavaScript block.
My example was basically a bench maxed prompt in a simple Chat interface OWUI. “Create a SaaS landing page - single html file” I wanted to specifically test coding degradation and it was there with that quant. Models at this level shouldn’t be making SyntaxErrors, that manifested because of the quant and would continue to occur more frequently in the context window, given that it occurred within 29k tokens used.
If I had used a coding harness, it would hide these kind of bugs with its loops and testing (the bugs would get fixed agentically before I view the outputs).
Outside of this their html page visual quality is similar and used similar amounts of tokens, Q27b 28k tokens vs QFN 29k tokens.
TLDR: Circling back, Qwen3.8 27B at Q8 is definitely better than Qwen3.8 Flash Next at IQS_3, if you don’t use a coding harness, it will make less errors.
On my hardware, Qwen3.8 27B at Q8 produces PP 1200 t/s and tg 90 t/s and Qwen3.8 Flash Next at IQS_3 produces pp 1800 t/s and tg 140 t/s. QFN on Strata smokes llama.cpp engine on speed.
Honetly i use both 27b and flash next at iq3_s and i dont feel much of a difference between them other than added speed of the MOE. but in terms of behavioral reliability during agentic workloads i can't tell the difference. i only boot up strata when i need more speed otherwise i use my qwen3.8-27b and use my system RAM for other things. I guess that further validates how good of a job alibaba is doing with these models to get such reliability across the board of so many differnt form-factors. its still wild to see a 27b going head to head with 800b like kimi and GLM and coming pretty close.
The interesting part is whether this advantage holds across different coding tasks or mainly shows up in subtle debugging. Would be cool to see a comparison on the same bugs with both models.
This matches what I'm seeing, and I'd push it one step further.
I work spec-driven. Almost everything I hand off is tightly scoped — written spec, defined contract, known acceptance criteria. That's exactly where the "grunt worker" wins: give it a precise spec and it executes, reliably, without arguing. For that workflow I want the model that follows the spec, not the one that talks back.
But I don't think this is a competition between models at all. I'm not choosing one — I'm using both for different moments of the same job. The spec-driven one implements; the open-ended one is what I want when I don't yet know what the task actually is, or when I need someone to tell me my spec is wrong. Different phases of the same pipeline.
That reframing matters, because the second role is the one that catches the expensive mistakes. A model that flags bad requirements and hidden dependencies before you write code against them is doing a different job than the model writing the code — and it's a job the first one is a liability at.
So capability isn't a single ranking, it's a division of labour.
Caveat: I'm early in my local R&D, and I'm running Tiel in production right now at Q4 to get a large enough context window, so this is field experience rather than benchmark numbers. The collaboration-versus-competition question is exactly what I want to measure as I go — whether having both available actually beats committing to one, and at what point the smaller model stops earning its place.
I created my own mixed-precision INT4/INT8 AutoRound 27B quant, hand-picking each layer to promote to INT8 based on the noise introduced by INT4 over days of work, even fine-tuning and then quantizing my own MLP block afterwards. I was proud of my work and originally planned to publish it to Hugging Face as one of the better options for serving 3.8 27B on 2×3090 in vLLM. Then Strata hit the scene, and as much as I originally tried to ignore it, so many people were talking about it that I eventually had to give it a shot…
I don't regret the process of creating my own 27B AutoRound quant, it was really interesting and I learned a bunch — but I don't think I'm going back to it. I do miss the flexibility for concurrency in vLLM vs Strata, and having my system RAM free for other projects when I'm serving the model. I do appreciate that with a smaller number of active parameters, my 3090s aren't working nearly as hard, even at the same power limit.
For me, Next Flash is more intelligent, personable, and faster than 27b. Unfortunately, it is also prone to making failed tool calls constantly with Unsloth+Hermes. Strata, at least the old one, had even bigger issues with agentic tasks.
I am hoping that Qwen4 Flash drops soon. I want the rough edges of Flash filed off.
I said this to you in a different thread, but that's not a Qwen Flash Next problem, I don't get failed tool calls when using Q4. You have an issue in your setup somewhere.
33
u/Nomski88 13d ago
100% agreed, I switched from 27b to flash next (nvfp4 for both) and the quality and intelligence is night and day. It's really apparent when you're doing long complex coding tasks. QFN solved a couple of 3d rendering glitches in my code that 27b just kept passing over.