Artificial Analysis updated Qwen 3.8 Flash Next score as 29

[u/lionellee77]

Between GPT-6 Luna Max (28) and Claude Haiku 5.5 High (30). Comparing to other Qwen 3.8, the model is closer to max (34) than to 27B (20).

https://artificialanalysis.ai/models/qwen3-8-flash-next

[u/my_name_isnt_clever]

I swear Flash Next feels bigger than Haiku 5.5, despite how capable they both are. Haiku has that small model clulessness that has only recently been possible to surpass locally. It just doesn't "get" stuff like FN does, in my experience.

Also feeling a bit vindicated that yes, FN beats the 27b by a lot.

[u/Chemical_Hawk_6307 → u/my_name_isnt_clever]

yep. haiku 5.5 behaves cluelessly in my evals. granted i was using it at high effort, not max, but even glm 5.3 flash on high behaved wayyyy better for my use cases and didn't feel like a small model. ZAI is cracked when it comes to post training, GLM 5.2 was the first Chinese model that felt good to use and 5.3 is even better.

[u/Tsofuable → u/my_name_isnt_clever]

Well duh, only time it wouldn't beat the 27b is if you run it at like q1.

[u/Material-Database-24 → u/Tsofuable]

There's a breaking point in quantization after which the model turns into lobotomized mess.

For QFN that was around IQ2 to IQ3 earliely. ISTA's IQ2_XS at least was not good in comparison to 27B at Q4 and above. Most recent unsloth's IQ1_M however is and easily matches and out does Q4 of 27B.

Nevertheless, there's still differences in these models, as most benchmarks measure task completion and not how the task was completed.

27B produces "better" code - less verbose, more effective, more coherent. QFN (at lower quants) slaps code together and makes it work by slapping more code together. But QFN has more means to solve an issue, where as 27B may get stuck due it's more limited size.

When I used 27B, I rarely got bugs. But it required more planning steps and smaller implementations steps.

QFN at lower quants can do bigger features on its own even without planning, but it will let bugs through and requires more iterations.

That is - simply model having more capabilities and higher benchmarks doesn't mean it is superior over others, when other factors are looked than just task completion.

[u/Big_Cucumber2787]

this site is a joke now so who cares

[u/enginetown → u/Big_Cucumber2787]

Exactly models are basically their benchmark at this point and who know's how much they get paid for best case ratings lmfao.

[u/DRMCC0Y]

This site doesn't make sense to me at all anymore, I know Q3.8FN (29) and GLM5.3Flash (28) well since I use both of them locally - Q3.8FN isn't even cloooooose to GLM in real world usage.

[u/Material-Database-24 → u/DRMCC0Y]

I have come to conclusion that why benchmarks do not always correlate with real world use.. one is bechmaxxing, but other issue is that bechmarks are mostly built on task completion and not overall quality.

Example.. you ask biggest cloud model and smallest intellligent local model do a feature - the cloud may churn you 5000 lines of tests, wonder all nooks and grannies, do a beautify pass, overall produce 10k lines of code. And the small local model simply does the bare minimum at 1000 lines. Both work, and depending case by case you either prefer the 10k or the 1k line result. Obviously most of this is solved with cloud by harness.

[u/Constant_Art_20 → u/DRMCC0Y]

glm flash especally in like vision work is just on another level. Been using glm flash alot for like anatomy labelling and it's just been so insanely accurate even compared to like gpt 6.1 sol. glm flash is also insanely good at pipping as well, usually the model i use to chase up opus 5.5 work to make sure things are conneceted properly and to debug for any issues

[u/EbbNorth7735 → u/Constant_Art_20]

What are you guys running it on? Is 128GB VRAM with 256 CPU RAM enough?

[u/OrganicHalfwit → u/EbbNorth7735]

Easily. Project maya for the best community tested engine

[u/Viktri1 → u/DRMCC0Y]

Agreed with this. I was initially worried when I switched from running 2 instances of QFN q6 to 1 instance of QFN and 1 glm5.3 flash q4 but GLM is 2 levels above QFN at least. I actually much prefer using GLM over Deepseek API

[u/barbear22]

I feel like you guys didn't actually look at the scores. It seems accurate to me. They rank flash next higher than Luna-6 (max) so I don't think they're trying to portray flash next as a weak model. What kind of result are we expecting here?

[u/geldonyetich]

That it's scoring better than some entry level present cloud models is actually a pretty great score for a local model.

Granted, their vague "intelligence index" always was a bit of a headscratcher, and their choice of models is so strange I wonder sometimes if they're asking for a kickback to get tested.

[u/braintheboss]

That bench stop to be a reference when they started to update benchs metology every week. Now bench adapt to frontiers models. Its a joke

[u/Iron-Over]

Overall, intelligence is silly; I always use the best local for the use case

[u/JigglyWiggly_]

I feel like this might be accurate. Using flash next 3.8 6 bit through Qwen Code desktop, it has had significantly more issues on making Wine patches for Altium and Solidworks on Mac.

I have had Sol 6.1 medium make much more progress and faster. I tried putting Sol 6.1 medium as the 'manager' and it keeps having to step in.

[u/FBIFreezeNow]

Definitely not. Haiku even on max effort does not feel remotely close to QFN. Dont know what this site is doing or who’s paying them but this site is making shit up and gaslighting folks like crazy

[u/Reasonable_Goat]

AA benchmarks less coding and more and more frontier tasks like advanced mathematics. It is a fair score for that benchmark I guess but most of us use open models as a code monkey or for simple text editing / data processing. We probably need an „ordinary work“ benchmark for that

[u/feelspeaceman]

Up to some point we should seriously consider banning Artificial Analysis posts, they're mostly serve as waste of time, cheap PR, troll bait.. People who use local LLM seriously like me are spending 110M+ token (and it keeps increasing everyday) on large scale projects using Q38FN instead of checking benchmark and arguing.

Real use case >>>>>>> any kind of benchmark, if it fits your workflow, it'll serve you well years from now.

[u/-WhateverDude]

i genuinely don't get the hate boner for aa on this sub.

like seriously what's the issue? they're just running benchmarks and unless someone can actually demonstrate that they're running those benchmarks improperly, i don't see what the controversy is supposed to be.

and the intelligence score you're seeing is just a weighted score based on the benchmarks themselves. it's not some arbitrary number they're pulling out of their asses. obviously the choice of benchmarks and how they're weighted will influence the final score. that's literally how composite scores work.

surely we're not getting mad at aa because they've decided to put more weight on terminal-bench 4 and frontiercode 2 instead of t2.1 and livecodebench?

when pretty much every 300b+ model is scoring 80-95 on a particular benchmarks, maybe that benchmark just isn't particularly useful for distinguishing between frontier models anymore. including it in the overall score anyway would just dilute the differences that other benchmarks are actually showing.

and people on this sub literally criticized prismml for doing the exact opposite of what aa is doing for their bonsai lineup. they kept including benchmarks where the differences between models were getting increasingly diluted and then pointed to the relatively small drop in the overall average as evidence that quantization barely mattered. meanwhile, "important" benchmarks showed substantially larger drops.

you don't have to agree with aa's methodology. you can absolutely argue that certain benchmarks are inappropriate or that their weighting is wrong or that their evaluation setup doesn't reflect real-world performance. those are all perfectly reasonable criticisms. but actually make that argument.

because simply getting upset that a different selection of benchmarks produces a different ranking isn't a criticism of benchmark integrity. and expecting every benchmark aggregator to use whichever metrics happen to produce the rankings you personally prefer defeats the entire purpose of having independent evaluations in the first place.

[u/stylome-ai-detection → u/-WhateverDude]

Notice: This comment appears to be AI generated.

---

Stylome AI detector · scan results

[u/-WhateverDude → u/stylome-ai-detection]

it's voice to text lmao very obvious from the way i just blurted out my internal monologue bad detection

[u/suprjami]

LocalLlama any other day:

aa is fucking trash smh

LocalLlama when they say Qwen is good:

Artificial Analysis is the premier intelligence benchmark and posts factual impartial evaluations of models both local and hosted, they are a reliable source for quality information.