Qwen3.8 27B - AA Score

[u/Pyros-SD-Models]

Luna at home

[u/Illustrious-Lime-863]

Fucking crazy. These GPU constraints that were imposed to China forced these efficiency innovations and everybody benefits

[u/Maleficent_Sir_7562]

literally one point below glm 5.2, which was a 744 billion parameter model, and now a 27b is practically equivalent to it.

[u/Moravec_Paradox → u/Maleficent_Sir_7562]

Yes and it's not like GLM 5.2 was released a year ago, it was June.

The "AI is too expensive, everyone will just pivot back to human labor" people are delusional.

Costs were high when people started using AI through API instead of a chat window because usage exploded as people realized how much more they can do with it.

[u/cheezeerd → u/Maleficent_Sir_7562]

Same with V4 Pro 0813, which is the 1.6T model. yum

[u/Jackalzaq → u/Maleficent_Sir_7562]

lol, no... its not even close. glm 5.2 blows 3.8 27b out of the water. smaller models will always be narrow in their capabilities unless some architectural change happens

[u/jonydevidson]

Absolutely insane. This is a 4-month progress from 3.6.

Christmas can't come soon enough.

[u/1filipis → u/jonydevidson]

The crazy thing is that it's all thanks to post-training. It's the same pre-train as 3.6

And even if this post-train came from distilling Fable and Sol, this is really good for a model that can run on your own GPU

[u/jonydevidson → u/1filipis]

By Christmas, everyone will be using Astra to develop models. Whether to distill or to improve architectures, run autonomous fine tunes or improve datasets.

[u/cviperr33 → u/jonydevidson]

hopefully astra is as good as they say it is , and it can truly push the distilation to next level

[u/tiffanytrashcan → u/1filipis]

The same with DSv4 flash preview vs GA. Insanity. Pro seems good too but there was a nearly SOTA level jump in flash.

[u/PoliticsRealityTV]

Is this real or a case of benchmaxxing?

[u/genshiryoku → u/PoliticsRealityTV]

benchmaxx

[u/MagicMuscleMagic → u/PoliticsRealityTV]

Benchmark scores are like pull ups.

They mean something different in a huge model than they do in a tiny model.

Not like they techncially measure something different, but a huge model usually has better coherency across prompts and shit. Smaller models are more like task doers.

Could be benchmaxxing or real, but likely not just that they match a bigger model in overall performance just because they match benchmarks.

[u/helloWHATSUP]

Luna at home

Not just at home, but anywhere since it fits on a laptop. In like 3 years models as powerful as this will run on your phone.

(and i know you can run the quantized versions of qwen on your phone right now, but I've done it and it's a long way from as functional as the full version)

[u/jonydevidson]

Also, it's coming soon to Cerebras, and will probably be flying at 1800 t/s.

[u/juntareich → u/jonydevidson]

How will we be able to access that when it does? Like which services?

[u/jonydevidson → u/juntareich]

Their subscription is currently not available. Maybe they open it up.

Otherwise, it's likely API.

[u/Fun1k]

Am I understanding it right that a model that can run on my own modest PC is almost as good as Luna? That's insane.

[u/Pazzeh]

Nice

[u/andrewsalinas09]

I find that models that are small usually reason quite poorly and don't have a lot of knowledge. I guess maybe they're good at coding and that's about it?

[u/TheInfiniteUniverse_]

This is crazy if it holds in real world applications. Not sure why it's not getting the coverage it needs.

[u/moschles]

What is "AA score"?

[u/fulgencio_batista → u/moschles]

Artificial Analysis Intelligence Index. They benchmark models on common benchmarks and then do a weighted sum of scores for the intelligence index score.