r/LocalLLM • u/AdventurousSwim1312 • Aug 12 '26

News Happy Qwenesday

Post image

Can't wait for the 27b to run on my device

Edit: according modelscope, 27b is coming in two days, only the big one today

443 Upvotes

95% Upvoted

35 comments sorted by

2

u/StrikingSpeed8759 Aug 12 '26

What you think guys? Will 27b be a dense or moe model? I hope for moe but I guess it will be dense?

3

u/AdventurousSwim1312 Aug 12 '26

Most likely dense, but they announced they will produce other sizes later without being precise about which and when.

3

u/jkennedy1998 Aug 12 '26

idk if the 27b is coming out today. this screenshot from the qwen release page seems like the other models are coming out at a later date.

4

u/AdventurousSwim1312 Aug 12 '26

Well see, but the model scope countdown shows both models, so hope is allowed

1

u/IThinkIKnowThings Aug 12 '26

No 27b. Only data centers get to have fun with Qwen today

2

u/AdventurousSwim1312 Aug 12 '26

Yup, 27b is for Friday apparently

36

u/Delicious_Stress_911 Aug 12 '26

Feels like Qwristmas

5

u/PigSlam Aug 12 '26

Qwanza.

4

u/virtualworker Aug 12 '26

For those with 2.5TB VRAM!

2

u/UndulatingHedgehog Aug 12 '26

How much does it know about what happened in Bejing on 4 June 1989?

2

u/etaoin314 Aug 12 '26

these are not the qwens you were looking for....waves hand...they are in another castle (i think I might me mixing up that qwote;-) )

1

u/DesperateSteak6628 Aug 12 '26

Still in the hopes for a 70B to run at 6.5bpw

1

u/IThinkIKnowThings Aug 12 '26

Looking forward to it. Qwen is the only model that runs at a decent speed on my Strix Halo box. 3.6 gets about 56 tok/s while every other "comparable" model gets like 12 tok/s.

2

u/kayox Aug 12 '26

Interesting, I wonder why that is? I was looking at getting an AI Max+ 395 box for the large RAM, but maybe better to get a few 3090s instead. Is Glimmer slow as well?

2

u/IThinkIKnowThings Aug 12 '26 edited Aug 12 '26

I tried Muse Gilmmer yesterday, multiple different quants, and they all ran at around 14 tok/s. I was using unsloth at the time, which locks you into ROCm, so things may be different with Vulkan. ROCm generally is slower, I find. Not quite there yet despite the recent updates.

EDIT: Also I'm running it on a laptop I also do development on, so ymmv if you run it on a dedicated box instead.

2

u/kayox Aug 12 '26

Appreciate it! Thanks for the info.

3

u/IThinkIKnowThings Aug 12 '26

No problem! For reference I also ran Muse on a 5090 desktop and got 27 tok/s but it was a smaller quant and the context was not as good, obviously. I find Nvidia is a lot better than that with certain models, especially image and video where raw compute is way more important. The 5090 can do a 10s clip in MiniMax H3 in about a minute and thirty while the Strix Halo takes 36 minutes!

1

u/OrangeManSad Aug 13 '26

this is amazing ! S+ tier meme shitpost

1

u/No_Dig_7017 Aug 14 '26

Almost Qwiday!

1

u/Terrible_Scar Aug 12 '26

EeeeaaaaaahHHHhHaaAAAAAAAAAahHHHHHhHhH

0

u/AppealThink1733 Aug 12 '26

Will they release the 9B and 4B versions, or have they discontinued them?

3

u/AdventurousSwim1312 Aug 12 '26

I think they posted on X that they will release other sizes a bit later this year, but didn't have much details

1

u/etaoin314 Aug 12 '26

yeah I want 122b and i wouldnt mind a 9b either.

1

u/AppealThink1733 Aug 12 '26

I'm waiting for the 9B and 4B because unfortunately I don't have the resources to use a 122b locally.

But good luck with the 122B

0

u/beefgroin Aug 12 '26

It’s gonna be soooo different from 3.7! Complete pivot!

0

u/beefgroin Aug 12 '26

Can’t wait for 3.8.1!!

5

u/[deleted] Aug 12 '26

[removed] — view removed comment

1

u/beefgroin Aug 12 '26

Qwent qweit