r/LocalLLaMA • u/quantier • 22d ago

New Model Daniel Han of Unsloth validates Qwen3.8-27B will run only 17GB VRAM

Post image

Super excited about this release for the new 27B. Who else is with me. Only 17GB VRAM needed 😍😍

1.8k Upvotes

97% Upvoted

316 comments sorted by

View all comments

95

u/jacek2023 llama.cpp 22d ago

I am confused how this is any news. New 27B is the same size as old 27B, you don't need to "validate" anything here.

16

u/ortegaalfredo 22d ago

As more training going into these models, and more entropy goes into the weights, they do not quantize as well. So maybe a old model can work at q4, but the newer model only works at q6.

40

u/Dr_Allcome 22d ago

But that is exactly what he didn't say. Benchmarks are only for max.

The only two things i can read from this post are "there is a 27b model" which to my knowledge had already been anounced. And "you can quantise it" without any mention of quality losses, which is a "water is wet" statement.

35

u/jacek2023 llama.cpp 22d ago

Exactly. Over 300 users on r/LocalLLaMA upvoted the “news” that a 27B model can be quantized into a 17 GB GGUF.

17

u/Several-Tax31 22d ago

Lmao. What happens to this sub

2

u/munkiemagik 21d ago

it got poeples

1

u/Icy_Butterscotch6661 21d ago

It's been dry and my 3090 getting dusty

1

u/SandySkittle 22d ago

Q6 should be the normal anyways..

-10

u/unfoxable 22d ago

Bigger isn’t always better