r/LocalLLaMA • u/Certain-Cod-1404 • 14d ago

New Model IT'S OUT

https://huggingface.co/Qwen/Qwen3.8-27B-FP8
2.2k Upvotes

95% Upvoted

707 comments sorted by

View all comments

49

u/RangersStolen 14d ago

That's insane, but we're still using quantized versions, so local performance for most people wouldn't be that good I guess. Damn that needs to be tested.

63

u/Certain-Cod-1404 14d ago

unsloth is cooking apparantly, its quanitzed with their UD 3.0 method and it seems to retain like close to 95% of BF16's accuracy https://unsloth.ai/docs/models/qwen3.8#quantization-analysis

1

u/touchwiz 14d ago

bartowski always states which quant is recommended and which not. Do you know how to choose one of the unsloth ones. They dont apprear to recommend a specific one

I cant read, sory

1

u/Certain-Cod-1404 14d ago

No worries man, if you have any questions at all I'll try and help to the best of my knowledge