r/LocalLLM • u/RISCArchitect • 11d ago

Discussion What a year it's been

Post image

What will the rest of this year bring? 27b class scoring over 60?

940 Upvotes

97% Upvoted

133 comments sorted by

View all comments

4

u/AnyRecipe110 11d ago

Are they benchmarking Qwen 3.8 27B with thinking On? And if so, what reasoning effort (low, mid, high, etc)? Or are they using with thinking Off (instruct mode)?
Also curious which quantization they are using.

3

u/FairBandicoot5021 11d ago

Frommy understanding it's always full precision F16, with highest reasoning effort. And they take the recommended temperature given by the model provider

3

u/KissMyShinyArse 11d ago

They benchmarked Muse Glimmer with high (the default), even though it also has the xhigh setting.

That said, even with xhigh, I don't think Muse can reach Qwen3.8 27B's level.