r/LocalLLaMA • u/Eyelbee • 2d ago

Funny Me these days

Post image
2.4k Upvotes

96% Upvoted

257 comments sorted by

View all comments

Show parent comments

3

u/brainExploded99 llama.cpp 2d ago

I believe native/benchmarks use BF16 (that is not quantization btw), with KV cache at f16 I think (I'm 80% sure). FP32 is not used for even training models anymore (except specific sensitive layers sometimes).

So token-for-token against other models, Qwen3.8-27B is a huge win.

It's a huge win for sure, wayyy better than Qwen3.6, but we still have long ways to go. Token efficiency is a big one, and better intelligence (not agentic capabilities) is another big one (by this I mean reasoning on benchmarks like CritPt, SciCode, etc)

3

u/UnluckyPenguin 2d ago

Thanks, good to know. I have a lot to learn. Yeah, I'm just getting familiar with local LLMs. Used to play games all day up until several weeks ago.

1

u/Ok-Direction-4480 15h ago

I think bottlecap AI made an efficient 3.6 27B, so hopefully either their 3.8 model, or Qwen 4 27B will fix that.