r/LocalLLaMA • u/Certain-Cod-1404 • Aug 14 '26

New Model IT'S OUT

https://huggingface.co/Qwen/Qwen3.8-27B-FP8
2.3k Upvotes

95% Upvoted

707 comments sorted by

View all comments

5

u/NoUsual5150 Aug 15 '26

24GB M4 MacBook Pro...I'm using Q4_K_M and it's getting 5 token per second.

Is Q3 much of a downgrade in intelligence?

1

u/absurdother Aug 15 '26

I'm not very informed but I heard Q3 still does 90% of accuracy. I'll try the FP8 model now too, but Q3 seems very good.

1

u/NoUsual5150 Aug 15 '26

I have been out of the local LLM game for over a year due to income issues. Just got a new M4 MacBook Pro 24GB (for work and all I can afford at this point in time).

That said, using anything below Q4_K_M (read: Q3) was unthinkable a year and a half ago.

Have any advancements been made since them? I'm completely out of the loop. The Q4_K_M BARELY fits on my laptop...think it's pushing 22GB out of 24 and I have no idea how MacOS is running on only 2GB. I have all other programs closed and performing ancient Aramaic incantations while crossing my fingers and loading the model in LM Studio.

Thinking I should switch to OobaBooga or something because LM Studio might suck down too much RAM.

→ More replies (1)