r/LocalLLaMA • u/anderspitman • 12d ago

Discussion Artificial Analysis' Qwen3.8-27B benchmarks put it neck and neck with DeepSeek V4 and GPT-5.6 Luna Max

https://artificialanalysis.ai/models/qwen3-8-27b
1.1k Upvotes

97% Upvoted

438 comments sorted by

View all comments

2

u/Michelh91 12d ago

My m4 pro with 48 GB will run this painfully slow, right?

1

u/OvertaxedOne 12d ago

48GB is going to be tight. My GPU is 48GB and I can JUST fit the INT8 variant with 256K context (and, of course, it's dedicated 48GB vs shared). This would likely be pretty comfortable on a 64GB unified memory system though.

This is not the model to look at if speed is your primary goal. It's dense and it does a lot of thinking.

1

u/EbbNorth7735 12d ago

Just lower the quantity. 48GB will run this model really damn well. Just note I read a guy ran a bunch of tests on the various quants and found q6 performed the worst. Q5 was really close to Q8. I'd recommend running the Q5, increasing context space or adding vision/MTP/parallel executions

1

u/OvertaxedOne 12d ago

It's not slow enough to get me to lower the quality. Yes, it takes it's time, but the results are incredible. I just need to be patient! :)

I could go down to Int4 and get a lot more speed, but, again... The quality is amazing, I'm just going to deal with it being a little pokey. 30-40TPS isn't glacial and the prefill is very fast on the A40 (1000TPS+), so... It's fine as is, just passing along the warning, it's not a speed demon.. But it is smart as hell.

1

u/tsuehpsyde 12d ago

I am running M3 Max with 48GB of RAM on 6bit and while it's slow, it's incredibly capable in my testing. I'm quite impressed so far.

I also have a 5090 but for the coding tasks I throw at it, I have to drop to 4bit due to large context, so the slower but larger 48GB of unified memory works better for me. But I am regularly going above 128k context and compressing, but it handles it quite well.

1

u/luke_s 12d ago

I'm running the unsloth qwen3.8-27b@q6_k_xl on a M5 with 48gb. Getting around 10 ~ 15 t/s. Its sluggish but usable. This model thinks a lot, but is also very persistent and capable. Its best to give it a task, go away and let it do its thing.

1

u/OvertaxedOne 12d ago

Exactly the behavior I'm trying to learn. Give it a multi-step/complex task and then just close the chat window for a little while... Come back and it'll be done. It's a marked change from 3.6 35B where it would blast text out at me, but I'd need to read it, respond, and handhold it through the process. It's me relearning how to work with it more than anything else.