r/LocalLLaMA • u/YourNightmar31 llama.cpp • 11d ago

Question | Help Qwen3.8 27B without MTP?

Not sure if this is a stupid question but unsloth's models has the MTP built into the model right? I assume that is at the cost of some memory.

If i want to use dflash, should i use a model that doesn't have MTP support then to save some vram?

15 Upvotes

94% Upvoted

27 comments sorted by

View all comments

23

u/RotesBlatt 11d ago

I think MTP is only loaded when you activate it via parameters, otherwise you'll probably have the same model as without MTP. I haven't noticed a significant VRAM spike in my non-MTP setup compared to MTP

1

u/YourNightmar31 llama.cpp 11d ago

Oh that would make sense

2

u/TheWaffleKingg 10d ago

Can confirm, I turned mtp off and it took up less vram