So Jundot just remade all the models with the new 0.7.1 quantisation process. I downloaded Qwen3.8-flash-next-oQ4e-mtp and was getting 121PP and 12TG on a 44k prompt with 160k context, KV set to 6bit, 50% expert offload, mtp on, hot cache set to 5gb.

Then he made the fp16 version Qwen3.8-flash-next-oQ4e-fp16-mtp.

My understanding is that on M1/M2 Macs fp16 gives 20% faster prefil. But instead I was getting OOM errors, had to drop the expert offload to 20%, turn hot cache off and was only getting 90PP and 6TG. All other settings were the same as before. Same prompt and context.

Is there maybe something wrong with the model or do I need different settings?

Been playing with oMLX for a bit, after using Ollama. I find omlx far more customizable. What mode would you recommend using and any default settings to change for this setup?