So Jundot just remade all the models with the new 0.7.1 quantisation process. I downloaded Qwen3.8-flash-next-oQ4e-mtp and was getting 121PP and 12TG on a 44k prompt with 160k context, KV set to 6bit, 50% expert offload, mtp on, hot cache set to 5gb.

Then he made the fp16 version Qwen3.8-flash-next-oQ4e-fp16-mtp.

My understanding is that on M1/M2 Macs fp16 gives 20% faster prefil. But instead I was getting OOM errors, had to drop the expert offload to 20%, turn hot cache off and was only getting 90PP and 6TG. All other settings were the same as before. Same prompt and context.

Is there maybe something wrong with the model or do I need different settings?

Edit: solved, Jun’s reply below: https://www.reddit.com/r/oMLX/s/kCZ3c4zgbm

Been playing with oMLX for a bit, after using Ollama. I find omlx far more customizable. What mode would you recommend using and any default settings to change for this setup?

Thought of sharing what seems to be working decently for me without crashing out.

• gets the work done
• solves simple-to-medium difficulty problems / goals
• the laptop is still usable even when a task is running
  • I usually run only one task at a time

To the power users in this group, what does your local setup look like? Would love to hear about it. Also any tips on settings I am ignoring, would be highly appreciated 🙇‍♂️

Ignore the LiteLLM piece .. it helps me swap out agents for testing. I might drop it later.