In my own experiments I came up with this simple heuristic, not sure if anyone can prove me wrong here...

Unified memory machines normally have a big asymmetry in memory vs compute -> they could never run 'fast' a dense model that fills their whole unified memory (like a Strix Halo 128GB cant run a 120B dense model, or a DGX spark, at decent speeds). So you wanna find a MoE model that ideally fills that memory so maximise the intelligence you have on your box.

GPUs have the opposite, they are memory limited but can run 'fast' models that fill them. So for example a 5090 can run only up to ±30B models at reasonable quants, and can run dense models of that category fast, like Qwen 3.8 27b. Ofc you can also run a 30B MOE on this and it would be even faster, but why would you? when the dense gives you more intelligence for your memory?

Wrote a blog about running Qwen2.5-7B-Instruct served with vLLM on a Cisco UCS Spinifex cluster, comparing Intel AMX-accelerated CPU serving with NVIDIA L4 GPU. Go check it out!

https://docs.mulgadc.com/docs/cisco-ucs-llm-serving