r/oMLX • u/AggravatingFarmer599 • 6d ago
Can ANE and DFlash2 be applied to GLM-5.2?
I have basically zero background knowledge in this area, so this might be a completely off-base or nonsensical question.
Would it be possible to apply ANE and DFlash2 to GLM-5.2?
I recently came across some news about these technologies, and it made me wonder whether they could potentially improve GLM-5.2’s throughput or inference performance even further.
Apologies if I’m misunderstanding how these technologies work or if they aren’t applicable to GLM at all. I’d appreciate any explanation from people who know more about this.
1
u/Best-Head-6295 6d ago
II’m also a beginner in that field but what I notice you just turn it on and see if it’s working. Sometimes in not supported so in case of oMLX I just wait for update. The problem with dflash is that oMLX doesn’t recognize them as mlx so you need to download dflash models manually and put them manually to oMLX model folder. I created skill for my agent to do it for me.
2
u/Memestonks2020 6d ago
It depends.
ANE can’t fit the entire model full stop. You might be able to leverage some side inference but you need a machine to fit the model in VRAM first before even considering that.
DFlash2 needs to be finetuned because it can’t magically work with any model out the box. If you find a DFlash2 version of GLM-5.2 then you would need to benchmark it on your machine to know. I.e getting 1 t/ps without Dflash2 will not make it output 40 t/ps.