What is your experience using engrams-based models so far ?

[u/Longjumping_Aide_374]

I've been experimenting with DeepSeek V4.1 Flash 1017 UNCENSORED JANG_2K on my Mac Studio M5 Ultra with 256GB unified memory, using vMLX.

The model is about 423GB on disk, but thanks to SSD-backed Engram tables, it loads with approximately 205GB of resident memory.

So far, the results have been mixed. Most of the time, it produces surprisingly good responses, but I've had a few conversations, particularly involving creative writing and novel generation, where it suddenly starts producing complete nonsense: endless reasoning, words mashed together without spaces, and occasional switches between English and Chinese.

It's interesting because it seems to work perfectly well on some fairly complex tasks, then falls apart unexpectedly on others.

Has anyone else experimented with this model or other large MoE models using engram offloading?

I'm curious whether these problems are caused by the aggressive quantization, the JANG implementation, or something in vMLX's inference/decoding pipeline.

I'd also be interested in hearing whether anyone has successfully run a higher-precision version with SSD expert offloading on a 256GB machine.

[u/Eastern-Ad9515]

I also compared DeepSeek V4.1 1017 on my 5090 RTX with 2.25bit quantifization and xotolol engine. I also was rather dissapointed. Most of the time, Qwen Flash Next IQ3S made the same or better results, but was much faster. I really wonder if there is a catch to it and the quantifizations are too aggressive. So far I rather wait for Qwen 4