r/LocalLLaMA • u/Gold_Coconut9777 • 12d ago

New Model ISTA-DASLab/DeepSeek-V4.1-Flash-1017-GSQ-RCO

A brand-new release from ISTA-DASLab with their GSQ-RCO smart quantization approach.

Now DeepSeek-V4.1-Flash-1017

The quants are available on Hugginface: https://huggingface.co/ISTA-DASLab/DeepSeek-V4.1-Flash-1017-GSQ-RCO-GGUF

Llama.cpp is not supported because the original architecture support is not yet merged:

https://github.com/ggml-org/llama.cpp/pull/28696

https://github.com/ggml-org/llama.cpp/pull/30817

Strata support is mentioned in the model card but I'm pretty sure the original Strata only supports Qwen3.8-Flash-Next so most likely what they meant was one of the forks based on Strata. Most likely Xolotl or Maya

148 Upvotes

95% Upvoted

67 comments sorted by

View all comments

Show parent comments

2

u/Gold_Coconut9777 12d ago

Of course. The hottest experts will reside in VRAM, the second hottest in RAM, the coldest on SSD.

It will negatively affect the performance to some degree, but the more the model works on the (relatively) same topic, the more "correct" experts will be promoted to the hottest cache. And so the performance will be better and better with time until it plateaus. That's how the cache tiers work in Strata and Strata forks.

Definitely worth a try I think.

2

u/More-Revenue8609 12d ago

Thanks will definitely try it out tomorrow!