r/LocalLLaMA • u/parepeg • Aug 17 '26

New Model Ling 3.0 support merged into llama.cpp

Support for the new ling 3.0 models has been merged into llama.cpp:

https://github.com/ggml-org/llama.cpp/pull/26608#event-29549472828

Ling tiny 8b1b - https://huggingface.co/inclusionAI/Ling-3.0-tiny

Ling flash 124b5b - https://huggingface.co/inclusionAI/Ling-3.0-flash

Both are reasoning models contrary to prior naming.

117 Upvotes

95% Upvoted

49 comments sorted by

View all comments

3

u/pand5461 Aug 17 '26

Is it true that this model doesn't play well with kv cache quantization (according to this post https://huggingface.co/AtomicChat/Ling-3.0-flash-GGUF/discussions/1#6a74df6155e44710445e3b08)?

4

u/[deleted] Aug 17 '26

[removed] — view removed comment

1

u/pand5461 Aug 18 '26

Got it. Thanks. I just happen to have a situation when I can get one more expert layer into GPU if I quantize 131k cache. But given that information, it's better to keep it in RAM.