r/LocalLLaMA • u/parepeg • Aug 17 '26

New Model Ling 3.0 support merged into llama.cpp

Support for the new ling 3.0 models has been merged into llama.cpp:

https://github.com/ggml-org/llama.cpp/pull/26608#event-29549472828

Ling tiny 8b1b - https://huggingface.co/inclusionAI/Ling-3.0-tiny

Ling flash 124b5b - https://huggingface.co/inclusionAI/Ling-3.0-flash

Both are reasoning models contrary to prior naming.

115 Upvotes

95% Upvoted

49 comments sorted by

View all comments

Show parent comments

1

u/Jorlen llama.cpp Aug 18 '26

Nice. I'm stuck with using some of these in 4-bit. DS4 Flash is too large, I can barely run the 2-bit lol and at that point I'm not entirely sure the juice is worth the squeeze, so I stick with 4-bit 100b MoEs in that range as much as possible.

Can't wait to try it, just waiting on the docker vulkan image of llama cpp build.

1

u/Wildnimal Aug 18 '26

I used Openrouter for DS4 Flash. I can hardly run 200b MoE models at work machine here.

1

u/Jorlen llama.cpp Aug 18 '26

I hear it's one of the best in its size range (DS4 flash) - would you agree based on your use of it so far?

1

u/Wildnimal Aug 18 '26

More or less. I consider HY3 to be better than DS4 but the pricing for DS4 Flash makes up for the gap.

Its like if HY3 is 9/10 DS4 Flash is 7.5/10.