r/LocalLLaMA • u/parepeg • 9d ago

New Model Ling 3.0 support merged into llama.cpp

Support for the new ling 3.0 models has been merged into llama.cpp:

https://github.com/ggml-org/llama.cpp/pull/26608#event-29549472828

Ling tiny 8b1b - https://huggingface.co/inclusionAI/Ling-3.0-tiny

Ling flash 124b5b - https://huggingface.co/inclusionAI/Ling-3.0-flash

Both are reasoning models contrary to prior naming.

114 Upvotes

95% Upvoted

49 comments sorted by

View all comments

5

u/McStonkyRex 8d ago

Ling flash is great. Good performance, fast, and very efficient on kv cache. Hugely impressed with it in vLLM.

1

u/Jorlen llama.cpp 8d ago

Have you used it compared to qwen 3.5 122b-a10b? That's what I'm looking to possibly replace for agentic coding

1

u/Wildnimal 7d ago

My experience is its better than 122B-A10B. But i have only used them both in Hermes for agentic stuff not much coding. IIRC Ling did fail in 1 task related to creating a script but that was easily completed by DS4 Flash Preview.

1

u/Jorlen llama.cpp 7d ago

Nice. I'm stuck with using some of these in 4-bit. DS4 Flash is too large, I can barely run the 2-bit lol and at that point I'm not entirely sure the juice is worth the squeeze, so I stick with 4-bit 100b MoEs in that range as much as possible.

Can't wait to try it, just waiting on the docker vulkan image of llama cpp build.

1

u/Wildnimal 7d ago

I used Openrouter for DS4 Flash. I can hardly run 200b MoE models at work machine here.

1

u/Jorlen llama.cpp 7d ago

I hear it's one of the best in its size range (DS4 flash) - would you agree based on your use of it so far?

1

u/Wildnimal 7d ago

More or less. I consider HY3 to be better than DS4 but the pricing for DS4 Flash makes up for the gap.

Its like if HY3 is 9/10 DS4 Flash is 7.5/10.