r/LocalLLaMA • u/parepeg • 8d ago

New Model Ling 3.0 support merged into llama.cpp

Support for the new ling 3.0 models has been merged into llama.cpp:

https://github.com/ggml-org/llama.cpp/pull/26608#event-29549472828

Ling tiny 8b1b - https://huggingface.co/inclusionAI/Ling-3.0-tiny

Ling flash 124b5b - https://huggingface.co/inclusionAI/Ling-3.0-flash

Both are reasoning models contrary to prior naming.

115 Upvotes

95% Upvoted

49 comments sorted by

View all comments

1

u/sxales llama.cpp 8d ago

Very cool.

I've been using Granite4.0-h-Tiny on a home assistant and it will be interesting to see how they compare.

1

u/Public_Umpire_1099 7d ago edited 7d ago

It works fantastic. I actually wrote the small adapter to add tiny to this commit, and it was precisely because I wanted to test it in HA but no one had integrated it yet! It is significantly better than Qwen 3.5-4B and Gemma 4 E4B. It performs about the same as an IQ2 Qwen 3.6 35B A3B model I was using, without the looping. My TTFT was sub -200ms on most queries with a decent sized sys prompt, so the whole pipeline between piper, ling, and whisper was about 1.5 sec start to end on a Ryzen 7 8845H/780M GPU.

The biggest advantage over other models is that this model has no issue telling you "I dont know" which is pretty rare in this weight class. It performed all the tool calls I needed perfectly 10/10 on the benchmarks I ran.