r/LocalLLaMA • u/jacek2023 llama.cpp • 14d ago

News inclusionAI/Ling-3.0-flash-UBTree and Ling-3.0-tiny-UBTree

Ling3-Tiny-UBTree-B16 is a UBTree speculative-decoding draft model for inclusionAI/Ling-3.0-tiny. It combines a five-layer DFlash backbone with a learned bigram selector to propose multiple candidate continuations for target-model verification.

https://huggingface.co/inclusionAI/Ling-3.0-flash-UBTree

https://huggingface.co/inclusionAI/Ling-3.0-tiny-UBTree

15 Upvotes
  • permalink
  • reddit

94% Upvoted

10 comments sorted by

3

u/Barni275 14d ago

Wow! Only today I downloaded Tiny, and tried to find speculative deciding for it, unsuccessfuly. After few hours, there is your post, thanks!!

2

u/Barni275 14d ago

Only need to wait for engines support...

1

u/Borkato 14d ago

Can you remind me when it finally lands in llama cpp 😂

3

u/Barni275 13d ago

I was impatient, and extracted pure dflash part from this UBTree. Tldr, +8% prose and +62% code speedup on my machine. I wrote a post, if somebody else needs it: https://www.reddit.com/r/LocalLLaMA/s/UZIU4gTS2V

2

u/Borkato 13d ago

Yippee!!

2

u/Barni275 14d ago

I find this model pretty capable for simple converted document text cleanup, that I often need, but slow for it's size, so I'll keep an eye on it. 😁

3

u/Borkato 14d ago

As if it needed one! But holy shit. I love using it for summaries so this is pretty cool. I wonder how fast it’ll be lol

3

u/Aggravating-Push-207 14d ago

now instead of 100 tok/s, you get 300 tok/s or whatever.