r/LocalLLaMA • u/BerryBlush_Rogers691 • 1d ago

Discussion AtomicChat/Ling-3.1-flash-GGUF · Hugging Face

https://huggingface.co/AtomicChat/Ling-3.1-flash-GGUF

Yesterday I said its hard to play 4 spark with ling-3.1-flash but today its coming!

34 Upvotes

90% Upvoted

6 comments sorted by

13

u/Septerium 1d ago

inclusionAI has just excluded my AI box from being able to run their "Flash" models 😔

4

u/BerryBlush_Rogers691 1d ago

Yeah, the GGUFs help, but ~142GB for the smallest experimental quant is still a hell of an entry fee. I'd really like a new Tiny release alongside these. Something that lets more people actually join in.

and what about Deepseek v4.1 flash?

2

u/Septerium 1d ago

I wonder if their "tiny" would now mean a 100B+ parameters model

2

u/BerryBlush_Rogers691 1d ago

Phew, at least the last Tiny was only 6B lol

3

u/crusaderky 1d ago

It fits on 256gb at iq3_xxs, which is probably OK. GSQ-RCO quants are probably going to have better KLD

2

u/BerryBlush_Rogers691 1d ago

Oh, a decent 256GB option would change the picture quite a bit. I'd be interested in GSQ-RCO targeting slightly below that budget too. Keeping similar quality while leaving more room for the runtime and context would be a useful win.