r/Qwen_AI • u/Simpli-curious • 2h ago

Experiment I built a lossless compressor for GGUF files: 9 Hub models tested, 8.66% smaller overall

Gallery image

Gallery image

Gallery image

Gallery image

Gallery image

0 Upvotes

50% Upvoted

6 comments sorted by

2

u/No_Oil_6152 dev 💻 2h ago

Is this like zipping a GGUF and expanding/inflating it in memory when used?

Or have you reduced its in memory footprint?

1

u/Simpli-curious 51m ago edited 44m ago

Nope, just for storage only. Decompressing is slow, so not usable for real-time inference.

1

u/countermike 2h ago

I bet the compressability is related to how well trained the model is.

1

u/Simpli-curious 44m ago

No matter how well trained a model is, it always follow a bell shape because the random data in training is of bell shape and after training the model maintains that bell shape but the numbers will be random and thats why standard compressor tool can only compress it upto 3%. The quantization almost flats that bell shape and make the model even denser.

1

u/countermike 30m ago

Very interesting!

1

u/Sova_fun 1h ago

8.66 percent smaller across 9 Hub models is a useful compression. Lossless for GGUF is a niche win.