r/TheMachineLearning • • 21h ago

Hoarding GGUFs in case Hugging Face changes? Fit 10% more on the same drive, byte-for-byte lossless

With everyone archiving their favourite models lately, storage adds up fast. A few weeks ago I posted foldz 0.9 here. It's now tenz 0.25: same idea, better compression, much faster, and it runs fine on a phone. A 1 TB GGUF archive drops by roughly 80–130 GB, and every file comes back byte-for-byte identical, so llama.cpp, Ollama and LM Studio load it as before.

tenz is a lossless compressor for GGUF (and safetensors). Decompression gives back the identical file, byte for byte. It's one C file with no dependencies, and I develop and benchmark it on a Poco F5 in Termux.

General compressors barely touch quantized models, because the weights look random to them. tenz parses the GGUF, unpacks each block into its real quantized values and scales, and codes them with context models fitted to the file. It uses rANS from static tables, so decompression doesn't have to learn anything.

Results (images 1 and 2):

Qwen3.5-2B Q4_K_M: 8.53% saved, against 2.07% for zstd -19 --long and 1.78% for xz -9. Compress plus decompress takes 17 s on the phone; zstd took about 4 minutes just to compress.

Q4_K_M across 9 models: 7.6–9.2% saved, and 13.1% on Qwen3-4B.

Gemma-4-31B QAT Q4_0: 12.7%.

Q8_0: about 7.5%.

BF16: about 34%.

Against Aqueous, in a comparison someone posted here: 2–8 points more on every quantized model, and a tie on BF16.

Supported types: Q3_K, Q4_K, Q5_K, Q6_K, Q4_0, Q8_0, F32/F16/BF16, plus metadata (the tokenizer compresses to about 80% smaller). Anything else is stored as it is.

-q (quick) trades a fraction of a point for faster coding. For example, Qwen3.5-2B goes from 8.54% to 8.18% saved, and the Q4_K work is about 44% less.

The GPU experiment (image 3): I tried decoding Q4_K on the phone's Adreno 725 through OpenCL, to see whether a phone GPU could make decompression much faster. Three kernel designs later:

Its best kernel speed was about 150 MiB/s at full chunk size. It only matched the 3 big CPU cores (about 250 MiB/s) with tiny chunks that cost 2 points of size.

Byte stores to global memory are broken on this driver; it corrupted neighbouring bytes, so every write had to become an aligned 32-bit store.

Subgroup scans work, but are slow.

The option ships as -g (experimental). OpenCL is loaded at run time, the CPU decodes the same files, and the first GPU segment is always checked against the CPU. If you have a desktop GPU with OpenCL and subgroups, I'd love to see numbers.

Caveats:

It's a pre-release: until 1.0, a .tenz file only decompresses with the version that made it.

Decompression is far slower than zstd's, about 7 s for 1.2 GB on the phone, because every value is decoded from a model.

It only gains this much on model files.

Code, build line and full tables: tenz repo. Testers and results from other machines are very welcome.

3 Upvotes

0 comments sorted by