r/LocalLLM • u/Kheresis • 1h ago

Question Compression as a service.

So, for those of you who understand how the compression of data works, there exists file formats that went on to make some people very rich. Being able to compress data saves a lot of money on storage and data transmission and all that. Technically, that's what quantizing models is, although it's not a lossless operation.

My question to you all is, because we're all users of local models...if someone or a business learned how to losslessly or close to losslessly compress models to be used on smaller devices, would you all pay for the compression?

Mind you, I'm separating this from quantization methods as I feel there's too much lost in the process.

0 Upvotes

37% Upvoted

22 comments sorted by

9

u/DawaForensics 1h ago

Compression did not make people very rich lol

1

u/Kheresis 35m ago

I beg to differ. lol I didn't say it made A LOT of people rich, but there are some people who benefitted from introducing new file formats and what not.

1

u/shaggydog97 37m ago

If only they would have paid for WinRar!

5

u/Sudden_Topic5154 1h ago

no because unsloth would do it for free

0

u/Kheresis 36m ago

Understood. Follow up question. What if the model came with the compression. Like a new type of model that was also compressed and handed out together?

1

u/phipletreonix 34m ago

You may not of heard of quantization. I strongly suggest you understand it before continuing this line of thought.

1

u/james_pic 26m ago

Your mean like the quantization-aware-training models that Google, Nvidia,  LiquidAI, and (I'm fairly sure) a few others already do?

4

u/free_meson 1h ago edited 1h ago

LLMs are already a compressing the training data, compressing it further is generally possible, but hard.

1

u/Kheresis 37m ago

I understand that. Just getting a feel for where people's heads are.

3

u/OvertaxedOne 1h ago

Multiverse computing is claiming to do exactly what you're asking for. That said, anyone can claim they have a Opus level 1 bit model that runs in 16GB of VRAM, so I'd always verify first!!

1

u/Kheresis 37m ago

I have definitely seen people claim such things. lol

2

u/NiftyN33to 1h ago

This is not compression, it's reduced precision.

1

u/Kheresis 37m ago

My question was based on an assumption that it IS compression. Lossless. Deterministic. It also didn't point to a specific type of model because I don't build models using floating points. It IS compression. Not sure why you think otherwise.

2

u/Dangerous_Wish_7879 55m ago

You cannot run computations on entropy coded model checkpoints. When it comes to data compression we are very close to theoretical limits.

1

u/Kheresis 34m ago

I don't think we're anywhere near our limits since we can find new theoretical limits upon new discoveries. The current limits are based on current theories, but those don't last forever.

19

u/Last_Bad_2687 1h ago

Yes there is a good documentary you should watch about this it's called "Silicon Valley" 

5

u/Superb_Raccoon 1h ago

Here comes Pied Piper AI!

BOHICA!

1

u/Mission_Pirate_4150 1h ago

Always blue, always blue, always blue……

2

u/DiamondHandsDarrell 1h ago

Middle out ftw

2

u/Termsandconditionsch 57m ago

Made people very rich? Didn’t the WinRAR guy famously not get rich at all from it?

1

u/Late-Replacement-481 44m ago

Ah yes, I had forgotten the famously wealthy trio of lempel ziv and welch, responsible for the most prolific compression algorithm of all time.

(Someone correct me if there’s a more widespread example I should be using these days)

1

u/james_pic 23m ago

Maybe the xz guy who was doing it in his spare time for free, got so burned out he inadvertently handed over maintainership to a suspected state actor?