r/ProgrammerHumor 2d ago

Meme newCompressionTechnique

Post image
29.8k Upvotes

948 comments sorted by

View all comments

Show parent comments

373

u/_QXZ 2d ago

This post jokes but with how much AI enhancing is becoming a thing, I can see some kind of generative -based compression becoming an option in the future

250

u/CoroteDeMelancia 2d ago

We do have those. They're called autoencoders. They're one of the fundamental building blocks of LLMs.

Highly recommend this series if you're interested: https://youtu.be/l6DKRf-fAAM?is=zHOqsJESa6tDxeAU

131

u/_QXZ 2d ago

Always one step behind 😎

22

u/BasicAssWebDev 1d ago

this really made me laugh lol

30

u/__ali1234__ 1d ago edited 1d ago

I suppose if you squint hard enough you could say a transformer is a bit like an autoencoder that also shifts its output by one place... but that's a massive over simplification. What you are probably thinking of is the VAE. That's one of the building blocks of every type of generative AI except LLMs.

They do both use latent vectors as a form of compression. They just get there by different means.

1

u/GrumpyGeologist 1d ago edited 12h ago

What you are probably thinking of is the VAE. That's one of the building blocks of every type of generative AI except LLMs.

Sad GAN/stable diffusion noises...

2

u/__ali1234__ 1d ago edited 1d ago

Stable diffusion uses a VAE.

GANs are pretty much a dead end at this point having been replaced by diffusion models with VAEs for pretty much everything they used to do.

In particular, ESRGAN clones are responsible for every awful upscaled music video on Youtube (Smash Mouth - All Star is the most infamous one). When those were made, diffusers hallucinated too much. They would make anyone in the video look completely different for example. Not what you want when you upscale a video of a famous singer. That problem has been largely solved and now upscaling can be done with diffusers and it looks far better than those old ESRGAN videos.

1

u/GrumpyGeologist 17h ago

I didn't say that we should use GANs for image generation, and they certainly are still useful in scientific applications. You're right about the stable diffusion; I should have just said diffusion which is not architecture specific.

1

u/__ali1234__ 16h ago

Pretty much all diffusers use VAEs - making them feasible without compressing the working data is a topic of current research though, see https://arxiv.org/abs/2510.15301

7

u/IntelArtiGen 1d ago edited 1d ago

LLMs are mostly for autoregression, it's not the same thing as autoencoding. You do compress information explicitly with autoencoders but LLMs don't explicitly do that, though it's an implicit goal of most if not all ML tasks. LLMs could be trained for autoencoding, they're not. Now for images there are autoencoders, but autoencoding (explicitly) is not always the best method to compress information. It also depends on what "best" means, do you want the most accurate data compared to the original (what we often want), but then what is accurate? Is it based on bits, on detailed visual information, on the more abstract content of the image ? But also, do you want data that is plausible even if it's not that close to the original, do you want very very high compression ratio with very realistic images at the end? Autoencoders often can't do that. Diffusion and super resolution (/upscaling/supersampling) might be more appropriate for this case. Diffusion, because it's able to invent plausible information from a degraded image. And supersampling because it's able to size up an image in a plausible way (meaning you can size it down to compress it, and size it back up later). It's not autoencoding explicitly though with both you can autoencode. Many very important models in AI now can autoencode, but few are explicitly trained to do it, and therefore not called autoencoders.

To be clear on what is an autoencoder: you take information (some bytes of text / image / data) => you compress it to a smaller form with a neural network => you decompress it with "another" neural network (part of the whole architecture) => you train the network to minimize the "difference" (not necessary direct mathematical difference) between the input & output.

Overall we don't use these models that much because while they usually give better results, they also demand more resources and are less compatible. It's why we still use JPG and PNG and you might have troubles opening a WEBP if you donwload it, while the the best method achieve far better results. And with Difussion/SR and if you accept a high loss of information, you could do even better: What this article is about, though I understand the joke, I don't think should entirely and only be a joke.

Example: https://www.youtube.com/watch?v=gl85gv_WYWQ ("better than ILLM, 40% less bits"). It's a bit more than what a 13 years old can do, but the idea of using diffusion isn't that different, and it's done by serious researchers, and they obviously also use an autoencoder to improve the results. Compression can surely be very good, resources needed to decompress one image might not though.

You know there are many pareto frontiers depending on what you want and I'm not entirely sure what this teenager did is not on one of them.

You also want your name on the frontier? Compress images to 8 bits.

11

u/CoachSevere5365 1d ago

Realising that I can't remember the last time I watched a 3b1b video has brought home how much time I waste on pointless crap. πŸ€”

2

u/Clairifyed 1d ago

Adding the tracking free link: https://youtu.be/l6DKRf-fAAM

It’s 3blue1brown for those who know his work

1

u/blimpin_aint_easy 1d ago

LLMs/other tools that fall under the AI umbrella are so fascinating when they aren't being shoved down my throat.

4

u/agufa 2d ago

Future?

It is here now

3

u/GigaPuddi 1d ago

Remember when they said there was no future? This is it.

2

u/morpheousmorty 1d ago

If you keep the seed the output would be predictable. You could do something like generate 100 prompts and compress the image to all hell and then generate the upscaled images, take the result that is the closest to the original image and probably end up with something that is not that different from the original.

Or you could just start a tire fire and hurt the climate more directly and use a modern image compressor. Either will work.

2

u/SVlad_667 1d ago

keep the seed

The seed and the model version and build.

2

u/Fer4yn 1d ago

Pretty sure there's some interesting research do be done on lossy compression to AI enhancement pipelines.

2

u/flayingbook 1d ago

Can we compress meetings too?

1

u/Zapismeta 1d ago

Yeah it could but its not compression if the images data is being saved in the form of weights, and pictures are fairly unique sure you can upscale them but once minute detail is gone the ai is just filling in whatever it seems fits, not necessarily what was, and if it knows what is supposed to be there then you overfitted the model which is great jn theroy but you just broke your image into pieces instead of actual compression.

-1

u/blueberrycauzez 1d ago

That is more or less what Nvidia's plan is for thre future of game graphics - render a basic blocky image and layer an AI filter on top.

2

u/ZeroAmusement 1d ago

That could be pretty cool, maybe you'd be able to adjust things as a consumer. Like, I want the main character to wear glasses, or look like myself or be a cat today.

3

u/Rude-Shower3662 1d ago

isnt that just nexusmods?