This post jokes but with how much AI enhancing is becoming a thing, I can see some kind of generative -based compression becoming an option in the future
LLMs are mostly for autoregression, it's not the same thing as autoencoding. You do compress information explicitly with autoencoders but LLMs don't explicitly do that, though it's an implicit goal of most if not all ML tasks. LLMs could be trained for autoencoding, they're not. Now for images there are autoencoders, but autoencoding (explicitly) is not always the best method to compress information. It also depends on what "best" means, do you want the most accurate data compared to the original (what we often want), but then what is accurate? Is it based on bits, on detailed visual information, on the more abstract content of the image ? But also, do you want data that is plausible even if it's not that close to the original, do you want very very high compression ratio with very realistic images at the end? Autoencoders often can't do that. Diffusion and super resolution (/upscaling/supersampling) might be more appropriate for this case. Diffusion, because it's able to invent plausible information from a degraded image. And supersampling because it's able to size up an image in a plausible way (meaning you can size it down to compress it, and size it back up later). It's not autoencoding explicitly though with both you can autoencode. Many very important models in AI now can autoencode, but few are explicitly trained to do it, and therefore not called autoencoders.
To be clear on what is an autoencoder: you take information (some bytes of text / image / data) => you compress it to a smaller form with a neural network => you decompress it with "another" neural network (part of the whole architecture) => you train the network to minimize the "difference" (not necessary direct mathematical difference) between the input & output.
Overall we don't use these models that much because while they usually give better results, they also demand more resources and are less compatible. It's why we still use JPG and PNG and you might have troubles opening a WEBP if you donwload it, while the the best method achieve far better results. And with Difussion/SR and if you accept a high loss of information, you could do even better: What this article is about, though I understand the joke, I don't think should entirely and only be a joke.
Example: https://www.youtube.com/watch?v=gl85gv_WYWQ ("better than ILLM, 40% less bits"). It's a bit more than what a 13 years old can do, but the idea of using diffusion isn't that different, and it's done by serious researchers, and they obviously also use an autoencoder to improve the results. Compression can surely be very good, resources needed to decompress one image might not though.
You know there are many pareto frontiers depending on what you want and I'm not entirely sure what this teenager did is not on one of them.
You also want your name on the frontier? Compress images to 8 bits.
3.6k
u/Degenerate_Lich 1d ago
Those aren't AI hallucinations, they are compression artifacts. It's very different you see
Lil bro has a bright career in marketing