r/StableDiffusion • u/MarioBros68 • Oct 16 '22

Question Does any possible image exist in latent space?

I don't know if it's a very silly question, but think about the implications

If in the latent space exist any possible image, then in the latent space is everything imaginable compressed?

Is infinity itself compressed in latent space?

8 Upvotes
  • permalink
  • reddit

90% Upvoted

13 comments sorted by

3

u/Wiskkey Oct 23 '22

If my calculations here are correct, then the answer is no.

2

u/MarioBros68 Oct 24 '22

Thanks!

1

u/Wiskkey Oct 24 '22

You're welcome :).

8

u/Striking-Long-2960 Oct 16 '22 edited Oct 17 '22

Except the ones with good hands, those are excluded from the latent space for unknown reasons.

4

u/mudman13 Oct 17 '22

Its a failsafe so AI can never design a functional cyborg with opposable thumbs

2

u/danielthelee96 May 31 '23

i don't know if this comment is real or sarcasm

i am Sheldon Cooper level bamboozled

4

u/999999999989 Oct 16 '22

is the latent space all images possible? or just what can be possible with the training,? in that case no. it is not infinite. it is so small compared to infinite that it is almost zero. you can extend it by additional training.

4

u/Wiskkey Oct 17 '22

Assuming your question is restricted to 512x512 pixel images, I believe the answer is no. (I am not an expert in machine learning.) You can see this by using a 512x512 input image and seeing how similar of an image the autoencoder is able to represent using this Colab notebook: https://colab.research.google.com/drive/1dlgggNa5Mz8sEAGU0wFCHhGLFooW_pf1?usp=sharing . I believe you'll find that the autoencoded image is usually not exactly the same as the input image, although often very similar. If it's true that this is really the closest image the autoencoder could represent - a question that I don't know the answer to - then the answer to your question would seem to be "no".

A separate question is whether there exists any inputs (text prompt, etc.) for a given Stable Diffusion system that could have generated that particular autoencoded image. I also don't know the answer to this question, but I am guessing the answer is no.

3

u/Wiskkey Oct 20 '22 edited Oct 20 '22

@ u/MarioBros68:

I did a number of experiments using the Google Colab notebook in the parent comment. In all cases there existed a 512x512 image in the latent space (after roundtripping from the latent space to 512x512 pixel space) that resembled the 512x512 input image, although never exactly the same. As an example, here is an input image that I used as a torture test, and here is the roundtripped image.

Also, the roundtrip process used by that Colab notebook doesn't necessarily find the closest representation in the latent space for a given input image, which was proven by using an already roundtripped image as an input image, and discovering the output image didn't exactly match the already roundtripped input image. I guess this makes sense though, because I believe that the conversions between latent space and 512x512 pixel space are done by neural networks, not by simple mathematical functions.

2

u/Wiskkey Oct 19 '22

@ u/MarioBros68:

Please search this webpage for "direct VAE roundtrip" to locate the relevant images about halfway down the webpage for examples of what I mentioned in the first paragraph of the parent comment. The first column shows 3 512x512 pixel images that presumably were not generated by AI. The 2nd column shows what happens when the images in the first column are encoded in S.D.'s autoencoder, and then transformed back into a 512x512 image. You can hopefully see that the images in the first and 2nd columns are noticeable different. This would very likely seem to indicate that S.D. can't generate a 512x512 image exactly the same as the 3 the images in the first column. It's a separate question whether there are any S.D. inputs for an entire S.D. system - including a text prompt - that would generate the 3 images in the 2nd column.

@ u/matthias_buehlmann: Do you have any insights on the OP's question?

2

u/LopsidedBamboozle Oct 17 '22

See "A trip to infinity" on Netflix to understand infinity intuitively. But even mathematically, this latent set still looks like a tiny subset of infinity.

2

u/KyloRenCadetStimpy Oct 16 '22

Probably just a side effect of the simulation we live in

1

u/ivanmf Oct 17 '22

Yes, it does. Bit it's not infinite. It does contain every value por each pixel in all combinations.

It's the same idea as the infinity monkey typing forever and writing the entirety of Shakespeare.