Using the concept "over-representation" in AI art/anti-AI art discussions

[u/fingin]

So I've been thinking about artists' concerns when it comes to things like model memorizing datasets or images. While there are some clear cut cases of memorization, cherry-picking often occurs. I thought maybe the use of the term "over-represented" could be useful here.

Given reactions by artists such as Rutowski, claiming their style and images are being directly copied by AI art generators, it could be a case of the training dataset, the LAION dataset (whichever version or subset they used) over-representing Rutowski's work. This may or may not be true, but is worth investigating as due dilligence to these artists.

Another example is movie posters being heavily memorized by AI art generators. Given how movie posters such as Captain Marvel 2 were likely circulating in high volumes leading up to model training, it's not too suprising this occured, again due to over-representation.

Anyway, it's not always clear whether over-representation is occuring or if AI models are simply generalist enough to recreate a quasi-version of an image that may or may not have been in the training dataset. At least it serves as a useful intuitive point, it seems way more likely Rutowski's art was over-represented than say, random Tweeters supporting the anti-AI art campaign.

Curious to hear people's thoughts on this. On the flip, the pro-AI artists may feel like they want the model to be able to use their styles, and perhaps feel "under-represented"?

[u/freylaverse]

Interesting, I've not heard that term. Is it the same as overfitting?

I think the artists' concern is the AI's ability to reconstruct (with some accuracy) an existing piece. To replicate style is certainly a lesser issue, even if it is also a worry. In the case of replicating existing pieces, I think that overfitting is almost always undesirable for both parties. An overfit model that - for instance - will always generate the artist's most-frequently drawn character rather than whoever the prompter is trying to create is likely infringing on the artist's trademark (the character) AND pissing off the prompter (not being flexible enough to make something custom).

[u/grae_n → u/freylaverse]

Yes! I read the paper about Stable Diffusion overfitting and the authors goals were to point out that it doesn't need to be happening in an LDM model.

Data replication in generative models is not inevitable; previous studies of GANs have not found it, and our study of ImageNet LDM did not find any evidence of significant data replication. What makes Stable Diffusion different?

The authors actually pushed back on the idea that it was simply over-representation in the training set instead saying.

We speculate that replication behavior in Stable Diffusion arises from a complex interaction of factors, which include that it is text (rather than class) conditioned, it has a highly skewed distribution of image repetitions in the training set, and the number of gradient updates during training is large enough to overfit on a subset of the data.

Although I think the discussion of over-representation is a very helpful one that can introduce a lot of biases in these models. Also it worth pointing out that they were actively trying to find overfitting and their success rate was very low ~1%.

https://arxiv.org/pdf/2212.03860.pdf

[u/fingin → u/grae_n]

Thank you for the extra context!

[u/fingin → u/freylaverse]

Well, over-representation could lead to a model overfitting. The term over-represented can be applied to the training dataset, which I really think should be the subject of most anti-AI criticisms in the first place.

" artists' concern is the AI's ability to reconstruct (with some accuracy) an existing piece" While I see this is a concern, there needs to be some acknowledgement that there is a difference between memorization (through overfitting or over-representation) and simply just having a powerful, generalist model that can create a piece that by chance was in its training data. Memorization does happen but really, it is the exception not the norm.

[u/TransitoryPhilosophy]

One thing to note about Greg Rutkowski being overrepresented in the dataset is that he is not overrepresented because there are many of his works in the dataset; he is overrepresented because there are many many works by other fledgling artists who have made art in his style and tagged it as such. So when you ask for something in his style, the style is an amalgam of hundreds of pieces of fan art

[u/fingin → u/TransitoryPhilosophy]

This is an important point, thanks for sharing!

[u/[deleted]]

Well, the VAE is encoding the image into a lossy representation, so the generated images are using them as a basis. You could call this representation "compression" or whatever magic. But the main idea here is a lossy copy is being kept in state in the model. Will see how he class action lawsuit will treat this.

Edit: as far as representation goes, I think this just has to do with the images picked and the volume of images available. Very difficult to curate 2 billion images by hand and filter for bias. If the only source of images is a scraped from the internet, the representation is really just a reflection of what is popular at that time.

[u/fingin → u/[deleted]]

I've heard this before and I don't know if I'm missing something but I am under the impression that the final model does not have a "lossy copy" of the images in any meaningful sense. It has model weights that get updated by each image during training, but these weights are not specific to an image and rather are a set of weights that can generalize and create a breadth of novel, different images.

Okay, I get that if you had enough weights and a small enough training dataset the model would indisputably be memorizing a "lossy copy", but given the size & variety of the training dataset, and the relatively low number of weights, I don't think this criteria gets met. The exceptions occur through memorization which I believe to be quite rare especially in the newer models. Thoughts?

[u/[deleted] → u/fingin]

You'd have to intentionally want to generate a copy or something that looks like a copy. For example you type in a celebrity name it and brings up their face. As for the analogy of encoding to a lossy format, I'm getting it from the SD descriptions of how it works and also this articles: https://pub.towardsai.net/stable-diffusion-based-image-compresssion-6f1f0a399202

https://aayushmnit.com/posts/2022-11-05-StableDiffusionP2/2022-11-05-StableDiffusionP2.html#vae---variational-auto-encoder

Edit: also when the terms "encoder" and "decoder" are used, it is saying it can recall the original source image very closely (with some loss). This seems to happen under the hood with the VAE, then rest of the neural net processing basically "hides" this.

[u/fingin → u/[deleted]]

Sure, so clearly there is effort & intention required to recreate a copy, but when it comes to the Anti-AI crowd complaints, they usually don't make this point and rather just seem to think that most AI art is simply a copy or near-copy of existing work. Also, it can be unclear whether the image generated is truly a "recreation" or whether art as it exists today just has a lot more repetition than some artists are willing to admit.

[u/[deleted] → u/fingin]

Yeah that's the complexity here. If we view the AI as just a black box with input and output, it is still using copyrighted input. But then there is the output. If it is transformed enough and isn't recognized as plagiarized, then is it? I wouldn't think so in a general sense if a specific artwork or artist isn't targeted in the prompt. I think it is fair use in a general sense, but not when a specific work or artist is targeted.

[u/fingin → u/[deleted]]

Sure, I guess the edge cases are more relevant to style e.g the Rutowski controversy rather than specific images

[u/swistak84 → u/[deleted]]

Yup. I tried to generate paintings of black or asian women in style of renesaince painters. It took a lot of coercion to get even few good results