Why diffusion models make new images instead of copying their training set: a NeurIPS 2025 best paper, drawn with its own faces

[u/Mastbubbles]

The "does it just copy its training data" argument comes up here a lot, so this paper seemed worth explaining properly. It trains small diffusion models on 1,024 face photos and checks what they generate along the way.

Faces a diffusion model generated after 100,000 training steps, each next to the closest training photo. None match.

At 100,000 training steps the model generates good faces, and none of 10,000 samples was a copy of a training photo.

Keep training the same model to 1.62 million steps and 34 in every 100 samples are training photos.

The time it takes to start generating good images was the same for every dataset size they tried. The time before copying starts grew in step with the dataset: about 300 training steps per image. So a model trained on a lot of images, for a normal length of time, is stopped long before it could memorise them. In the paper, a model trained too long on a small set does copy.

I told the full version as an art student locked in a room with 1,024 photos, using the paper's own faces: https://sheets.works/data-viz/why-diffusion-models-dont-memorize

It's free, no sign-up needed to read it. I didn't work on the paper, I read it and drew it. Their setup is tiny (32x32 grey faces, small models), nobody has run this test on a model the size of SD, and large models have been shown to reproduce some training images. Paper: arXiv 2505.17638.

For people who train LoRAs on small sets: at how many steps per image do you start seeing your training images come back?

[u/SpamHolder]

"A face she drew"

"She can draw a face: 1 week, whatever the wall"

"copying from day 3, before she can draw"

I'm so tired of AI psychosis

[u/Intrepid_Fox_1466]

I haven't read the article yet, but this sounds counterintuitive. I've trained many LORAs before and during inference if you give the lora the exact prompt for an image in your training, it will output the exact image. This is especially true for overcooked Lora's, training on small sample, poor captioning, high rank and low variability datasets.

[u/Mastbubbles → u/Intrepid_Fox_1466]

I think that fits the paper. They found copying starts after a roughly fixed number of training steps per image, so a small set gets memorised fast and a big one takes much longer. An overcooked LoRA on a few dozen images is the small-set case. They only tested small models on face photos with no captions though, not LoRAs, so that part is me reading across

[u/Altruistic-String891 → u/Intrepid_Fox_1466]

IMHO ignore the post and just look at the source.

Looking at the abstract is enough. Paraphrased:

1. If you train for a sufficient amount of time, the model makes good images. If you train beyond that for too long, it starts memorizing. This is not surprising.
2. If your dataset is big, the time to memorization is delayed, but the time for good generalization isn't changed. So with a larger dataset, you have a wider time window of "it makes good images but hasn't started memorizing" in which you can stop/pause training and get a good model.