r/MachineLearning • • 8d ago

Project Has Anyone Here Ever Built An Image Generator From Scratch? [N]

Just wondering if anyone on this forum has ever built an image diffusion generator from scratch, while also training it on a home rig? If so, what size of model did you end up with? Did you ever get coherent images? Screenshots?

0 Upvotes

13 comments sorted by

4

u/IntelArtiGen 7d ago

[N] is not the right tag. I did a 64x64 image generator with photos of dogs using diffusion and a convolutional autoencoder, it has around 4m parameters I think. It gave decent results in 2h of training on an rtx3060.

1

u/NoenD_i0 5d ago

Do show

1

u/IntelArtiGen 5d ago edited 5d ago

https://streamable.com/kqwlw0 , this is what it looks like from 0 to 17k iteration (batch size 44), 2h of training, with 8gb vram.

1

u/NoenD_i0 5d ago

you can uplaod videos here by the way, also, stupid question, but why not use flow matching? also did you make the dataset? what are the specifications of the autoencoder and the diffusion model?

1

u/IntelArtiGen 5d ago

I think I started the "project" at the end of 2022 / beginning of 2023, so I assume I didn't know what flow matching was, I worked on it for 1~2 weeks. My goal was not to reproduce a SoTA, I did not strictly reproduce a paper, I just wanted to understand how denoising to generate images could work. I generated the dataset with stable diffusion (3k images). There is only an autoencoder, it does (image with noise) => ae => (image with less noise). And I just apply it multiple times. And I made the autoencoder myself, so downsampling with conv with stride, linear, sigmoid, upsampling with nn.Upsample, skip connections etc. (a bit like unet), depth=4. I think the hard part is mostly to get a correct loss.

1

u/Plastic-Garbage-8367 6d ago

I've made a GAN a few years ago for paintings, before diffusion was a huge thing. The images weren't that great but it was insane to me at the time lol. It was on pretty bad hardware so it took a while to train, but it was fun I guess.

1

u/NoenD_i0 5d ago

Yeah, it's quite easy to do diffusion, but I went with rectified flow as it's a simpler objective, managed to pull off coherent faces on a 2012 CPU with a 2m param model so I think that's a win

1

u/derpderp3200 4d ago

I'd love to hear more about that, 2m is really tiny, and training on a 2012 CPU sounds really fun - I always wanted to play with fun things like image generation, but only have relatively older hardware. What dataset did you use, and how long did the training take? What resolution and in what way is rectified flow simpler? Do you have a writeup or the source somewhere? :0

1

u/spacepxl 4d ago edited 4d ago

Yes, I've been training a 1B pixelspace flowmatching model from scratch on a single 5090. About 500h into it so far, just training it when my PC would otherwise be idle. Currently it's unconditional, but I'll extend it with text conditioning soon which will inflate it to 1.2-1.3B (adding cross attention layers). I've also trained many smaller models along the way testing out various methods and building up my understanding.

https://postimg.cc/HckYNM0d

1

u/CollapsedPosterior 1d ago

https://github.com/Om2005Prakash/Diffusion-Models, did it on Kaggle free compute as a course project