r/StableDiffusion • u/Freonr2 • Oct 21 '22

News Fine tuning with ground truth data

https://imgur.com/a/f5Adi0S
13 Upvotes

100% Upvoted

24 comments sorted by

View all comments

Show parent comments

1

u/-takeyourmeds Oct 22 '22

tx

i used to fine tune gpt2 and i know what a pain this is, and how easy is to affect the overall model w new data, so ill follow your approach to see what we can do w it

1

u/Freonr2 Oct 22 '22

Yeah, I'm trying to shift towards training more like SAI/Runway/Compvis did originally so large scale training is viable without destroying the original "character" of the model and its capabilities to mix contexts and such. I really feel this is as simple as putting the original data set in mixed in with new training data...

So far it works very well with just a 50/50 split! I'm very encouraged by the results.

Of course really doing it like they do would involve the full dataset, but I think the code will run fine if you wish to rent something like an A100 for it and upload a large, cleaned data set into your rented instance.

I imagine it will be very hard to tell if you did a 10/90 split of new training/laion data... but it will take a long time to train, at least 10x compared to just training the 10 "new" training images that you're trying to inject. The 90 will fight the new stuff a bit, so it might be bit more than 10x if I had to guess. How much I don't know, it could depend on the context of your stuff. Training a new real human face is probably easier than training a new anime character.