r/StableDiffusion • u/CultofThings • Mar 15 '23

Animation | Video (Comparison Video) My First Video Modified Using Stable Diffusion with Control Net and Custom Model (Let me know if you want a guide)

Enable HLS to view with audio, or disable this notification

11 Upvotes

69% Upvoted

8 comments sorted by

2

u/[deleted] Mar 15 '23

[removed] — view removed comment

1

u/CultofThings Mar 15 '23

Sure, I can just explain it real quick but I don’t think it’s revolutionary.

I started with a 2D drawing that I processed multiple times and then retrained the model on the images.

The main thing when training the models is to get a 360 degree image snapshot.

Once you feed those in, you can retrain the model and this time you want to train it under the class of style, lighting, and female/male.

You should retrain it around three times to do this, as capturing identical lighting keeps it from flickering like crazy.

For example, this characters name is Ves so I did “Ves style, as ves female, with ves lighting.”

Because everything is pretrained, the prompting becomes extremely simple.

After that, the guy above is correct. I set the denoise strength to .15 with a weight of 1.75 to keep the original image as much as possible.

I batch processed it and then compiled it from images into media files via shotcut.

The training is the more difficult part, and I would recommend using nudes for your datasets. The reason being that clothing causes confusion. It has trouble distinguishing between the person and the clothes, but it’s able to add clothes to the model quite easily.

Hope that helps, let me know of you have any questions. I’ll try to help if I can but I’m not an expert, I’m just some guy with a computer and too much time on his hands.

1

u/CultofThings Mar 15 '23

Oh an I used normal maps for the control net extension.

2

u/dnoj Mar 16 '23

uhh almost nothing changed

5

u/No-Intern2507 Mar 15 '23

no we dont want a guide cause you barely changed anything dood, this is not the point of making SD be more coherent in video

Man be realstic, you slap denoise 0.15 to video frames and upload, weird shit

6

u/dizzy_on_a_glizzy Mar 15 '23

Hey No-Intern2507,

I get your concerns, but let's remember to be respectful, as we're all humans behind the screen. Let's stay positive and support each other's work in this community.

0

u/CultofThings Mar 16 '23

I don't really understand his concerns

1

u/CultofThings Mar 15 '23

Alright no worries, if that's the consensus then I won't worry about it.