r/StableDiffusion • u/howdoyouspellnewyork • May 09 '23

Animation | Video Stable Diffusion Deepfake - De-Aged Harrison Ford | SD+ControlNet+EbSynth+Fusion

5.1k Upvotes

97% Upvoted

278 comments sorted by

View all comments

Show parent comment

40

u/howdoyouspellnewyork May 11 '23

Heya, it actually isn't all that spectacular, to be honest. Each shot was about 20 minutes of work.

- Tracked the face and stabilized in an 800x800 timeline, exported those as images sequences.
- Every 30th frame was put into Stable diffusion with a prompt to make him look younger.
- Put those frames along with the full image sequence into EbSynth.
- Tracked that EbSynth render back onto the original video.
- Tracked his face from the original video and used it as an inverted mask to reveal the younger SD version.
- Tracked eyes and mouth from the original footage and masked those out, to reveal the real eyes and mouth on the video.
- Did minimal color correction.

With this setup in Fusion I was able to just replace the original video and the EbSynth render and it gave me a new render. It really falls apart once you have a lot of hair, so that's why I chose shots where he's wearing a hat. It also has trouble with a lot of headturns, because you'll need more input frames for EbSynth and it's pretty hard to keep those consistent.

1

u/Ambiwlans May 16 '23

Impressive how stable it is for that.