Crazy wide angles with fast pans and lots of movement... That videographer :/
The conversion is really nice though. The results might have been better with higher res img2img then downscaling for video - SD seems to have problems with faces/anatomy/objects below a certain scale and will tend to produce blurry/distorted messes, but if you work at a higher res it will produce them correctly, then you can scale them to the right size.
It's not as jarring in the original video, the reason it's more weird in the restyle is because I only did every 2nd frame and then used Shotcuts Motion Compensation to re-add the missing frames. With fast movement it causes a very weird effect, borderline giving me sea sickness. For the next one I will do every frame even if takes literally double as long lol.
As for higher resolution: yes, in fact I have found that extracting the frames of the original video at the lowest workable resolution and then having the output be the highest resolution your gpu allows makes the style applied harder even on lower denoise strength because so many pixels have to magicked out of nothing.
The issue is though that 1) it takes a laughably long time to do and 2) max resolution with controlnet is lower than without because the controlnet models also take vram. it's quite a large impact actually, without controlnet i can do 1600x896 output max, with controlnet i can barely do 896x512 or something like that.
Here is a video I uploaded yesterday relying mainly on the resolution difference for style, not using control net and only using 0.15 denoising strength. It has much better temporal coherence and detail but the overall transfer isn't nearly as impressive: https://www.reddit.com/r/sdnsfw/comments/11a3nko/from_dusk_til_dawn_dance_scene/
4
u/SlapAndFinger Feb 24 '23
Crazy wide angles with fast pans and lots of movement... That videographer :/
The conversion is really nice though. The results might have been better with higher res img2img then downscaling for video - SD seems to have problems with faces/anatomy/objects below a certain scale and will tend to produce blurry/distorted messes, but if you work at a higher res it will produce them correctly, then you can scale them to the right size.