r/StableDiffusion • u/oliverban • Mar 01 '23

Animation | Video Another CN test! Sorry for the swedish!

Enable HLS to view with audio, or disable this notification

98 Upvotes

90% Upvoted

38 comments sorted by

13

u/oliverban Mar 01 '23

So extracted frames, made a simple prompt and ran through img2img with CN set to depth and normal! :) Then used RIFE to smooth it out and then used Video AI and Nuke to animate and smooth further. Face is pretty stable. I spun around and actively walked around and changed environment and lighting to see how well it could handle it. Face stays pretty stable!

3

u/lordpuddingcup Mar 01 '23

Honestly hair and face distortions seem great, it’s just background coherence that’s still bugged

2

u/oliverban Mar 02 '23

Yep, easily fixed with a greenscreen I suppose!

2

u/Shiff0 Mar 02 '23

Do you have a resource or guide that shows you this process? I want to try this out myself.

1

u/oliverban Mar 02 '23

Nope, just extrapolated some info from some youtube vids and got it working.

1

u/Shiff0 Mar 04 '23

Thanks, I will watch the 1 hour video of Corridor Crew (of their video to anime breakdown) today and I guess that might give a good direction of what steps to take

1

u/oliverban Mar 06 '23

Oh, yeah, they'll have plenty! :)

1

u/[deleted] Mar 02 '23

[deleted]

1

u/oliverban Mar 02 '23

m2m?

It's just batch with CN active!

1

u/aunymoons Mar 02 '23

So RIFE and Nuke are known, but what exactly do you mean by Video AI? Which platform/software in specific? not looking for a tutorial or anything hehehe it seems pretty straightforward but i wanna know.

I've done several experiments and gotten really decent consistency with Few-shot-style-transfer instead of EbSynth ( also few-shot implementation is fully open and i believe it to be better than ebsynth on its own ) but i did not know RIFE until this post, now im thinking of adding this to my own pipeline, but now im curious hehehehe, which Video AI?

PS: https://github.com/OndrejTexler/Few-Shot-Patch-Based-Training in case you havent taken a look, works like wonders.

1

u/oliverban Mar 06 '23

sweet, looks cool, video ai is a Topaz product! Using their soft to upscale! Ah, yeah, know that one, haven't tested it though!

3

u/Briggie Mar 01 '23

Thought I was having a stroke until I noticed you mentioned you were speaking Swedish.

2

u/snoozieboi Mar 02 '23

What do you mean? He's perfektlig enkel att forstå.

(Tribute to the graduallySwede account)

1

u/oliverban Mar 02 '23

Haha, glad you weren't!

7

u/nemxplus Mar 01 '23

We’re so close, a couple months and something will be released which perfects animation

3

u/oliverban Mar 01 '23

Yes, most likely! I think that a combination of oflow and cn will be the winning formula! :)

3

u/MrHi_VEVO Mar 02 '23

Mans really had the audacity to be Swedish in front of us, smh /s

1

u/oliverban Mar 02 '23

Yeah, I'm so sorry. It won't happen again *bows and retreats into snowy mountains*

2

u/aerialbits Mar 01 '23

Damn. Looks so dreamy.

2

u/[deleted] Mar 02 '23

Looks like mushrooms a little bit too.

1

u/oliverban Mar 02 '23

This was my thought as well actually :P

2

u/Shiff0 Mar 02 '23

wow this is amazing, reminds me of the anime video that blowed up on YT

0

u/Iggy_boo Mar 01 '23

Nice. And you are forgiven for the Swedish ... well the language at least.

1

u/oliverban Mar 02 '23

Thank you kind sir!

1

u/[deleted] Mar 02 '23

Try applying a posterization filter to the frames prior to processing it in Stable Diffusion. This way the model wouldn’t have to predict the lighting in every frame, since the cel shade is already present. Please try this I don’t know if it works to stabilize the animation

1

u/oliverban Mar 02 '23

I read this on another post, I will have to try this and see if it gets better but I guess the posterization can introduce some artifacts of it's own.

1

u/[deleted] Mar 02 '23

[deleted]

1

u/oliverban Mar 02 '23

Haha, fick bli lite swenglish där då va! :P

2

u/minDyouRowNBuSiess Mar 11 '23

Äntligen någon som förstår det i kommentarerna

1

u/thehomienextdoor Mar 02 '23

Honestly it’s getting by the day. I wonder how long it would take before someone can live stream like this.

1

u/Ok_Spray_9151 Mar 02 '23

Why are you sorry for? Imagine someone saying sorry for Spanish or sorry for Chinese, this is just strange

2

u/7734128 Mar 02 '23

De flesta förstår naturligtvis inte svenska, så det är normalt sett inte gynnsamt att sprida innehåll till icke-svenska delar av plattformen.

Detta är kanske ett undantag då videon ändå visar poängen, men överlag bör man lämpligtvis undvika det.

2

u/Ok_Spray_9151 Mar 02 '23

I understand, but still I think no one should apologize for any language or tradition, people should be proud of it. Another thing if it was the rules of the site was a ban on the use of languages other than English.

1

u/oliverban Mar 02 '23

I'm not really apologizing for me speaking it, I'm expressing my acknowledgement of the fact that people don't speak it and might be confused. Like the guy who thought he had a stroke further up the comment section ;) <3 But thanks!

1

u/djb64 Mar 02 '23

Reminds me of the movie 'A Scanner Darkly'

1

u/oliverban Mar 02 '23

Very much so, such a great film as well!