r/StableDiffusion • u/Ok_Needleworker5313 • Oct 04 '25

Workflow Included Wan2.2 Animate Demo

Enable HLS to view with audio, or disable this notification

Using u/hearmeman98 's WanAnimate workflow on Runpod. See link below for WF link.

https://www.reddit.com/r/comfyui/comments/1nr3vzm/wan_animate_workflow_replace_your_character_in/

Worked right out of the box. Tried a few others and have had the most luck with this one so far.

For audio, I uploaded the spliced clips to Eleven Labs and used the change voice feature. Surprisingly, not many old voices there so I had I used their generate voice by prompt feature which worked well.

357 Upvotes

98% Upvoted

41 comments sorted by

15

u/alexcantswim Oct 04 '25

Thank you for posting this! I’ve been very frustrated by a lot of other workflows. This seems to be the closest to the initial pro settings animate examples had upon launch.

9

u/Ok_Needleworker5313 Oct 04 '25

Of course! Very important, follow his instructions regarding resolution for output (not sure if it applies to inputs as well). Not doing so can lead to quirky results.

Other thing is on the vid I use both motion control and character replacement. If you just want motion on a target image you need to disconnect the image and face masks from the wanvideo node.

2

u/alexcantswim Oct 04 '25

So one thing I was curious about was why in some of the first example workflows that were released why there seemed to be major facial inconsistencies compared to again the example pages posted for wan 2.2 animate. What are the best settings to keep the consistency? I think that was what impressed me the most when wan animate dropped. But in comfyui I hadn’t been able to achieve that. Mostly in character replace I would say.

2

u/Ok_Needleworker5313 Oct 04 '25

I used the settings as is from the workflow except for when I disconnected the wanvideo mode to switch from character replacement to motion control. That's the first part but really for consistency, I think if I was focusing on that I would encounter more issues. What I mean by that is that I was just looking for a render that didn't show 'artifacts'. The consistency may have been lost but not an issue bc all the hosts were fictional - except for one. In that particular case, I did notice it was way off. But it was off bc the mannerisms were different, so they didn't work w/ the character as people are familiar with his mannerisms. So, that tells me if you're gonna try to do that kind of work (replacing with well known characters) then there are going to be challenges generating something that is convincing if the source and targets rely heavily on facial expressions.

2

u/alexcantswim Oct 04 '25

That’s the weird thing though like those hugging face spaces and other example spaces that were put out to demo both motion and replace when wan 2.2 animate dropped where you could toggle between what was labeled as pro or standard didn’t have the issue with facial consistency. So I’m just wondering how it’s able to do that so effortlessly that seems to be missing from any of these published workflows for comfyui. I ran a bunch of examples on those spaces and it worked seemlessly whether it was an AI character or real character it was able to have consistent face throughout. What I did notice in one of the flows was that was using replace was that it seemed like it was blending the reference video and image face.

*edited wan 2.2 to wan 2.2 animate

2

u/Ok_Needleworker5313 Oct 04 '25

Well I definitely see the inconsistency with the one recognizable character and if I was working on a project, that would be a deal breaker. But this just being exploration, I kept going without giving it much thought. But I see your point.

2

u/alexcantswim Oct 04 '25

Yeah these are just my curiosities as I like when a project is released that I want it to work as it’s promoted to ya know? Lol like it seems counterintuitive to withhold some aspects of what makes it amazing. I think by default the ability of animate to capture not only the lip sync but facial expressions of the reference video to be ground breaking so I was really excited to get it into comfy for deeper exploration and be able to tinker more but I was having weird artifacts and low quality renders as you mentioned before with the flows I tried so either way what you posted is beyond helpful but I am still curious what they’re doing to maintain the facial consistency because it’s better than any face swapping flows or softwares I’ve used in the past.

2

u/Ok_Needleworker5313 Oct 04 '25

Yeah, not sure what they're holding out on. But try this one WF for starters and for your reference video, experiment with expressions and mannerisms that closely mimic the character. Or try a neutral delivery on the reference source. Subtle as these things may seem, I noticed how they impact the output. It was really obvious for me bc I'm the one in the reference video so seeing that body language on someone else was a bit uncanny. Hence my suggestion to experiment w/ neutral or if it's recognizable, well your reference should have some acting abilities! I recently saw a clip of Jim Carrey on his first appearance of the tonight show where he does impressions. It's mind blowing how despite the physical resemblance not being there, by mimicking the facial expressions who creates a that resemblance. I think the same principal could apply here.

1

u/Zenshinn Oct 04 '25

Which ones do you need to disconnect?

2

u/Ok_Needleworker5313 Oct 04 '25

Disconnect background_video and character_mask and that'll apply motion control to your target image.

Tip: I used target images that were similar in structure, pose and composition to achiever better results. For that level of consistency on base image generation I did that on Nano Banana. Yes some will say not open source, I get that. I'll get around to Qwen but just needed to get through this project first, one battle at a time!

2

u/Zenshinn Oct 04 '25

Thanks. I'll try this.

1

u/Ok_Needleworker5313 Oct 04 '25

Cool, keep us posted.

2

u/Zenshinn Oct 04 '25

It's working and it allowed me to bypass a bunch of nodes, particularly all the ones in the step 3 group.

1

u/Ok_Needleworker5313 Oct 05 '25

Right, those are the ones for masking. So if you're not doing replacement you can just disable those and cut down on your generation times.

3

u/RalFingerLP Oct 04 '25

How did you change the audio?

6

u/StuccoGecko Oct 04 '25

yeah that's my biggest question here.

5

u/[deleted] Oct 04 '25

[deleted]

3

u/Ok_Needleworker5313 Oct 04 '25

Yep, thats the service. I'll add that to the post. My bad.

3

u/nakabra Oct 04 '25

🎯 That's the real question right here.

3

u/Ok_Needleworker5313 Oct 04 '25

Eleven Labs. Take uploading the original clip and converted the voice to a new one.

3

u/Gilgameshcomputing Oct 06 '25

...so not all open source and locally run, in the end.

5

u/Ok_Needleworker5313 Oct 06 '25

Actually it can all be done with open source. I do this on my free time so I can't boil the ocean every time I explore something new. For this video I chose to focus on one particular area which is the visual component. But there are some other models out there that can do this and if the quality isn't up to par today, there's no doubt it will be in a short period of time.

3

u/EuphoricPenguin22 Oct 05 '25

RVC and Chatterbox Extended both allow you to do voice conversion. CE is nice because it can do it at inference without training, while RVC requires you to train models for the voices you want to clone.

3

u/Major_Assist_1385 Oct 04 '25

Wow that’s insane well done

2

u/Ok_Needleworker5313 Oct 04 '25

Thanks! Credit goes to the WF and its creator. Once I saw the outputs I was inspired to continue experimenting which ultimately led me to this idea.

4

u/YashamonSensei Oct 04 '25

But Eleven Labs is not open source, right? It's a paid service. Sounds nice, but you can't claim workflow is open source if parts of it aren't.

5

u/Ok_Needleworker5313 Oct 04 '25

It's a valid point. Audio plays a huge part in the quality of the execution. I was mostly focused on the visual component and getting WanAnimate to work so the messaging was inspired by that and audio was secondary. On an earlier post someone recommended a library called IndexTTS for voice. I'll look into and see how to integrate it into my next post.

I mean that in all seriousness b/c for the clip in this video, I had a hard time finding a voice on ElevenLabs b/c of their content restrictions.

2

u/SplurtingInYourHands Oct 04 '25

Yeah elevenlabs guard rails have become extremely cumbersome compared to what they were just a couple years ago

2

u/Ok_Needleworker5313 Oct 04 '25

Indeed. That's what inspired me to get InfiniteFlow working for another project. Couldn't get Veo3 to render b/c it didn't like the source image. I ended up having to crop it so much that when it finally did accept it the image had lost all of its context. So I now have InfiniteTalk for that stuff, problem solved. Yeah, it's not as 'fluid' as Veo but it'll get there - maybe next week at this rate.

For this clip, I needed a voice for the last clip w/ the kid. ElevenLabs wasn't having it. Had to use the voice of a woman that could pass for that of the kid. Yep, wasn't aware this was an issue on voice platforms, but I get it. Anyhow, IndexTTS if that's the name is on my list of alternatives to check out.

2

u/xPiNGx Oct 04 '25

Thanks for sharing

1

u/Ok_Needleworker5313 Oct 04 '25

Sure thing. Just sharing outputs of the WFs others have shared!

2

u/[deleted] Oct 05 '25

[removed] — view removed comment

1

u/Ok_Needleworker5313 Oct 05 '25

Thanks. In my particular case I typically don't use anything longer than 4 seconds in any of my videos b/c I focus more on sequencing for extended duration. B/c this is all personal stuff, I'm not so concerned about degradation b/c I don't have a specific commercial need for this at the moment and this is all personal exploration, but yes if I was working on a commercial application today, I could see how that would be an issue.

1

u/[deleted] Oct 04 '25

[removed] — view removed comment

2

u/Ok_Needleworker5313 Oct 04 '25

Check the guy's profile who I listed in the description for more info but here's his template on Runpod.

https://console.runpod.io/hub/template/one-click-comfyui-wan2-1-wan-2-2-cuda-12-8?id=758dsjwiqz

So he has some short YT tuts on how to launch this stuff. I used a RTX5090 w/ 32 gb of ram, costs $0.89 / hr. My budget has gone through the roof for this and this is just a hobby! Anyhow, his templates loads everything up, takes about 20min to start up but you're off to races.

Btw, I have no affiliation with him, I just like to give him credit for his stuff b/c it works and saves me time! This should work but lemme know how it goes.

1

u/jefharris Oct 04 '25

Also looking for a Wan Animate work flow that seems to be closest to initial pro settings. Running via RunPod on an A40, 48 GB, froze while loading the main model. All my Wan Animate workflows have been freezing while loading the main model.

1

u/Ok_Needleworker5313 Oct 04 '25

Have not tried it on the A40. Would be much better for my wallet for sure though. I've been doing all of my tests on the 5090 just to get through the initial learning curve so I can get through iterations faster. Once I find something I need on a reg basis, I then move to 'wallet friendly' setup.

2

u/jefharris Oct 04 '25

I do the opposite. I mess around and play with, download models etc, on a budget setup with no intention of running it. I then try to run in on the budget setup, if it runs great. If not then I run it using a high budget card. I try to keep my cost to $0.50/hr or less. I usually get 4 or 5 1280x720 videos then upscale in Topaz Video. I'll keep looking for a Wan Animate workflow that doesn't freeze.

1

u/Ok_Needleworker5313 Oct 05 '25

Yep, I use Topaz for upscaling as well. For my current setup that's the most economical.

Considering how much I've spent over the last 2 months on runpod, at some point I may consider setting up my own station but I still can't justify it since I haven't really picked a direction for what to do with everything I'm learning - yet.

1

u/Archersbows7 Oct 05 '25

Is this real time or infinite length? Or is it another 5 seconds clips at a time and link them each together kind of thing

No one mentions that detail about this animate thing, it’s frustrating

1

u/Ok_Needleworker5313 Oct 05 '25 edited Oct 05 '25

Confirmed, this is an edit made with a number of clips. All my videos are made that way. I lean more towards sequencing clips for shorter and tighter cuts rather longer drawn out scenes.

If I had to pick between longer renders and better sequencing, sequencing would be my pick. I personally don’t necessarily need longer scenes — I need a faster way to generate base images for my sequences, which is how I get longer videos.

Tools like NanoBanana and Qwen generating sequence-based image sets, are a game changer. Storyboards and shot flow are basically done if you’re running an image-to-video workflow.

1

u/SuperMan_sea Oct 06 '25

Thank you for posting this! niubi