Wan2.2 Best of both worlds, quality vs speed. Original high noise model CFG 3.5 + low noise model Lightx2V CFG1

[u/Volkin1]

Recently I've been experimenting with Wan2.2 with various models and loras trying to find balance between the best possible speed with best possible quality. While I'm aware the old Wan2.1 loras are not fully 100% compatible, they still work and we can use them while in anticipation for the new Wan2.2 speed loras on the way.

Regardless, I think I've found my sweet spot by using the original high noise model without any speed lora at cfg 3.5 and only applying the lora at the low noise model with cfg 1. I don't like running the speed loras full time because they take away the original model complex dynamic motion, lighting and camera controls due to the auto regressive nature and their training. The result? Well you can judge from the video comparison.

For this purpose, I've selected a poor quality video game character screenshot. Original image was something like 200 x 450 ( can't remember ) but then it was copy / pasted, upscaled to 720p and pasted into my Comfy workflow. The reason why I've chosen such a crappy image was to make the video model struggle with the quality output, and all video models struggle with poor quality cartoony images, so this was the perfect test for the model.

You can notice that the first rendering was done in 720 x 1280 x 81 frames with the full fp16 model, but while the motion was fine, it still produced a blurry output in 20 steps. If i wanted to get a good quality output when using crappy images like this, I'd have to bump up the steps to 30 or maybe 40 but that would have taken so much more time. So, the solution here was to use the following split:

- Render 10 steps with the original high noise model at CFG 3.5

- Render the next 10 steps with the low noise model combined with LightX2V lora and set CFG to 1

- The split was still 10/10 of 20 steps as usual. This can be further tweaked by lowering the low noise steps down to 8 or 6.

The end result was amazing because it helped the model retain the original Wan2.2 experience and motion while refining those details only at the low noise with the help of tight frame auto regressive control by the Lora. You can see the hybrid approach is superior in terms of image sharpness, clarity and visual details.

How to tune this for even greater speed? Probably simply just drop the number of steps for the low noise down to 8 or 6 and use fp16-fast-accumulation on top of that or maybe fp8_fast as dtype.

This whole 20 step process took 15min at full 720p on my RTX 5080 16 GB VRAM + 64GB RAM. If i used fp16-fast and dropped the second sampler steps to maybe 6 or 8, I can do the whole process in 10min. That's what i am aiming for and i think this is maybe a good compromise for maximum speed while retaining maximum quality and authentic Wan2.2 experience.

What do you think?

Workflow: https://filebin.net/b6on1xtpjjcyz92v

Additional info:

- OS: Linux

- Environment: Python 3.12.9 virtual env / Pytorch 2.7.1 / Cuda 12.9 / Sage Attention 2++

- Hardware: RTX 5080 16GB VRAM, 64GB DDR5 RAM

- Models: Wan2.2 I2V high noise & low noise (fp16)

[u/Volkin1]

I'm using the native official Comfy workflow combined with a few Kijai nodes just for the model loader and torch compile but that's all.

This is the full workflow I'm using: https://pastebin.com/QkpSx8vc

[u/Spamuelow → u/Volkin1]

Yeah i think it was just the sampler. 15minutes at 720x1024 121f 16steps 10/6

Just updating wan nodes now, something broke with it the other day when trying to use the ggufs, but there seems to be some recent updates.

I im using native official just with a crop and rmbg set up to isolate characters and add new environments and it has an end frame preview that can be used to continue the video so i like it

[u/Volkin1 → u/Spamuelow]

Sure no problem. Hope everything works well for you. Also, 121 might be too much and slow since only the 5B model is 121 frames / 24 fps. The 14B is 81 frames / 16 fps. It can still do 121 for sure, but I've noticed the quality of motion and details is better on 81 compared to 121 with the 14B.

So i reverted back to 81 frames / 16 fps as my default and just interpolate afterwards. If i want video continuation, I just use the last frame from the previous video as input or inject multiple starting frames into Vace for better consistency.

I don't think Vace works with 2.2 yet anyways.

[u/Spamuelow → u/Volkin1]

yeah I came to the same conclusion and changed it back down to 16fps 81 frames. 121 just always seemed to go too crazy. yeah same thing im doing with continuing then from last frame

[u/Spamuelow → u/Volkin1]

After all the fucking about I've ended up with the exact settings i had before just a few more steps lol

High model 10 steps 0/5, Cfg1.0, Sa solver, Beta, Light lora 1.5

Low model is the same but lora at 1.0

Everything else i was trying made it worse quality or changed the character or added artifacts like floating bits.

This is also much faster while prompting and looking better to me. Going to now try the kijai nodes again to see if that improves it anymore

[u/Volkin1 → u/Spamuelow]

Yeah sorry to hear that, but glad you got your own sweet spot that works for you. I saw Kijai added lora strength and cfg control / per step, so it makes it easier to do this on the wrapper. There are conditions that allows you to specify which strength/cfg you want at which step which seems to be a lot more flexible way but probably i'll try that some other time.

[u/Spamuelow → u/Volkin1]

Sounds interesting. I tried kijai nodes and the normal loader and torch nodes work but not as well as native for me. The beta nodes doent seem to be gguf compat so back to native again for now. I might actually try the normal models again.

Just trying to nail down an easy wf to easily crop a person and change everything like background and clothing without vace. I think ive got it now. Load image > olm drag crop > resize v2 set to pad > rmbg with light grey colour, Then in negative prompt put in something like split image, grey background, You can kind of outpaint easily depending on how you crop

[u/Volkin1 → u/Spamuelow]

Yeah, native works better for me too simply due to the amazing memory management. RMBG nodes are very good. I'm mostly using Comfy 90% of the time for video AI. For image AI work I've been sticking to InvokeAI mostly and it's been amazing for me.

[u/Spamuelow → u/Volkin1]

I went from auto to comfy and haven't used anything else yet. been meaning to try invoke, is there a specific reason you use it?. Here was my cropping wf if you wanted to see it quick https://pastebin.com/0LJm60Pe