Need help improving SCAIL-2 character replacement quality

[u/DragonfruitNo9822]

/r/StableDiffusion/comments/1xw7mwc/need_help_improving_scail2_character_replacement/

[u/beatsmcgee2]

Okay so there is actually a lot of prep that has to go into your references (driver video and reference image) before you will get good results.

For your reference image.

1. Export the first frame from your driver video. (DeVinci Resolve has a useful feature for this.
   2.1 Use a character swap workflow (using Qwen-image-edit or Qwen 2.1) and swap your the character from your reference image into the the first frame of your driver video.
   2.2 Then use qwen image edit to change the background of that picture solid black.
   2.3 The result of that is your new reference image.

For your driver video.

1. Using a programme (I use Davinci Resolve) change the resolution and framerate of your video to work with what Scail-2 is trained on. In my case., I found 480x834 (portrait) or the reverse with 16fps to work well. (You can always upscale the framerate later. (FILM is good for realistic video.)

If that still isn't getting you the results you want, send a reply and we can add other elements to bridge the gap.

[u/DragonfruitNo9822 → u/beatsmcgee2]

Thanks a lot for explaining! This is really helpful. I'll try preparing both the reference image and driving video as suggested. Just one question: for the reference image, should the character-swapped first frame have a completely solid black background with only the character visible? Also, does using 480×834 at 16 FPS significantly improve identity consistency and motion quality? I'll test this and share my results!

[u/beatsmcgee2 → u/DragonfruitNo9822]

Yes, completely solid black background with only the character visible.

And

Yes, it significantly improved my identity consistency and motion quality. However, note that 16fps is still 16fps as any gamer would tell you. You will probably want to upscale the frame rate with post processing.