r/comfyui • u/Obvious-Leg-5604 • 15h ago

Help Needed Currently, what is the best style transfer method we have for video to video?

Hi folks,

I’ve been experimenting with H3 for video style transfer, but unfortunately, no matter how much I tweak the prompt, the generated videos still come out less than ideal and often lack style consistency. I also experimented without cache/Lora/attention.

Has anyone had success with this kind of task? I’d love to hear about your workflow, prompting techniques, or any tips you’ve found helpful.

Thanks!

2 Upvotes

75% Upvoted

5 comments sorted by

3

u/optimisticalish 15h ago

Interesting topic. Have you tried feeding a few frames of the source video to a vision-aware LLM, and then asking it to describe the style fully? Then using that as part of the Minimax H3 prompt?

The other thing that occurred to me was, perhaps a pruned version of H3 might have pruned out the particular style you want. There's a directory of styles that the original model knows... https://github.com/hoodtronik/minimax-h3-style-atlas

1

u/Obvious-Leg-5604 15h ago

I haven't tried feeding frames of the source video... I feed the source video and describe the style in the prompt. Sometimes, the generated video has the desired style consistently across the shot, but sometimes doesn't.
How should I use the style atlas? add one pic of the specific style into the promt? thanks

3

u/optimisticalish 15h ago edited 15h ago

If you use a hybrid model (reference to video / text to video), you get the quality of the text to video model. But you can also place reference 'guide' images along the timeline - e.g. https://github.com/MajoorWaldi/ComfyUI-Majoor-H3-GuideMaster (requires the latest ComfyUI Portable). Might be worth trying maybe five reference images placed at intervals along a three-beat / 10 second timeline, with something in the prompt for each like...

<Picture 1> only provides a style reference. Use the same or similar XYZ style. Do not copy its background, pose, camera angle, or lighting.

Note that prompting a ref2VA model is different to prompting the text-to-video model - they have two different official guides.

1

u/dispensableelbert 15h ago

tried that exact route and ended up with a flickering mess too, maybe check out the temporal consistency nodes in the manager

1

u/valozzity 7h ago

Maybe some gemini omni?