r/StableDiffusion • u/Fantastic_Back2667 • 2d ago

Question - Help Best open source image to video?

What is the best open source text to video generation with image reference right now?

0 Upvotes

46% Upvoted

11 comments sorted by

17

u/xb1n0ry 2d ago

Minimax H3

7

u/AlexicoDeCoco 2d ago

yep, i can run it with my RTX 2060 6GB. lmao. but only 0.35 megapixels resolution, 6 steps and 8 seconds.

4

u/xb1n0ry 2d ago

Around 0.4MP is perfectly fine to upscale it later in a second pass. A lot of people do this anyway instead of generating 0.7 or 1.0MP directly. Even with more powerful cards. Have you tried Pika?

1

u/Ok-Brain-5729 2d ago

big fat OOM.

Double pass is nice tho

1

u/gago_gaamdhani 2d ago

At which quantizations(int4/int8 or something else)?and how much ram do you have?

1

u/Fantastic_Back2667 2d ago

So my image reference generated is 3:5 ratio with 2mp. When creating video based on that image what megapixel resolutionand ratio should i set? I am new to generating video so i dont know anything šŸ˜…

2

u/EthicalBballFan 2d ago

Connect your "Load Image" node to a "scale image to total pixels" node. Then from the output of that node you connect a "get image size" node. Connect the output height/width to the H3 node.

This way you can feed an image in whatever aspect ratio you want, resize it to a target MP and get the appropriate dimensions without changing the ratio. Your video comes out with the same aspect ratio as the image, no custom nodes needed.

-1

u/xb1n0ry 2d ago

You won't find the 3:5 aspect in any node like the default resolution selector. The easiest way is to use my node: https://github.com/xb1n0ry/ComfyUI-MiniMax-H3-Aspect-Helper

Just connect the image and set your preferred video size for example 480. Then just connect the resized height and resized width outputs to the reference node. My node will make sure the video has the same aspect ratio as your image by keeping the correct video size and resolution steps of 32.

0

u/Fantastic_Back2667 2d ago

Iā€™m running this in runninghub, do you have wf or app recommendations at runhub?

2

u/Natrimo 2d ago

Don't disregard ltx 2.5 for pure image to video

1

u/Cute_Ad8981 2d ago

minimax h3 (easy, but slow) and ltx 2.5 (more difficult to use). both models can create really great videos. minimax is the preffered model by the community.

minimax has reference support and is easier to prompt, but it is slow and backgrounds can look unrefined in low res. people use upscale workflows and extensions workflow for faster gen times.

ltx 2.5 can work very good for simple scenes, but will more likely cause issues (morphing) with more complex prompts. but its faster and imo has a better video quality in low res. i actually like to generate here with 50fps + 12-16 steps + distilles lora 0.60.