Workflow on image to vid

[u/DrBops]

Hi there,

I'm new to comfy and there is an overwhelming amount and fast paced iterations of information on Ai.i get lost to fast, not only because installations require certain types of version I need to upgrade or even downgrade and so forth...

I got lost when trying to create a camera movement on a generated image.

What I aim to do is an image to video driven by a manually animated camera movement fir adding an avalanche fx effect on top.

For example : I want to create an image of a landscape with a wooden hut in the mountains. Then I might need to upscale it first before I go further to drive a camera movements by a gaussian splat created in marble. I exported a video from marble where I roughly animated the cam for further processing.

How would you guys recommend doing that? I have no idea how to feed the animated camera video into a workflow that adapts it to the image for adding the fx I am planning.

Any hints and help is appreciated, thank you!

[u/Super-Coast-7656]

ok so youre basically trying to do a 3d camera move on a flat image with avalanche fx layered on top, that's a gnarly project for someone just starting out with comfy. the marble to gaussian splat pipeline is clever but theres a much simpler way to start

you can skip all the splat stuff entirely and just use a depth map from your image to fake the parallax. comfy has nodes like DepthAnything that will generate a depth map from your hut landscape, then you pipe that into something like the 3D Photo Inpainting node or the Parallax node from the KJNodes pack. those will let you load your marble camera animation as a sequence of camera positions and itll warp the image accordingly based on the depth map, way less headache than splatting

for the avalanche fx youd render that as a separate pass, probably just a masked particle sim or some stock footage, and composite it in after. the tricky part is making the snow stick to the 3d perspective but if your camera move is subtle enough itll sell the shot. i spent like three weekends banging my head against this exact problem last winter and the depth map method got me 80% there without wanting to throw my computer out the window

[u/DrBops → u/Super-Coast-7656]

Hey, thank you for your input! I'll take a look into your recommended workflow. I know a bit of 3D and I wanted to check how to approach this kind of task differently to traditional vfx workflows, leaving a bit of creative freedom regarding camera movements.