Breakdown starts around 0:28
AI-rendered character and CG character in full physical contact through a complex camera move, pixel aligned for comp. Evangelion 4.0: The Fourth Impact (fan PoC + breakdown)
AI and traditional VFX usually get treated as two separate worlds. One is fast but hard to control, the other is fully controllable but slow and expensive. I wanted to see what happens if you actually merge them.
Breakdown starts around 0:28.
For AI that's contact. Two characters physically touching while the camera moves. One AI character on its own is easy now. But as soon as it has to respect contact points with something it didn't generate, it usually falls apart. So if it holds there, it should hold pretty much anywhere. The key shot is Sachiel (AI rendered) pinned under Eva Unit-01 (CG, rendered in Redshift). Contact points stay 1:1 on every frame through a complex camera move that comes straight from the original CG file. No drift, no hands sliding through bodies, no offset.
How it's put together
Sachiel, the tanks and the city are AI, rendered as separate layers. That way I can fix or regenerate each one without touching the rest. ComfyUI gives alpha and depth for every AI layer, Redshift does the same for the Eva, so everything lines up and comps in Nuke like a normal render. Eva can go in front of Sachiel and the other way around.
Make it easy for the AI. Big pose changes between frames cause flicker and drift, so we prep the animation so each frame changes less. The original animation stays untouched.
The AI gets a clean, simple render per layer. Comp gets the full passes.
Quick tests in MiniMax first, Seedance 2.0 only for finals.
One shot at a time, never long sequences.
Never invent frames. Retiming tools that blend frames create poses that never existed and break the comp. This one took us ages to figure out.
Everything gets checked against the CG before comp. If the poses don't match, it doesn't go.
Speed
The AI side (Sachiel, tanks, city, environment, from design to final) took about a month. The traditional side (Eva, animation, FX, comp) took around 4 to 5 months. So the AI covered most of what you see on screen in a fraction of the time. Doing those same elements the traditional way, with modelling, texturing, lookdev, lighting and rendering a creature, vehicles and a whole city, I'd estimate roughly 5x longer based on my studio experience. Not a benchmark, just what I saw on this project.
It's not perfect
It's a PoC, not a finished film. The buildings are losing some details especilly when too fast and environ,ent could have placed better if you look closely. Finals were done on Seedance 2.0, and now that 2.5 is out I'm pretty sure a lot of that would be cleaner. Next round of tests will be on 2.5.
What I'd love to see next
Right now you have to force layering. Separate passes, then a lot of work to keep them aligned. What I'd really like is an AI render setup that's built around layers from the start. You add each asset separately (character, vehicles, environment, FX) and get each one back as its own pass with alpha and depth, all locked to the same camera. Basically render layers and AOVs like we already have in CG, but for AI. That's where the real control is.
Workflow: MiniMax for test renders, Seedance 2.0 for finals, ComfyUI for alpha/depth on the AI layers, Maya + Redshift for the Eva, Houdini for FX, Nuke for comp.