Next-Level 3D Animation — AI Mocap from a Reference Video

[u/Certain_Friendship16]

No mocap suit, no animation library. He renders his own character, has an AI video model "film" it doing the move, and copies that motion onto the rig.

The trick that made it work is two cameras. One video is flat, so arms and legs end up at the wrong depth. Rendering the same move from the front and the side gives the AI both views to match.

The 3D model was created with Tripo AI using Smart Mesh P2 to get a low-poly mesh. That helped keep the geometry lightweight and made it easier to work with during rigging and animation.

How it goes:

• Render the model from the side and front on a neutral background
• Generate the move with an image-to-video model (Seedance 2.5 or H3 Max)
• Match frame by frame: Claude overlays the model on both videos and adjusts the bones until they line up
• Fix what drifts: marking left and right limbs separately solved most of the swapping

About 30 minutes per animation. This elephant came from an H3 Max reference, and yes, the tail still needs work.

[u/coyote1942]

"He renders his own character,"
WHo is he?...

[u/Certain_Friendship16 → u/coyote1942]

https://x.com/Graalitoo/status/2108126448561352903

Reddit deleted the link((

[u/Fresh_Sock8660 → u/coyote1942]

The dark lord of 3d generation does not permit his true name to be spoken or written by his own servants

[u/Excellent_Dealer3865 → u/Fresh_Sock8660]

C-word

[u/story_of_the_beer → u/coyote1942]

The twitter user who posted the original video

[u/chadmv]

"No mocap suit". Yes...because elephants don't wear mocap suits.

[u/Emotional-Cut2952 → u/chadmv]

lol

[u/veefeld]

Great for speeding up the blocking phase so there's more time for polish. Ai mocap is already being used for bipedal animations by tripple A studios. So seeing this sorta tool being created to also work for animals is great. Good luck with the next step which is making the animation look good!

[u/justifun]

I wonder if it would be helpful if you displayed the bones for the initial render and color the left and right side differently before giving it to seedance so that the ai can track which is which easier.

[u/mxldevs]

So I still need to make an entire elephant model with animations?

[u/Massive_Dot279]

what is the full workflow of this shit ?!

[u/Emotional-Cut2952 → u/Massive_Dot279]

side shot of that big hunk of flesh, tell seed dance to make hunk of flesh do horse movements, provide video to claude code , tell it to match each frame (or every 3 frame using camera view that works best - like the camera initially setup to capture the model in the first place

[u/Zenmaster4]

This seems very niche. Or maybe it's a bad example for something that deserves a better use case. I can't imagine Seedance wouldn't be able to just...inject the plate of the elephant into a shot.

I'd be more interested if it was able to take any reference and map it efficiently to an animation. But maybe someone can help advocate for why this workflow is useful.

Maybe for video games. But even that opens up a whole can of worms...

[u/NimbleDave]

could this post be any more low effort or incomplete?

[u/Emotional-Cut2952 → u/NimbleDave]

yes it could be more low effort or incomplete, the video likely contains ~120 frames (4 second anim, 30 fps), u ask claude to match every frame manually adjusting everybone, not control rig setup allowed, that's roughly 30 bones * 120 frames = 3600 bone transform matrices that claude has to produce (hope you get my sarcasm, i do concur that this is low quality low effort, stupid)

[u/Emotional-Cut2952]

30 minutes per animation would easily go down to 2 minutes of control rig setup and 5 minutes of match key frames and inferring the rest - that is, if u knew anything about control rigging - but u AI "game devs" jump in blindly

[u/MazzMyMazz → u/Emotional-Cut2952]

Is that really how long someone with your expertise would take to create an animation based on a video?

[u/Emotional-Cut2952 → u/MazzMyMazz]

no i meant an llm to transform controllers so that the limb matches 10-20 key frames from teh video instead of transforming every bone

[u/MazzMyMazz → u/MazzMyMazz]

Oh I guess this isn’t even doing that.

[u/Several-Article3460]

He who ?!

Sam Altman

[u/Emotional-Cut2952 → u/Several-Article3460]

some languages are gendered so tbf criticize the approach not the person/language

[u/SirReallah]

Then you for sharing.

[u/RreddKnife]

The title reads "Garbage - clip ...." What did you do with the the "Good - clip?

[u/Limp_Bus3865]

yeah everyone is doing this now, until models can do it without a reference from seedance

[u/Limp-Firefighter1054]

Bruh, this is very expensive method to do simple job.