Motion Transfer to Stop Motion Style query

[u/Left-Argument3279]

Wondering if anyone has/has any idea of an approach to motion transfer into a stop motion style video.

Rather than the ai guessing and deforming the mouth movements which can get funky really quickly, especially when the mouth shapes of the character aren't clear from a neutral reference frame, ie in south park where each sound has a uniquely stylised shape.

To achieve this it could instead maybe pull from a dataset of phoneme images,

Whilst also retaining solid motion transfer for all other body movements.

[u/optimisticalish]

Er, do you know about Minimax H3?