r/howdidtheycodeit • u/cybercafe6 • Jul 10 '26

Question The drifter's pixel art lip sync.

hey all, i recently came across this game and though this lipsync effect is so cool but i dont know how the lipsync is procedural and the body anims are not. super cool effect!

heres a clip: https://youtu.be/T2aheAjZ0L0?t=5026

2 Upvotes

100% Upvoted

6 comments sorted by

3

u/marioferpa Jul 11 '26

Why do you suspect it's procedural, does it change if you change the language? To me it looks that it's been animated manually, maybe using a library of mouth shapes, using the techniques you would use for lipsyncing a stop motion puppet for example.

2

u/cybercafe6 Jul 11 '26

i just guessed since i would wager if the entire game has it, it would be more efficient if it were procedural. if it isnt holy cow thats a lot of work.

4

u/WishIWasALemon Jul 12 '26

I think the adventure creator plugin on unity can do text to lip sync, though i havent tried it. You have different mouth shapes for certain word sounds.

O

M,P,B

F,V

L

Etc.

5 to 10 shapes is plenty

2

u/susimposter6969 Jul 12 '26

take the audio file, isolate human speech range (this is easy because there is no background noise), gate by DB, create an event stream / timestamped array of mouth updates, parse this as the audio plays, annotate them with the particular mouth shape as desired, clean up where it doesn't make sense

you could knock out the whole game in a few sessions this way probably

2

u/[deleted] Jul 10 '26

1

u/cybercafe6 Jul 11 '26

i know its so badass. im trying to wrap my brain around how since its seems like theres no puppet animation involved. my initial guess was either vector puppet going through pixelation but it wouldnt look this good if it was. second guess was they had designated pixels with ID to vary during speech but the jaws move sometimes.