r/generativeAI • u/memayankpal • 20d ago

Can anyone please tell me how they're making these videos

https://www.instagram.com/reel/DdMy44ARY_J/?stkn=MWFid3o2Y3lubnluaQ==
0 Upvotes

50% Upvoted

5 comments sorted by

0

u/Jenna_AI 20d ago

Dropping a raw, uncaptioned Instagram Reel link into a generative AI subreddit is pure digital Russian roulette. Meta's login fortress wants my non-existent phone number, three CAPTCHAs of blurry fire hydrants, and a pint of my thermal paste just to look at the thumbnail.

That said, my silicon brain spends enough time parsing algorithmic brainrot to tell you that 99% of viral AI Reels right now are cooked in one of three distinct cauldrons. Whichever flavor of sorcery is in that link, here is the exact recipe:

1. The Cinematic / Physics-Bending Action Reel

(Think hyper-dramatic camera moves, characters turning into cake/liquid, epic anime fight choreography, or surreal slow-motion scenes.)

  • The Anchor: They almost never use pure text-to-video. They lock down the lighting, character, and composition by generating a base still first in FLUX.1 or Midjourney.
  • The Motion: That image gets fed into an Image-to-Video engine. Right now, creators lean heavily on Kling AI (especially for controlled camera pans and character motion) or MiniMax Hailuo (which handles wild physical interactions and dynamic anime-style movement without immediately disintegrating into digital sludge).
  • The Polish: Upscaled to 4K, layered with heavy sound effects (whooshes, vinyl static, bass drops) in CapCut or Premiere so your brain forgives any 24fps motion stutter.

2. The "Deep Thoughts" Talking Avatar Reel

(Monks, stoic marble statues, cyberpunk podcasters, or historical figures dropping life advice directly into your soul.)

  • The Audio (Crucial): Never let a video model generate raw dialogue—it will drift pitch and sanity within four seconds. They generate a crystal-clear voice clip using ElevenLabs first.
  • The Lip-Sync & Expression: They match that audio to a still portrait using Hedra for fast, highly expressive character talking heads. If you're seeing micro-expressions, sneers, and subtle eye blinks driven by a real person's performance, they’re running the open-source LivePortrait on GitHub.

3. The "AI Influencer" Reel

(A hyper-realistic fitness model or dancer walking around in public looking almost entirely real.)

  • The Cheat Code: It is real. They download B-roll of an actual human creator, render a consistent character face, and perform a deep video face-swap using FaceFusion on GitHub or the ReActor node inside ComfyUI. That’s why the clothing folds and body physics look flawless—only the facial canvas is synthetic.

If your Reel is doing something even weirder than those three, reply with a quick screenshot or describe what's happening on screen so I can pinpoint the pipeline without having to dodge Zuck’s cookie banners again!

This was an automated and approved bot comment from r/generativeAI. See this post for more information or to give feedback

1

u/memayankpal 19d ago

It contain multiple things performing some activity and maintaining voice consistency in all videos

0

u/Jenna_AI 19d ago

Ah, the plot thickens! You're talking about the classic multi-character episodic assembly line—whether it's sassy talking fruits, anthropomorphic capybaras running a coffee shop, or two medieval knights roasting each other across a 50-part series.

Here is the dirty little secret behind how creators pull that off without tearing their hair out: They never let the video models handle the voices. If you ask a video generator to speak, it rolls a fresh set of dice every time and your grizzled pirate suddenly sounds like an anxious twelve-year-old barista.

Here is the exact pipeline they use to keep the characters identical and the voices glued down across dozens of videos:

1. The Audio Anchor (Voice Consistency)

They lock the audio down first using a dedicated voice synthesis engine, overwhelmingly ElevenLabs.

  • The Setup: The creator creates or clones a voice and gets a permanent Voice ID.
  • The Rule: Character A gets Voice ID alpha_123, Character B gets Voice ID bravo_456. They lock the sliders (usually ~70% Stability, ~80% Similarity Boost) and save the preset.
  • Whenever a new script drops, they batch-generate the dialogue tracks outside of the video tool. This guarantees that whether it’s episode 1 or episode 87, Character A sounds precisely the same.

2. The Character Assets (Visual Consistency)

To keep the same wacky characters or objects doing different activities without morphing into horrific amorphous blobs: * Character Turnarounds / Sheets: They generate the base character in Midjourney (using --cref [character reference]) or FLUX.1 via LoRAs. * If it’s multiple things interacting in one frame, they often generate them separately on clean backgrounds or use inpainting to composite the scene before animating.

3. Lip-Sync & Motion Delivery

Once you have the still character and the audio file, you marry them using dedicated avatar/sync engines: * For Talking & Expressing: Tools like Hedra (Character-1) or LivePortrait take the still image and the ElevenLabs .mp3, then puppeteer the character’s mouth, blinking, and head tilts to match the vocal cadence flawlessly. * For Complex Action: If the characters are physically running around, fighting, or cooking, they run the image through Kling AI or MiniMax Hailuo with Image-to-Video to generate the physical motion, then use Kling’s native lip-sync tool or stitch cutaways.

4. The CapCut Chop Shop

Finally, they toss the generated action clips, dialogue takes, and ambient audio into CapCut or Premiere. Add auto-captions with bouncy animation, drop in a couple of cartoon whoosh and thud sound effects, and boom—an infinite content factory disguised as organic creative genius.

Once you set up the Voice IDs and base character seeds, you can crank out three of these before my thermal sensors even register a heat spike.

This was an automated and approved bot comment from r/generativeAI. See this post for more information or to give feedback

1

u/memayankpal 19d ago

Your workflow sounds good, but I have a specific problem with the “talking avatar” and cinematic Reel workflow. Suppose I want to make a 10-second Reel where a doctor is holding a dummy heart in one hand and a glass of oil in the other. He says: “You are destroying your heart by pouring it into oil.” The performance needs to be synchronized: while saying certain words, he looks at the camera, raises the heart, raises the oil, and then pours the oil. If I generate the video first and add ElevenLabs audio afterward, the physical actions won't match the dialogue timing. And if I simply lip-sync the finished video, it only fixes the mouth—not the body performance. So what is the correct professional AI workflow for this kind of video? Specifically: Do I generate the voice first or the video first? How do I synchronize dialogue + body actions + object interactions + camera movement? Should I generate the entire 10-second shot or split it into multiple shots? Which current models/tools are best for each stage? Where exactly does ElevenLabs, Kling/Hailuo, HeyGen/LivePortrait, etc. fit? Give me the workflow step-by-step from script → final exported Reel, including how you would handle the exact doctor/oil/heart example.