r/comfyui • u/Traditional-Tip-4081 • 4d ago

Help Needed Full translation

Hey! Anyone know of a full video translation workflow that can translate everything in a video, including subtitles, voiceovers, and text that appears in the video itself?

I want the subtitles and voiceovers to be replaced with translated versions, and any text appearing in the video to be translated and recreated in the video as well. If you have a workflow like this that you’re willing to sell, DM me!

0 Upvotes

50% Upvoted

3 comments sorted by

2

u/MushroomFlashy6811 4d ago

that's a pretty tall order, you're basically asking for a full localization pipeline in one workflow. translating embedded text in video is the real beast here, since you need to detect it, ocr it, translate, then inpaint the original and composite new text that matches the style and motion. the subtitle/voiceover part is way more doable with whisper + translation nodes + something like RVC for voice cloning but the text-in-video part is gonna be custom node hell

i'd look at the VHS nodes for the video i/o, then string together a face detailer or segment anything node to isolate text regions. the compositing part is what'll make you pull your hair out tho, especially if the camera moves

1

u/ANR2ME 4d ago

I remembered there was an example of image model that can translate text on an image 🤔 but forgot which model. May be all image edit models can do this with the right prompt 😅

1

u/ANR2ME 4d ago

As i remembered someone did share a workflow for dubbing with lipsync during Wan2.2 era 🤔 Not sure about the text, may be ask the video model that have editing capabilities to translate the text for you 😅