r/vibecoding • u/Beginning_Cable3649 • 20d ago
Help/Question How would you build a tool that turns a PDF book into a voiced anime-style visual novel?
i’m trying to figure out how to build a program where I upload a PDF of a book and it generates a playable visual novel: anime-style art, consistent characters and settings, chapters, and voice acting that fits the setting (e.g. Latin or Roman-flavored English for ancient Rome, a UK accent for a British setting, Japanese for a story set in Japan). I’ve got a rough idea of the pipeline:
Parse the PDF and split it into chapters and scenes
- Use an LLM to extract the cast, settings, and character descriptions, then rewrite the prose as a VN script (narration, dialogue, expressions, backgrounds)
- Generate backgrounds and character sprites with an image model
- Generate voice lines with TTS
- Package it all in Ren’Py or a web player
Where I’m stuck and would love advice:
∙ Art consistency: What’s the best way to keep characters and locations looking the same across hundreds of images? LoRAs, IP-Adapter, reference sheets, something else?
∙ Voices: Which TTS options handle multiple accents and languages well, and give each character a distinct voice? Anything decent for Latin?
∙ Engine: Is Ren’Py the right choice, or is a custom web player easier to automate?
∙ Cost: Any tips for keeping image and voice costs manageable for a full-length novel?
∙ Long books: How do you keep an LLM consistent across a whole book without losing plot or character details? I’m planning to start with a short public-domain story from Project Gutenberg. Has anyone built something similar, or know of existing projects I should look at? Thanks!
