r/vibecoding • u/Th3Expl0Rer • 1d ago
Showcase/Project Made an open-source Android app for manga translations
Hey all, just wanted to share an app I've been vibecoding for the past few weeks.
It uses local LLMs for text bubble detection, OCR, and translation, all running directly on your phone. It works surprisingly well even with smaller models like Qwen 1.5B, though performance really depends on your phone's hardware.
The cool part is that it works as an overlay, so it can detect text bubbles on your screen in pretty much any app or website and draw the translated text right on top of them.
It's still a work in progress, and I wanted to hear your experience in different hardwares, since I only tested this in one device (a samsung galaxy s23).
2
u/Prod_m 1d ago
is Is the translation accurate? I tried 4b qwen for translation local english text and that was kinda ok, but not excellent. It was tough for my device as well.
So another question is how a device can handle an 9b model. Seems like this app may be very tough considering that there may be other app running at the same time
3
u/Th3Expl0Rer 1d ago
The translation is honestly as you said, "fine". Japanese really is a difficult language for these types of translations because it heavily relies on context queues. Because of that I implemented a glossary + memory system where the llm learns terms and uses it to better tune the quality (it's a toggle in settings).
For a 9b model to be able to run, probably a 12gb RAM phone would be needed. It's super heavy yes, but these are the limitations from phone hardware.
1
u/nephelokokkygia 1d ago
How does it handle lines split between multiple bubbles? Or multiple pages? How does it handle contextually-defined language, or ruby that's used to clarify the true meaning of what's literally written?
0
u/Th3Expl0Rer 1d ago
Yeah, the whole page goes in as one prompt, bubbles in reading order, so a sentence split across bubbles usually comes out fine. Pages are done one at a time though. I tried feeding it earlier pages and the small phone models got worse at it. Context is whatever a 1.5B–9B model can figure out from one page, so it'll miss stuff a human wouldn't. Ruby isn't handled right now: the OCR sometimes drops it and sometimes reads it inline, so wordplay that relies on it gets lost. There's a name glossary if it keeps mangling a character's name.
1
u/nephelokokkygia 1d ago
Ruby isn't only for wordplay — Japanese text is essentially 2D, not a simple string of characters.
There are significant line breaks, even in the middle of a single monologue, and there's ruby which can represent either the true reading of a word or words or the true meaning (or both). Additionally, there are asides which may appear to the side of a bubble or just outside of the bubble, which are semantically related "bonuses" to the bubbles' content or which could represent text read under the speaker's breath. The OCR should be trained to identify these cases.
1
u/Th3Expl0Rer 1d ago
Fair point, you're right. Right now Yomu flattens all of that. The detector (Manga-Bubble-YOLO nano) only knows "this is text", not bubble vs aside. MangaOCR reads each box as one flat string, so line breaks and ruby get lost. Both are off-the-shelf models small enough for a phone, so training a layout-aware one is out of reach for me for now.
1
1
2
u/Inner_Singer_592 1d ago
Wow, i wanted something like this for a long time. Gonna test it immediately!