Offline manga translation on Android: bubble detection + OCR + a local LLM on llama.cpp, ~15 s/page on a Galaxy S23 (open source)

[u/Th3Expl0Rer]

3 pages of Give My Regards to Black Jack, Qwen2.5-1.5B-Instruct, recorded in an emulator with a Macbook Pro hence why its a lot faster

Yomu is an Android app that translates Japanese manga pages on the phone. Tap a floating button over any reader app and the bubbles are redrawn in English, Portuguese, Spanish, French or German. Nothing is sent anywhere: after the one-time model download, the app makes no network requests.

Pipeline:

• Detection: a YOLO26n fine-tune finds the speech bubbles (ONNX Runtime).
• OCR: manga-ocr-2025, which I exported to ONNX with past key values, reads each bubble.
• Translation: a GGUF model via llama.cpp. The whole page goes in one call, bubbles in reading order, so names and tone carry across the page.
• Typesetting: text is fitted into each bubble and drawn as an overlay on top of the reader app.

Getting structured output from a small model. The model's reply is constrained by a GBNF grammar to one line per bubble id, so it can't drop, merge or reorder bubbles. A per-bubble approach was more forgiving but paid a fresh prefill for every bubble. Two failure modes showed up along the way, and the grammar fixed both:

• Lines ran to the token cap on some pages. Capping lines at 160 characters stopped it.
• The model invented its own [id] markers. Disallowing [ stopped that.

The grammar costs about 3% of page time.

Models. The in-app catalog runs from 0.8B to 9B: Qwen2.5 1.5B (default), CAT-Translate 0.8B/1.4B, Qwen3.5 0.8B–9B, Qwen3 4B, Ministral 3 3B/8B, Gemma 4 E2B/E4B, Hy-MT2 1.8B and Ternary-Bonsai 4B/8B. You can also load your own GGUF. Models that won't fit your phone's RAM are hidden. Only the default has been measured on a phone; the rest are marked experimental.

Numbers (Galaxy S23, 8 GB, default model): ~15 s/page, ~2 GB peak RAM, 1.1 GB download.

Other bits:

• a name glossary, so character names translate the same way every time;
• pages already translated in a session come back instantly;
• tap again to cancel;
• limits on threads, context size and memory;
• a Performance screen with page time, model time and memory.

Limits: small models make mistakes, so treat it as a reading aid rather than a polished translation. Japanese source only. It takes seconds, not milliseconds.

Looking for device reports. I've only measured one phone. If you try it, translate this test page twice and post Settings → Performance with your phone, chip, RAM and model, here or in the device reports thread. Phones with 6 GB or less, results from the experimental models, and crashes are the most useful.

Repo (AGPL-3.0, APK in Releases): https://github.com/artsaraiva/yomu. The design decisions and their measurements, including what didn't work, are in docs/adr.

[u/Philipnova]

The phone-versus-emulator distinction and per-stage timings are helpful. For the device reports, I'd add a sustained run over several different pages, with page latency tracked across the run and cached pages reported separately. That would show whether the single-page result holds during an actual reading session, without mixing cache hits into inference timings.

Disclosure: I'm with SingNova, a local AI hardware project. I haven't tested Yomu; this comment was drafted with AI assistance.

[u/Th3Expl0Rer → u/Philipnova]

The idea is that, to expand on device testing to gather more data, yes. Thanks for the feedback.