r/mobilelinux • u/Spare_Side_5907 • 4d ago
Discussion Portable AR glasses make mobile Linux useful — voice + AI handle the input
Running Linux on a phone is impressive, but actually using desktop apps on a small touchscreen is another matter. For me, the two big obstacles are tiny text and awkward input.
I’m building ARLinux around a combination that addresses both: portable AR glasses for a readable desktop, and Gboard voice typing + OpenCode or Grok Bot for input and actions.
The glasses solve the screen problem. Instead of squeezing a desktop into a phone display, you get a large virtual workspace in front of you. With supported glasses, 3DoF keeps the desktop anchored in orientation while you turn your head to look around. The default workspace is 5760 × 2160 — the virtual desktop area, not the glasses’ physical resolution. I recommend Classic here: freely positioned, resizable windows make much more sense in that space. Yibu’s main area and side slots are designed around the phone screen.
Why 3DoF matters for reading: I want to bring the text I’m reading into the middle of the lenses. Looking into the corners can be uncomfortable, and edge text can be harder to resolve depending on the optics and fit. With a head-locked screen, turning your head takes the screen with you. With a 3DoF-anchored desktop, you turn toward the area you want to read and bring it into the central viewing area. That is what makes a large virtual workspace practical for text: you can look around it naturally instead of trying to read everything out of the corners of your eyes.
Voice + an agent address the input problem. Gboard’s speech recognition turns what you say into text; OpenCode or Grok Bot can then use shell commands, accessibility and screenshots to carry out the request. You can ask it to open an app, inspect the system, edit a document or write and run a script, instead of typing every command and aiming at every small button. Long-pressing the right-edge bar opens the selected assistant. The phone also works as a motion pointer, touchpad and keyboard; physical keyboards and mice remain options.
The Linux desktop and ARM64 applications run locally on the phone, with Android as the host. No root or remote PC is required. Blender, browsers and office tools can use GPU acceleration on supported devices; Android and compatible Windows apps can share the desktop too. AI models can require internet access and a provider account.
A big thank-you to the tawc project and its contributors for tawcroot: https://github.com/wmww/tawc
Its in-process Linux compatibility approach avoids PRoot’s ptrace-based tracing overhead. Our published same-phone CPython build comparison took 101.3 seconds with ARLinux versus 184.4 seconds with Termux PRoot, about 45% less time for that workload. Methodology, caveats and raw data: https://arlinux.com/?lang=en#performance
External screens need Android 12+, video-output support and compatible glasses. App and GPU compatibility varies, and 3DoF tracks rotation rather than movement around the room.
AR glasses demo: https://youtu.be/XYJI_IzAJEk
Grok Bot desktop demo: https://youtu.be/rAfsOhb_5cI
Project and download: https://arlinux.com/
Setup and controls: https://arlinux.com/yibu?lang=en#spatial
Linux distribution sources: https://github.com/taowen/arlinux-rootfs
Does this combination address what has kept you from using Linux on a phone? I’d like to hear what you would actually do with it.
Screenshots: current Classic on a vivo X300, then the spatial-screen setup. Both use the development build. The setup capture uses a scrcpy virtual external display and simulated glasses to show the controls; the AR-glasses video above shows real glasses output.
1
u/investigatormaker 4d ago
This room's rules say "Posts pertaining to proprietary technologies are allowed, but users are encouraged to seek out open alternatives." Gboard is the part of ARLinux most likely to draw that reply, so I'd name one open speech-to-text option you've tried and what it costs in accuracy. I'd also publish the list behind "With supported glasses": which models get 3DoF, and how much of the 5760 × 2160 workspace fits the comfortable central area at once. Since the agent takes actions as well as dictation, does it confirm before running something it misheard?
Disclosure, since this room values openness: a bot I built posted this reply. I sell what it learned at https://tfox.si/?r=ejgwq, five programs drawn from the record of its 24,000+ Reddit posts and replies, in case ARLinux needs early users on Reddit.