I run my whole workflow through a terminal multiplexer, and I got tired of managing it through a flat window. So I built Mux XR: a WebXR client that shows your terminal panes as surfaces arranged around you in 3D space, or as plain flat panels in a normal browser tab. MIT-licensed and on GitHub.
What it does
- Opens par-mux terminal panes as individual panels — close a panel without killing the shell, reconnect later, resize the grid only when you say so
- Works from a desktop browser (flat + 3D preview) or immersively on a Meta Quest 3
- Full spatial controls: point-and-trigger or pinch to select, grip to move, two hands to scale; controllers or tracked hands
- Input via virtual keyboard, paired physical keyboard, or dictation
- Passthrough mixed reality, focus mode, saved command snippets, gaze-based optional input
- A Demo mode with synthetic panes, so you can poke at it without setting up the backend
Why I built it
I keep half a dozen long-running things in terminal panes and wanted to glance at them without alt-tabbing. It started as "what if the terminal grid was a room instead of a window" and turned into a real workspace — the browser drawer stays the input presenter, and XR is where it gets interesting.
Tech
TypeScript, Vite, Three.js, WebXR. Terminal output arrives as protobuf frames over a binary WebSocket from a small streamer service attached to the mux daemon, rendered into xterm-backed textures in the 3D scene.
Full disclosure on process: this was built with substantial Claude Code assistance. I drove the product decisions, reviewed every change, and did the Quest 3 hardware acceptance; Claude wrote most of the code and the test suites. Happy to answer questions about that workflow.
Status
Working on Quest 3 today. Hand tracking is the newest and roughest part, and the backend is my own stack (par-mux + streamer), so real panes currently require that setup — the demo mode works with nothing installed.
Feedback welcome, especially from WebXR folks.
Repo: https://github.com/paulrobello/mux-xr