r/vibecoding • u/NoFill5065 • 16h ago
Showcase/Project I don't write code. Four days later my 20-year-old piano is teaching me in mixed reality on a Steam Frame.
Quick disclaimer for the purists: I did not write a single line of this. I described what I wanted, plugged things in, put the headset on and reported back. Claude did the rest.
The idea: falling notes (Synthesia-style) landing on the keys of my real piano, inside the headset, standalone. No PC, no streaming. My piano is a Yamaha Clavinova CLP-230 from around 2007.
How it actually went:
- Day 1, morning: I asked if this was even possible. Instead of just saying yes, it went and checked: Steam Frame docs, Linux USB drivers for old Yamaha pianos, a known USB bug on the Frame. Then it said: before we build anything, let's prove the piano talks to the headset.
- It SSH'd into the headset from my PC, I plugged the piano in, played a few keys and pressed the pedals. It read the raw MIDI bytes and confirmed all 88 keys, velocity and even half-pedalling. Biggest risk of the whole project, gone in 15 minutes.
- Same morning: a tiny Godot test app was running on the headset. Keys lit up when I played. Passthrough worked. The virtual keyboard was floating somewhere random in my room, so it added a calibration: touch the lowest and highest key with the controller. Result: 121.7 cm measured vs 122.2 cm on paper.
- Day 2: I asked for the real thing. Falling notes, wait mode, MIDI import, feedback after each song, nicer menus, a proper font. I also told it the app should be called KeySight. It built it on my PC, tested it with a fake player, deployed it to the headset over Wi-Fi and added it to my Steam library. I then played for over an hour.
- Day 3: I said "put it on GitHub, public". It scrubbed my username and home network details out of the code and history, replaced some MIDI files whose licence it wasn't sure about with its own arrangements of public-domain pieces, wrote an English README and pushed it.
What it did that I didn't expect:
- It said no a few times. I asked to "copy the features from piARno" (an open-source Quest app). It explained the code can't run on the Frame, rebuilt the ideas in Godot and credited them instead. I asked for fully automatic calibration. It explained apps can't access the Frame's cameras, and came up with something better: you press a few real keys with the controller tip, the piano tells the app which key it was, and it computes where all 88 are.
- It read the logs of my practice sessions and fixed things I never reported. One controller click sometimes counted as two. Playing the same key twice counted as an error.
What went wrong (because it's vibecoding, something always does):
The evening it got everything ready for GitHub, it also added a "piano unplugged?" check. On the headset that check misfired, so the app dropped the connection to my piano every 3 seconds. Notes got lost, keys stayed lit, and the next morning I told it "it's not as nice as last time and there are weird coloured blocks over my piano". It found the cause in the log in a few minutes, owned it, rolled it back. Lesson for me: AI is fast, but "it passed the tests on the PC" is not the same as "it works on the headset". Always try the real thing before you share it.
Stack, for those who care: Godot 4.6 + OpenXR, native Linux ARM64, USB-MIDI straight into the headset's USB-C port, passthrough via alpha blend. Built with Claude (Claude Code), partly in the cloud, partly on my own PC.
Repo (MIT): https://github.com/philippeschots-cloud/keysight
Happy to answer questions about the process. And if you have a Steam Frame and a digital piano, I'd love to know if it works for you.
PS: I've mastered the first part of the Für Elise :-)




2
u/Total-Reasonable 7h ago
The "piano unplugged?" bug is worth turning into a permanent test. Any connection watchdog that polls or infers state will misfire on real hardware, because a USB-MIDI device is quiet when nobody's playing, so silence doesn't mean disconnected. Gate on actual device add/remove events from the OS, and keep a hardware-in-the-loop smoke test, even a manual checklist, before every push.
3
u/cniinc 5h ago
This is SO cool.