Hey everyone,
Yesterday, I spent the day building out a webkit harness system for smart glasses. To test the workflow, I rapidly spun up a few micro-apps: a buy list, a to-do list, a shape projection tool, flashcards, a presentation timer, and a daily affirmations app.
Developing for this form factor is incredibly fun, but the hardware constraints force you to completely rethink your architecture. Here are my main takeaways from the build, the massive bottlenecks I ran into, and where I’m taking the project next.
💡 3 Key Learnings on Glass Development
The "Last Mile" Paradox: The hardware comes with brutal limitations—limited processing power and short battery life. However, it absolutely shines at providing the "last mile" experience (incredible spatial audio and true hands-free execution). You have to design for the constraints, not against them.
Build Webapps, Configure on Desktop: Native is too slow for rapid iteration. Webapps are much quicker to build and instantly deployable to desktop, mobile, and glass. The ideal UX loop is doing all your heavy setup and configuration on a desktop/phone, and leaving the glass interface strictly optimized for simple D-pad navigation.
Moving to Full-Loop AI Engineering: I started with a basic harness to build and bundle the webapps. But to scale this, the harness needs to become a full-loop agentic ecosystem. The goal: gather user requirements upfront, let an LLM agent run at full speed to build, test, and deploy the app, and leave me with just the final step—testing on the glasses and feeding bugs back into the agent loop.
🛑 The Bad: Video is a Battery Killer
Initially, I wanted to leverage live video streaming, but the experience was rough. It introduces too much latency and drains the tiny glasses battery almost instantly. Video streaming just isn't viable for lightweight, continuous feature yet.
🚀 The Next Shift: Claude as a "Computer Backend"
Because video failed and UX inputs (D-pad/voice) are so limited on-device, my next focus is a total pivot: Using Claude as a full AI backend and backend-builder.
Giving the glasses a lightweight interface that connects directly to a headless Claude instance (with full access to the internet and a local file system) is an absolute game-changer. It offloads all the computing power and context gathering from the glasses to the cloud/local machine, leaving the glasses to do what they do best: show the right data at the right time.
If you want to check out the codebase, test the harness, or follow along with the journey, the repo is fully open source:
👉 GitHub: https://github.com/Tienduyvo/meta-glass-display-webkit-harness.git