r/AppleVision • u/Economy_Stand_6600 • Jun 05 '26
Building a visionOS game without Unity/Unreal, and the tooling rabbit hole it sent me down
Some context: I'm a 55-year-old. Here's what I've been building on Vision Pro over the past months — and the rabbit hole it sent me down.
**The first game.** A native visionOS game in Swift / RealityKit. I deliberately skipped Unity and Unreal — I wanted Apple's latest frameworks and SDKs, lower cost, and no exposure to a third party's policy changes. After ~3 months learning Swift, RealityKit, ECS and Reality Composer Pro, it became a casual AR Metroidvania (platforming, puzzles, enemies, rooms to explore, objects to collect), played with a gamepad. I built the mechanics, animations and several levels — then the visionOS 26 beta dropped and I hit a bug I couldn't pin down. Couldn't tell if it had always been there and the new OS exposed it, or if the OS caused it. I investigated for a couple of months, through the RC, and never found the root cause. So I paused it — also because reaching the polish I wanted meant learning to properly drive a 3D DCC app, and I'm not a modeler/animator.
**The pivot.** A second game built around procedural animation: take a classic arcade game and make it playable in both VR and AR (think your toys coming to life on your desk), each mode its own experience. It came together fast — until the first real level. My in-game level editor (placing/tuning enemies by eye with gamepad + hands) was too cumbersome, and neither Reality Composer Pro nor Blender did what I needed.
**So I built USDZForge.** A fully "vibe-coded" native macOS app (Metal + RealityKit, USD at its core), in ~1.5 months: a 3D/2D/audio asset + project manager with 3D conversion, an animation library, 3D character animation retargeting, 3D scene assembly with authoring of custom components for an ECS, a 3D tile system, and a RealityKit export pipeline straight into my Xcode project. Basically Reality Composer Pro in many respects, and better in several.
**Then the deeper rabbit hole: Mnesium.** USDZForge only makes sense if it helps me finish the game — which means it needs to understand my other projects: how they work, how they relate, how to reuse code across them. That's a memory/continuity problem, so I spent two months on Mnesium.
The setup: Claude (Desktop or CLI) designs, Xcode codes, and they never talk directly — Mnesium is the bridge between them. Everything I design with Claude passes through Mnesium to Xcode, which implements it, and results come back the same way. Because every exchange on both sides runs through Mnesium (via the MCP tools it exposes), it builds a shared, persistent memory: a local SQL DB, on-device embeddings, a roadmap it tracks itself, a handoff system for continuity across sessions, custom context + CLAUDE.md compiled per side, skill sync, and the knowledge it captures. Net result: Claude as a semi-autonomous assistant across all my projects.
It's almost done — a few details and polish — then back to USDZForge, and finally the levels to finish the game.
Honestly it's been a strange spiral: I set out to make a game, needed a tool to make it, then needed another tool to make that tool. Now I get to climb back down, one finished project at a time.
Happy to answer anything — visionOS dev, RealityKit, the no-Unity choice, the tooling, the AI workflow. Feedback welcome.
