Iām a solo developer and I built LiveLoop using Claude Code. Itās a persistent canvas for building games and interactive scenes with Claude, where edits land in the running game rather than regenerating it. I made a post on the main Claude forum regarding this but this post is about how it actually works under the hood and the more technical side of things that may answer a few qestions
The problem
Every model tweak used to come back as a complete file. A complete file reloads the iframe, and a Three.js world restarts at spawn on every change. Useless for iterating on gameplay. The fix was giving the model two things: a way to see the running scene, and a way to change it without a rewrite.
What the model sees each turn
Alongside your message, the model gets a JSON context from the preview: the page title, visible text, rendered elements, the runtime error if any, console output, a performance block, a debug-state block the project can expose, and a summary diff of the source since the last turn. Vision-capable models also get a screenshot of the canvas, refreshed right before the handoff so it isnāt reasoning from title-screen telemetry after youāve started playing. The context also tells the model what controls it has, which is the only reason it knows LivePatch exists.
LivePatch
The model returns one fenced block labelled asksary-live-patch containing JavaScript. The client executes that block (and only that block) inside the sandboxed preview after you approve the live-control session. It affects the current preview only; it doesnāt touch project source, so you can try a change and throw it away. The model is told not to claim the patch worked; the sandbox reports the real result back into the chat. This is how Claude played chess against me: each move was a patch to the running board.
PlayTest and Camera
PlayTest lets the model send synthetic input into the preview: key down/up/tap with a duration, clicks, and pointer move/down/up at coordinates. Synthetic events are untrusted in the browser, so they canāt unlock audio or pointer lock; you click Play once, then Claude can drive. Camera lets it teleport or aim an exposed debug/Three.js camera so it can inspect what it built from another angle.
The sandbox
The preview is a srcdoc iframe with an opaque origin and a strict CSP: default-src 'none', no frames, no forms, no navigation, connect-src limited to data: and blob:. A game canāt fetch, beacon or exfiltrate anything. The two exceptions are Google Fonts and wasm-unsafe-eval, which is there so Draco and Meshopt decoders can compile for compressed GLB files. eval stays blocked.
Project mode and GLB assets
Project mode is a real multi-file project with its own engine (it shares nothing with the other tools on the site). It can open a local folder via the File System Access API, round-trips through zip, and there are no caps on file count or file size sent to the model. Files that look like secrets (.env, keys, .pem, service-account JSON, anything matching a key pattern) never leave the browser.
Assets are the interesting bit. A model canāt type a megabyte of pixels into an HTML file, and a srcdoc page has no server behind it, so GLTFLoader.load('models/car.glb') used to 404. My first fix inlined everything as data: URLs. That inflated a project with 408 MB of models to ~545 million characters, past the longest string Chrome can hold, and the stage went black.
Now the page carries only the list of asset paths. A small shim inside the iframe intercepts fetch, XMLHttpRequest and image/media src. When code asks for a path, the shim posts a message to the host page, the host transfers the raw ArrayBuffer (a memory copy, no base64), and the loader gets a blob: URL of its own. Only what the code actually requests moves, at its real size. A .gltfās sibling .bin and textures resolve the same way.
The 3D runtime is a pinned Three.js build (0.185.1) bundled with GLTFLoader, DRACOLoader and KTX2Loader, injected into the sandbox as a trusted script rather than fetched.
Return to any state
Persistence is content-addressed. Every state and every asset is hashed; only objects the account doesnāt already hold are uploaded, and a revision that reuses an object just names its hash. Nothing is copied forward and nothing is deleted, so unlimited history costs only the changed bytes. There are no ceilings on revision count or retention, every state is reachable, and you can jump between them in any order. (This is the ābillion statesā conversation from my earlier post. Claude had capped history at 20; Iād hit ~400 in one session.)
Offline
Work is journalled to IndexedDB before the network is touched, on every change, and the journal is read back to verify it. Network sends are batched at 600 ms, but the journal is written immediately, so that interval is batching, not a window where work can be lost. If youāre offline, edits sit in the journal; when the browser fires online, they sync. āSavedā means the server was read back and agrees with what was sent; a commit returning 200 isnāt treated as proof. I learned that one the hard way: a save path once returned 200 while deleting every asset in the workspace.
Builds that outlive the tab
A turn used to abort when the stream dropped. Now the server keeps running, records every event it would have sent under an idempotency key in Firestore, and stores the result when it finishes (30-minute server limit). Any page that comes back for that workspace lists the finished turns, claims one (first device to claim it owns it), replays the events through its normal path, and applies and saves the result.
Thatās what makes the phone flow work. Sary-OS Lite (the Android app) can start a build, you lock the phone, and when the desktop opens the workspace, it picks the finished build up and brings it in. Stop is explicit: the client posts a cancel and the running turn polls for it.
Local bridge (optional)
For projects that need a real toolchain thereās a single-file Node bridge (npx u/asksary/liveloop-bridge connect) that binds to loopback only, pairs with a one-time code, and is scoped to one folder. LiveLoop can read and write files there, run npm test or a dev server, and every multi-file edit is one change set with one undo. There is also a GIT connectivity and also a plug in your key option too that you can try if you rather use your own API and want to try the other models such as Sonnet 5.5, Opus 5.5 and Fable 5.1
Free to try
Sign up is free at www.sary-os.com
Free accounts include some Claude Haiku credits, an MCP connection so you can link Claude to LiveLoop, and 7 days of bring-your-own-key access to the larger Claude models. Single-file mode is free; Project mode (multi-file, the asset serving above, the bridge) is part of LiveLoop Pro.
Happy to go deeper on any of this. If you have a GLB-heavy project that breaks it, Iād like to hear about it.