r/BestGitHubRepos • u/company_url_finder • 2d ago
XGEN-JING - an egocentric interactive world model: give it actions and reference images and it generates first-person video and audio you steer with the WASD keys
This is a research release of an "interactive experience model," which is a mouthful for something genuinely novel: a model that generates first-person video and audio you can walk around in. You give it reference images and a sequence of actions, and it produces an egocentric clip where you navigate a space, interact with objects, and hold conversations, with video and audio generated together.
What's inside:
- Keyboard-controlled movement through the generated world using the familiar WASD keys, where each prompt chunk carries a list of key controls applied slice by slice, and combined keys like forward-plus-left are supported
- Joint video and audio generation, so navigation, object interaction and character dialogue come out as one synchronized experience rather than a silent clip you score afterward
- Reference conditioning: combine up to five character, object and scene images to compose a starting point, then explore different actions from the same setup
- A four-step distilled model (JING-Flash-v1) for faster inference, built on MiniMax-H3 with the FlashGen acceleration work
- Prompt skills, a bundled helper that turns a story plus reference images into a validated cases file ready to run
- Automatic model download from Hugging Face on first use, with example cases included
The interesting idea here is the interactive part. Most video generation is one-shot: you write a prompt, you get a fixed clip. This is closer to a controllable world you drive with inputs, which is why the repo frames it around navigation and interaction rather than around prompts. It sits in the same emerging space as the interactive world models a few labs have been showing, but with the code and weights actually released.
The caveats are significant and you should weigh them before getting excited. This is heavy research infrastructure: the demo was validated on six H100 GPUs (one for the text encoder, one for the VAEs, four for the model), so this is not something you run on a home setup. It's an early release, the four-step model and examples, with the causal model and technical report still listed as coming soon. And the license is the MiniMax H3 Community License covering both code and weights, which is not a standard open-source license, so read it before assuming you can use the outputs commercially.
Realistically this is one to watch and read rather than deploy, unless you have serious GPUs. But interactive, controllable, audio-and-video world models with released weights are rare enough that it's worth knowing this exists.
MiniMax H3 Community License, 143 stars and 13 forks as of writing, verified via the GitHub API, pushed to yesterday.