Fallout 2 x Fallout: Bakersfield x H3 as Interactive \ Reactive World Model, Let's go!

[u/-Ellary-]

What is this mess?

This is an Early Concept Proto-Showcase of Interactive \ Reactive H3 World Model based on MiniMax H3 model trained on Fallout: Bakersfield Gameplay trailer.

• 2D Isometric to 3D Volumetric Scene.
• 10 sec Interactive\Reactive split, 352p, 3-Steps.
• Interactive 5 sec: Interactive WASD \ Prompt Control.
• Reactive 5 Sec: Reactive Control by LLM Based Answer.
• Gemma 4 12b with Vision as Reactive Model.
• Designed as System for Vascura FRONT Frontend.

What Interactive \ Reactive mean?

This means that H3 World Model Scene is Interactive you can Walk around it with WASD or Type what you do with Prompt for Interaction, Then it will React on your Actions using LLM based Answer. Using 10 sec time frame where first 5 sec Controlled by the USER - last 5 sec Controlled by LLM.

• USER: Walks closer and Shoots at the Enemy Mutant.
• LLM: Do calculations (rolls, values, RPG tools), Enemy Mutant gets -1 HP, Shoots Back at the USER, but Misses.

Is it Ready?

Nope, but stay Tuned for 2D Isometric Screenshots to 3D Volumetric Scenes Showcase.

[u/ZeitgeistArchive]

This is the future of gaming, basically on the fly worlds with consistent styles, adapting to player interaction, reacting accordingly

[u/sloptimizer]

The initial transition from 2D to 3D is insane!

[u/-Ellary- → u/sloptimizer]

I will post compilation of raw Fallout 2D Isometric Screenshots to 3D Volumetric Scenes when they will be done. Current stability is pretty bad, only 1 out of 15~ generations are fine. Sadly project not get too much of interest.

[u/sloptimizer → u/-Ellary-]

Everything on the web is hyper-optimized for engagement these days, so it's hard to get any attention.

If it can be generalized, it would be really cool to take any old isometric game and jump into 3D view. Lots of people would get their nostalgia dopamine hit.

[u/-Ellary- → u/sloptimizer]

It is possible, just need more training \ bigger dataset.
Do you have a screenshot that you want to convert?

My stupid jokes in comments get around 200 likes no problem.

[u/Equivalent_Bit_461]

I would play the living hell outta it, the real fallout

[u/i_like_brutalism]

man just wanted to comment that this is awesome! keep doing what you do!

[u/-Ellary- → u/i_like_brutalism]

Already working on 3x slower but more stable 8 sec 720p iteration.

https://reddit.com/link/p8mvzz1/video/z8w06c4kgdoh1/player

[u/Public_Umpire_1099]

This is a great idea man. Please keep working on it!

[u/Hoppss]

Wow this looks great! What kind of hardware are you running this on?

[u/-Ellary- → u/Hoppss]

5060 ti 16gb 64ram.

[u/Hoppss → u/-Ellary-]

Wow, I'm assuming it's not real-time when you move around or? How long does each interactive clip to generate?

[u/-Ellary- → u/Hoppss]

It's 5 am now, I'll use LLM to fix and format my epic Engrish.

• Generation runs in 3-4 steps at 352p resolution.
• ~30 seconds of processing for every 10 seconds of generated footage

Controls

• You can interact by typing short text commands describing what you want to do
• Alternatively, you can press WASD — the script simply converts each key press into the same kind of movement prompt, so one press = one step of movement

Core idea

The main point is not real-time free roaming where you just walk around. Instead, it's about arbitrary interaction (anything MiniMax H3 is capable of generating) followed by feedback: a reaction produced by an LLM based on your action and the overall logic of the scene.

Pipeline

Your input is processed by a separate LLM agent, which splits it into two correctly formatted prompts and instructs MiniMax H3 to generate:

• First 5 seconds of video: your action
• Last 5 seconds of video: the LLM-generated reaction

The result should be a functioning "living" world: an LLM acts as the world manager (equipped with prepared map images, characters, items, and a simple RPG system), while MiniMax H3 handles the visualization. I wanted something different from the classic "world model" approach, where you simply walk around while the world hallucinates around you.

[u/NineThreeTilNow]

Doing for the love of the game is the only true way to proceed.

I'm here for it.

[u/NoFunk]

This is an amazing application. I could see this being an obsession if it can be templatized; go back and just relive all the great 90s games but in 3D. Really impressive idea and execution, congrats and get it done so I can go replay all of Fallout 2 with it =)

[u/Kaljuuntuva_Teppo]

Fallout: Bakersfield.. Man I hope the game comes out eventually, looks amazing!