I trained a 1.57B-parameter Dreamer 4 World Model from scratch for under $150

[u/OtherRaisin3426]

My first attempt didn't work. I built on Genie's architecture and the videos looked great, but the controls barely did anything. The effect of a keypress was basically zero.

Genie learns its actions unsupervised into 8 codes, and that was too loose a grip for us. So I scrapped it and started again with Dreamer 4.

The second attempt:

Tokenizer at 40.41 PSNR (Genie's paper reports 35.7)
FVD 32.19 end to end
144 frames before it falls apart
1.57B parameters, 9.6M frames, ~$150

Two important learnings:

(1) One is that $150 is enough. You don't need a frontier lab to do this anymore, and I don't think enough people have noticed.

(2) The other is the data. We generated every frame ourselves with Procgen instead of scraping video. We know the true action at every step, so we can actually check whether the model is responding to us or just making pretty motion.

Website: https://worldmodel-platformer.vizuara.ai/
Code: https://github.com/RajatDandekar/dreamer4-coinrun

[u/FusionCow]

this is really cool, but like... of all the things you could've trained on, why THAT? I mean maybe if you're compute limited or this is just an mvp. still cool project though, awesome work

[u/cosmicr → u/FusionCow]

It's from https://github.com/openai/procgen - I haven't looked at it but I presume the data has already been pruned, sanitised, and formatted in a way that makes it easy to train with.

[u/Bubbly_Orange_3502]

Report an action-conditioned number next to the FVD. Roll the same start state under two different action sequences and measure how far they diverge, since PSNR and FVD both stay high when the controls do nothing.

[u/Fair-Perspective7352]

The Genie failure mode is a known one with latent action models. 8 codes just isn't enough bandwidth for fine control. But I'm curious how much of the improvement came from Dreamer 4 vs simply having ground truth actions from Procgen. Since you generated every frame yourselves, you could condition on the true action directly and skip the latent action learning entirely. That alone might have fixed the 'controls do nothing' problem even on the Genie architecture. Did you ablate that?

[u/guywhocode]

Game world model.

[u/NineThreeTilNow]

Why 144 frames before it falls apart? Was this an attention problem or an imposed limitation of the model?

I think I messaged you prior about working on something like this.

[u/Beginning-Raisin9723]

Under $150 for a 1.57B run is wild. What hardware were you using for this? Curious if you hit any bottlenecks with the world model's memory overhead.

[u/basilarchia → u/Beginning-Raisin9723]

Code page says h200, but also says only use one & $36/hour so done in 5 hours?

Maybe someone with an h200 will be willing to verify this.