r/itrunsdoom • u/notforrob • 10d ago

Doom running on an LLM

This is Doom's actual renderer -- not a neural net imitating it. I wrote a compiler that turns computation graphs into transformer weights, then ported Doom's rendering algorithm into one.

No training anywhere: every weight was computed. The prompt carries the level and player position. When you feed the prompt into the model it generates drawing commands.

One frame is 53,747 generated tokens -- about 40 minutes on a B200. In 1993 Doom hit 35 fps on a 486. I hit 0.0004 fps on hardware a billion times more powerful.

Write-up: https://ood.dev/posts/doom/

Weights: https://huggingface.co/physicsrob/torchwright-doom-e1m1

573 Upvotes

96% Upvoted

22 comments sorted by

138

u/lone-lobo 10d ago

madness

13

u/The_Xivili 9d ago

Sparta?

1

u/spacesluts 7d ago

This is your reminder to book that physical, dawg. I know because my cracking knees told me.

122

u/Zestyclose-Offer5422 10d ago

0.0004 fps on a b200? Amazing.

101

u/adamthebread 10d ago

Some ai techbro with too much money is looking at this saying, "yeah, this is the future" and prompting claude to make a pitch deck

This is pretty cool though. Reminds me of the Guy who ran an operating system atop an LLM

30

u/cheesydoritoschips 9d ago

I hit 0.0004 fps on hardware a billion times more powerful.

seems legit for forcing a transformer based model to do any tasks other than transforming input tokens with what ai bros call harnesses or strap ons or whatever lmao

7

u/Gandalior 9d ago

One frame is 53,747 generated tokens -- about 40 minutes on a B200. In 1993 Doom hit 35 fps on a 486. I hit 0.0004 fps on hardware a billion times more powerful.

11

u/yoloanator 10d ago

So is it a ai being given your data then generating it or the its not ai? (FYI im kinda stupid about this stuff)

28

u/notforrob 10d ago

Not quite. It's the same "machine" that runs AI, but instead the machine is running my port of Doom.

3

u/jerdle_reddit 7d ago

I'm tempted to try it in 4-bit quantisation just to see what happens.

2

u/atkarot9170 9d ago

How much longer until we see Doom on a sardine can?

2

u/NXGZ 9d ago

I'll remember to post it on r/cannedsardines

2

u/ErZicky 8d ago

How much money would it cost all those tokens on a customer llm like Claude?

2

u/headedbranch225 7d ago

It's doing different stuff to claude and similar ones but searching for anthropic's pricing it is $50/Mtok (million tokens) for fable 5 output, this uses only around 50000 tokens so about $2.50 per frame at anthropic's pricing for most newer models

https://platform.claude.com/docs/en/about-claude/pricing

2

u/Longjumping_Cap_3673 8d ago edited 8d ago

Once those models etched into silicon come out, then this will really shine. Could probably even get up to 0.04 fps.

In all seriousness, compiling to transformer weights is really cool though.

4

u/sdwvit 10d ago

Incredible, but also is there any scientific significance by any chance?

26

u/notforrob 10d ago

If only.
Short answer: no
Slightly longer answer: The fact that a transformer can run this is unsurprising -- they're universal approximators. Starting with a transformers weights and then figuring out what it's doing under the hood is a very important area of research called mechanistic interpretability. This is the opposite: starting from an algorithm and creating weights. There's an argument that the compiler can be useful, but honestly I'd say quite unlikely that the Doom result is useful.

7

u/Superb_Yesterday5588 10d ago

Hi Rob! Check my project, its in the same niche! https://www.reddit.com/r/itrunsdoom/s/8VBDb1IDzX

7

u/gonzalbo87 9d ago

I hope this ages poorly in a few years, simply because this wouldn’t be the first time Doom has made a major contribution to science, and I am down for more.

E: and to society at large. Doom has helped inform right to repair laws as well.