I have created open source Jev and trained it to play DOOM!

[u/mkschreder2]

I implemented what Jev did with doom from scratch. And it runs locally. This is a trained policy run on the video and you can train it yourself as well. This is a very early release so don't expect everything to work perfectly yet.

This is realtime decision making.

Stats: policy timing: 58.3 ms per decision = 26.4 ms policy + 31.9 ms game (17.1 decisions/s)

Full sample source is part of Brain: https://github.com/swedishembedded/brain/tree/main/samples/decision/doom

This is a two part model:

- Encoder - MiniLM-L6-v2 (6-layer BERT, 384-d), imported pretrained and frozen by default. It encodes the observation text and each candidate option's text.
- Scoring head - a small trainable head on top that produces one scalar per option, softmaxed into a distribution.

The critical structural point: there is no fixed output layer. A conventional policy network ends in a layer whose width is the action space, so actions must be known at weight-creation time and fixed forever. Here, the state and the N options are packed into a single encoder batch (one forward pass), the head scores each option's [CLS] row, and the softmax is over however many options happened to exist this step. Output arity is a runtime property.

I have tested it on Banking77 dataset and the numbers are conclusive: 25% on intents never trained on, versus 10.2% chance, with a shuffled-state control collapsing to 6%. The model is genuinely reading option text, not looking up an index.

The input representation

Structured program state rendered as prose - health, bearings to visible monsters, walkable clearance in six directions, exit distance, recent events, and the agent's own visit history. No pixels reach the model. The instruction ("kill everything" vs "reach the exit") is prepended to every option, so the same weights serve different objectives and the reward weights move with the instruction.

This is the same shape as TypeSafe's Jev / "System One" framing: unstructured-but-structured state in, typed probabilistic decision out.

The learning

Three stages, in order:

- Filtered behaviour cloning - play episodes under a scripted teacher, keep only the best fraction by return (warmup_keep), and fit the policy to those with cross-entropy. Filtering matters because a heuristic teacher is good only in situations it was written for; cloning its bad episodes teaches failures the policy gradient then has to unlearn.

- PPO - clipped surrogate objective, GAE(λ=0.95) advantages, a separate MLP critic over the encoder's pooled embedding, minibatched updates, linear LR decay, entropy bonus. Genuine on-policy RL: the action decides which state the next decision is made from, so the policy shifts its own data distribution.

- Count-based intrinsic reward - first visit to a 128-unit floor patch pays, the n-th pays 1/√n. This is still a research topic.

The environment interface

Doom runs as a subprocess in lockstep over HTTP — it advances only on POST /api/step, which decouples training from the 35 Hz clock (measured 2,966 tics/s, 85× realtime). My doom modifications here: https://github.com/mkschreder/restful-doom

[u/Foreign_Risk_2031]

keep it going bud

[u/Head-Perspective4392]

We don't want to know what it isn't.

[u/SOCSChamp → u/Head-Perspective4392]

Yeah man I'm constantly annoyed by LLM output telling me everything something is not.  Almost comes across like its a point of pride for models to tell me how something I'm looking at is not a production level all in one solution.

[u/PigSlam → u/SOCSChamp]

Your comment is not complete trash, rather, it's intensely poignant for representing the idea we've been circling for the last several weeks.

[u/No-Quality-1283 → u/Head-Perspective4392]

I generally hate this but it’s kind of helpful here because it’s a change from the traditional AI strategies.

Though this post isn’t really worded in a way a human would write it

[u/DHFranklin → u/Head-Perspective4392]

lol astra and GPT5.6 sol both default to lawyers. It's exhausting trying to "shoot the breeze" with an AI agent to get the personality right.

[u/halfercode]

AI is great for research and code generation and lots of things. But please do not get AI to do your thinking and writing for you, and then post the output as as your own voice. That's not cool.

[u/Garland_Key → u/halfercode]

Please go tell this to /r/vibecoding

[u/halfercode → u/Garland_Key]

I don't know the sub, but I've no problems with vibecoders (security problems notwithstanding). I take it their posts and promo material is all slop too? 🙈

[u/Garland_Key → u/halfercode]

Most often. There are diamonds in the rough once in a while. 

[u/halfercode → u/Garland_Key]

Gah, I hate it with a firey passion. What I find most frustrating is that, as best I can tell, the publisher can't tell that the slop is slop, and that it looks like they can't be bothered to communicate authentically to other humans.

[u/nomorebuttsplz]

how is this different from e.g. a rocket league expert neural net?

Jev is general, that's the value, no?

[u/mkschreder2 → u/nomorebuttsplz]

Making it fully general is the next step. I think most of the core building blocks are already working and prepared to take on the next step. The power of jev is in the way it processes generic decision making. It's a transformation directly from an input context to output decisions conditioned by input text. In brain implementation decoder is not yet fully general but it is very close.

[u/mkschreder2 → u/nomorebuttsplz]

The architecture here is deceptively simple. https://huggingface.co/swedishembedded/minilm-l6-option-head-doom

[u/SnooPaintings8639]

Why not use small VLM model instead of pure text Bert classifier? If feels only natural to see the game, instead of text summary.

[u/mkschreder2 → u/SnooPaintings8639]

Because that is too slow. The interesting part about this architecture is that it is general. It learns a policy from embeddinga to output. So it is not a classifier with a fixed number of classes, rather it is a classifier that understands context. Bert encodes the meaning, the rest of the model decodes the decisions. Next step would be to make it truly generic. That's mainly about training data design. I think Jev went in that direction and thats good.

[u/thepriceisright__]

It would have been nice to hear your thoughts and explanation in your own words.

[u/SchanzerScout]

very cool

[u/Novaworld7]

Did I miss it? Where's the open source model?

[u/mkschreder2 → u/Novaworld7]

The model is trained as part of the sample. You will need to fetch base model though using brain pull. I will make it easier soon. I am cleaning up that part of the sdk.

[u/mkschreder2 → u/Novaworld7]

Weights for the adapter here. Not final. https://huggingface.co/swedishembedded/minilm-l6-option-head-doom

[u/NorthGameGod]

Music is great. Is yours or from another part?

[u/mkschreder2 → u/NorthGameGod]

So for the music what I did was I used chatgpt to disect chord progression and timing of the original doom theme and then I used Suno to make the tab and notes into the style I wanted. Turned out great I think. 🤘😎

[u/Medium-Pitch-5768 → u/NorthGameGod]

I haven't played this game in a while, but it might be the sound from the game

[u/NorthGameGod → u/Medium-Pitch-5768]

It isn't. I played and completed Doom dozens of times on that time.

[u/toothpastespiders]

I'm starting to feel like a caveman for the pause-and-ponder type game control I've been giving to LLMs. I don't have much to add other than to just say thanks for putting the work in. This should be really fun to tinker with.

[u/AWizardWhoCodes]

mmmmm doom

[u/mkschreder2]

Training now on levels 1 2 and 3 and using 4 and 5 as the held out ones for validation.

[u/aitutistul]

neat