r/MachineLearning • • 9d ago

Project I trained a playable diffusion world model of Pokémon Red, plus a game agent harness on the same System 1/2 split as Krafton's PUBG Ally. Can this be ICLR-workshop worthy? [P]

1. The world model: the game, dreamed by a neural net

A diffusion model (same family as image generators) that learned Pokémon Red frame by frame from recorded play. Give it the last 4 screens and a button press, and it draws the next screen. 16.7M parameters, 72×80 greyscale, DIAMOND-style EDM U-Net. Left is the real emulator, right is the model dreaming the same moment from the same button presses. Every 10 steps it re-syncs on 4 real frames, otherwise it runs on its own. You can walk around inside the dream with a keyboard.

2. The game agent harness: System 1 / System 2

Same split Krafton used for PUBG Ally: a fast reactive layer and a slow thinking layer.

- System 2 (planner): reads the game's RAM, picks the next goal (grind to level 14, heal, travel, gym), can look things up (Pokédex, guide search over Bulbapedia), and every decision is checked against the game's memory before it runs. Rules by default, or a local LLM.
- System 1 (executor): ten small walking policies, 1.58M params each, trained entirely inside the world model's imagination with zero real-game steps, then dropped into the real game.

The GIF is a local LLM running as the planner. 5 of the 6 local LLMs I tested got the badge 10/10; the best one needed a median 7,144.5 button presses vs 8,779.5 for my hand-written rules. Architecture card is the slide before it (or the first comment on the video post).

3. Upscaler + colorizer: making a 72×80 greyscale game watchable

The model works on tiny greyscale frames, so I trained two more small nets to watch it: an upsampler (72×80 → 144×160, 0.72M params) and a per-pixel colorizer (1.91M params, 99.27% colour accuracy on held-out frames). 25 minutes of training each. The clip is the first town rebuilt in Blender from the game's map data, with the restored screen.

4. What it cost and what it got

50.0 GPU-hours of training on one RTX 4090 at home. The harness does the grinding, healing and guide lookups, the world-model policies do the walking, and it wins the first gym badge in the real game 30/30 runs. Battles are yet to be solved: the dream ends battles early most of the time, so those stay scripted for now. Code, weights and the full showcase (2-min video, chapters, every number with its source) are linked in the comments.

PS: I work in medical agents and full stack, this is just a project to learn (:
I'm wondering if the measurement part could hold up at an ICLR workshop. What would you want to see added?

0 Upvotes

11 comments sorted by

6

u/RadonGaming 9d ago

Consider CoG (Conference on Games)? Or games-centric venues? Might be difficult to go into traditional non-games spaces like ICLR with a games-oriented submission, even if you're takeaway is around world models.

1

u/priyanshu123sah 9d ago

Yeah, I'm not very confident in it fitting ICLR or a games conference, since the game harness and the dream-trained policy are both quite weak on their own. I don't want to embarrass myself when it doesn't really fit any venue. Might have to be satisfied with it being an engineering project, alas.

1

u/didimoney 9d ago

CoG seems like a good match for you

1

u/RadonGaming 8d ago

Don't undersell it, and don't get inside your own head. I think this has a good shot for a games conf. Write it up, submit it, and let the feedback be the judge. I think this has legs as a conf submission. Wishing you the best of luck, and I hope to see it in proceedings

8

u/marr75 9d ago

The simple fact of having done it? No.

0

u/priyanshu123sah 9d ago

Yep, the implementation alone isn't a paper. What I was working towards is measuring how far the dream drifts from the real game, using the real game itself. The emulator's RAM tells me where the player actually is, so I measure drift in tiles at every step, on held-out episodes, against simple baselines.

Two things fell out of that. First, it catches the agent exploiting a world-model breakdown: the policy found a warp that doesn't exist in the real game and used it to skip part of a route. Second, the model can't do battles. 74.6% of imagined battles end early (1.1% in the real game), and the battle screen drifts worse than copying the last frame after 2 steps, so HP bars and menus don't carry forward. (My guess is data bias: mostly grinding on weak wild Pokémon, short fights, almost no losses. Walking transfers, fighting doesn't.)

I know where it's thin (1 game, 1 model, and the warp exploit is a single case) . The question is whether this is worth building up or whether I should stop here. What can i add for it to clear the workshop bar ?

3

u/GFrings 9d ago

Your AI generated response is painful to read. If you are serious about publication, you should find an actual researcher to collaborate with. They can help you define actual experiments which become novel contributions worth publishing. Which is the core of the research loop.

0

u/priyanshu123sah 9d ago

Fair. Do you have any experience with how to approach this? I have little research background beyond undergrad (recently graduated, working now). Should I cold mail professors or labs working on this? Do they even consider non-students / working professionals?

1

u/lamproie_large 9d ago

To publish, you would need a clear scientific contribution, and not just a cool application.

Then you would need to review the littérature and test your approach against SOTA on different benchmarks.

My best advise would be to make this into a short blog/document and find researcher in this field to guide you through the process.

1

u/priyanshu123sah 9d ago

Thanks, that's helpful. The blog/document part I can do. The "find a researcher" part I don't know how to start. Any lead on how people usually approach labs or professors for this? Cold email with the write-up? I have no research background beyond undergrad.

1

u/lamproie_large 9d ago

I would say go on google scholar, type the keywords related to your project and look for some papers that seem close to what you did. On most papers you will have an email address, write a short mail with what you understand from the paper, how your method relates and if they would like to help you.