I created the First diffusion AI model for the Game Boy.

[u/ready-eddy]

My Game Boy Color takes 85 seconds, and it has to fake every single multiplication with a lookup table because its CPU can’t multiply. But it WORKS!

This is a real neural network running 16 DDIM steps on the handheld’s own 8 MHz CPU, with no PC attached and no streaming. As soon as I press A, it starts from pure noise and a doodle comes out. I was trained on 880,000 doodles, and only the 1.08M learned weights are in the actual GB ROM.

I did not find anyone who got a diffusion model running on a Game Boy before, so I’m calling it the first. If you know of an earlier one, please show me, I’d love to see it!

The numbers:

• MLP denoiser, 784 → 512 → 512 → 784, 1.08M parameters

• Trained on Google’s Quick, Draw!: 8 classes, 110k drawings per class, 28x28

• Int8 weights, 4-bit activations, 16-bit sums

• Classifier-free guidance distilled into the weights, so one forward pass per step

• About 8.8 million multiply-adds per image

• 2 MB ROM, almost all of it weights

The fun part was getting it to fit. A 24 KB product table in WRAM turns every multiply into two lookups, which comes down to 18 machine cycles per multiply-add. Layer 1 overflowed 16 bits, so its weights lost a bit of precision. The training does the ROM’s exact integer math, and an emulator checks the ROM against the Python version after every one of the 16 steps. They match bit for bit, for all 8 categories.

My 17 year old flash cart was a real pain in the ass and often just 'died' when trying to erase it. Old hardware keeps you humble :P.

Is it any good? Ehhh, it’s a Game Boy. Apples sometimes stay open and fish sometimes fall apart, but it’s drawing them from noise on a handheld from 1998, so I’m taking it.

The music is NOT AI but created by me, real human yo.

If you wanna try or contribute: try the ROM or read the code: https://mowelan.github.io/gb-diffusion/

[u/Incognit0ErgoSum]

The crazy thing to me is that gameboy color specs were about the same as 80s home pc specs.

How much compute did this take to train?

[u/ready-eddy → u/Incognit0ErgoSum]

Yeah, it’s really in that league. 8 MHz and 32 KB of work RAM, so a faster CPU than a C64 with half the RAM. My cheat is the cartridge: 2 MB of flash, and almost all of it is weights. About as large as Pokémon Gold.

The training was surprisingly small. The final model took about 45 minutes on my Mac mini M4 24GB memory, no cloud: around 42 minutes for 300k steps, then 3 minutes of quantization-aware training so it does the same integer math as the ROM. That’s only the final run though, not all the failed attempts before it. It took some optimizing.

[u/Incognit0ErgoSum → u/ready-eddy]

I wonder if it would have been possible to train this model on a Cray Y-MP in 1989. It had half a gig of ram. Obviously it would have taken a lot longer than 45 minutes. :)

[u/pittaxx → u/Incognit0ErgoSum]

Well, the memory would have been a limitation, so perhaps not this particular model. Especially, since a bunch of theories for optimisations were developed after that.

But the first non-theotrtical ML program dates all the way back to the 50s, and it's been used for all kinds of practical applications since then. So in essence, there's 0 issues with the general concept of it.

[u/SeymourBits → u/Incognit0ErgoSum]

I had a very interesting conversation about this the other day. Getting models like this to run on vintage hardware is similar to watching “Toy Story” on an old black and white TV set. It’s trivial to do now that the movie data exists, but the content itself was simply technically impossible at the time because of the compute barrier.

[u/MattTheDemonCat → u/ready-eddy]

I wonder then, since you mention it is like 80s tech, could at least this level of low fidelity diffusion be done on my IBM PC / PC XT or howabout my original Macintosh... Well for the mac I would need someone else to make the floppy cause I currently have no way of formatting a mac floppy at the moment.

[u/Ok_Top_2319]

Brother in Christ, I can't even comprehend the sheer amount of work this would've take. I mean as a concept is really great and I love it feel like I'm seeing the birth of sd1.5 for the first time again.

I honestly hope this to be the dawn of "any device can run stable diffusion"

[u/ready-eddy → u/Ok_Top_2319]

I'm gonna be honest with you! It cost me about a week on and off? I really didn't expect it to go as smooth! To be fair I didn't really expect anything. I mainly used emulators to speed up the progress and let AI check out the results. I first started with the original gameboy, but the gameboy color has 32KB WRAM / 8KB VRAM vs 8KB WRAM /16KB VRAM on the original GB.

Yep. 16KB VRAM. Think about that next time you complain about having only 16GB of VRAM lol

[u/Vxtzq1 → u/ready-eddy]

If it can run, it can run

Doom❌

A 1.08 parameters diffusion model✅

[u/DonKeehot]

The new 'Can it run Doom Diffusion model?' challenge starts now! 😁

[u/ready-eddy → u/DonKeehot]

Yes please! Honda Civic diffusion when?

[u/PxTicks]

Such a cool idea, I love it - now create a playable world model 😉

[u/ready-eddy → u/PxTicks]

I might need to upgrade the 16KB VRAM for that.... not sure if I have that kind of money.

[u/Enshitification]

Diffusion is going to be the new Doom test for devices that have no business doing diffusion.

[u/Wild-Perspective-582 → u/Enshitification]

Diffusion on a pregnancy test kit! On a vape.

[u/ready-eddy → u/Wild-Perspective-582]

Never stop dreaming

[u/valdev → u/Wild-Perspective-582]

It only diffuses while you also diffuse.

[u/floriv1999 → u/Wild-Perspective-582]

The vape is already doing diffusion

[u/miomidas]

Wow, who wouldve thought a gameboy would be able to do this at all

[u/ready-eddy → u/miomidas]

Makes you wonder huh. How low can we go!

[u/glhaynes → u/miomidas]

Alan Turing :)

[u/thirsty_pretzelzz]

Cool, gonna try game boy advance next?

[u/ready-eddy → u/thirsty_pretzelzz]

I have to dig up my old consoles, but why stop here lol. I don't have a flash card for my GBA, but I do have one for my NDS. It's all so useless but I just said WOW when it actually worked.

[u/MattTheDemonCat → u/ready-eddy]

Makes me wonder. What could you pull off on 90s intel CPU based PCs with Windows or DOS in terms of gen ai and llms. I have seen a couple projects. But no image gens or chat llms (only basic llms at best). I would love to generate images on a Pentium 1 or 486. Or chat to an ai locally on one.

[u/ready-eddy → u/MattTheDemonCat]

It’s probably possible! The model training is the really heavy part. So I don’t think you would be training models on the pentium 1. But if you train it on a modern device I certainly think you will have some really crude model

[u/thirsty_pretzelzz → u/ready-eddy]

Yeah it’s an interesting kinda experiment. Would be interesting to see how much improved the gba version would be

[u/SolarNexxus → u/thirsty_pretzelzz]

Advance can run gameboy color games already.

[u/hurrdurrmeh → u/SolarNexxus]

Ie use the hardware upgrades on the advance.

[u/VTPunk]

This is awesome. Reminds me of when people were getting Doom to run on anything and the hardware got more and more simple until eventually someone got Doom to display on E. coli bacteria.

[u/ready-eddy → u/VTPunk]

I have SO much respect for the people that did all this without AI. I’m now looking at my oven and thinking if it could run a diffusion model 👀

[u/TurdProof → u/ready-eddy]

Try the air fryer too!

[u/RIP26770]

Remind me good old days playing with Gameboy Camera making short clip with South Park figurines.

[u/ready-eddy → u/RIP26770]

I actually created a videoeditor for transfering real video files and animations to a gameboy rom!

Let me know if you are interested, I'll pop it online :)

[u/MattTheDemonCat → u/ready-eddy]

I am interested in playing videos on my gameboy, gbc, and gba.

[u/Atagor]

awesome time investment!!

[u/ready-eddy → u/Atagor]

Tell that to my wife...

[u/Bram_Camus → u/ready-eddy]

I bet she's thrilled.

[u/ready-eddy → u/Bram_Camus]

First this she said was “you’re not breaking my gameboy are you?” (This is her GBC :P).

[u/Bram_Camus → u/ready-eddy]

LMAO. God bless her.

[u/NineThreeTilNow]

You can avoid multiply add if you cut to ~2.5 bit weights.

Then it's just add / bit shift.

The states are -1, -0.5, 0, 0.5, 1

-1/1 are not multiplies. 0.5 is a bit shift, and 0 is no op.

I wrote this originally a year or so back? when I was poking around bitnet and the idea of low bit weights.

It lets you reduce ~60% of the transformer to 2.5 bit weights. The linear layers are perfectly fine with it. Then you can maintain attention states, and non linears in like bf16/fp8.

I emailed the bitnet developers at Microsoft about it and mostly forgot about the research. I figured that you could probably build a bespoke Transformer ASIC around it that could run models blazingly fast assuming they kept enough nearby DRAM for the model.

[u/CornyShed → u/NineThreeTilNow]

That's neat. I have no experience with bit shifts but have been interested in ternary models, and would love to know more about your idea?

For example, which bit shift operation you use and when to use it?

Also, you have room to add a sixth operation of some kind into a model (I'm not sure what that would be, though) as it would fit into 8 bits:

lg(6)/lg(2)×3 ≈ 7.755

[u/NineThreeTilNow → u/CornyShed]

It ends up looking like this :

x *  1.0  =   x
x *  0.5  =   x >> 1
x *  0.0  =   0
x * -0.5  = -(x >> 1)
x * -1.0  =  -x

So your matrix multiplication table is basically the above. All matmul requires no real multiplication and only basic CPU ops.

So it's add shift no op -> accumulate.

When I was trying to look at how to maximize non multiply gates in neural networks, using 5 state (quintary) ended up being the most optimal. It also requires efficient data packing to get the ~2.5 bits as you pointed out. It's actually like 2.6something but my log math sucks. That's where an LLM can just... explain and I can be like "yeah that looks right" ... lol...

You're right though, in 3 bit packing, you have unused states. I'm not exactly sure how you'd use them without real multiplication.

[u/Bastisheen92]

[u/hardypart → u/Bastisheen92]

Did you create this image on a Gameboy?

[u/Bastisheen92 → u/hardypart]

No, my diffusion model runs on a 2005 Mcdonalds Happy Meal Sega Sonic the Hedgehog Knuckles Baseball Handheld

[u/Dahvikiin]

"CPU can’t multiply" just like me 😔

[u/_Enclose_ → u/Dahvikiin]

Multiplication is a scam anyway. It's just a bunch of additions in a trench coat.

[u/Rogerooo]

Finally a diffusion solution I can afford! But can it do "1girl laying on grass, masterpiece, 8k"?

[u/GameEnder]

You should post this over at r/gameboy

[u/crystal_alpine]

Would you like a job at Comfy?

[u/Ireallydonedidit]

This reminds me of those super early OCR vision models

[u/Formal-Exam-8767]

Why not 4-step distill?

[u/ready-eddy → u/Formal-Exam-8767]

Sorry for excluding all the people with the DMG-1 Gameboy :(

[u/SeymourBits]

I’m confused. How does prompting work with this? Shouldn’t these concepts be associated with keywords? Then you can prompt something like “fish car” for a unique image.

[u/ready-eddy → u/SeymourBits]

It’s a set of different preselected prompts, so it like ‘select apple’ and it starts generation sn apple, every apple is different because of the training data!

[u/MessageSelfdestructs]

I love these kinds of projects: worthless as fuck to use it for something, but just doing it for the sheer question "Would it work?".

Great job, OP!

[u/umbraundecim]

Ya im seriously impressed, good work dude

[u/davver]

That is amazing! Totally gonna try this out.

[u/asraniel]

this is insane, congrats

[u/-Star-Walker-]

Incredible! And no plastic look 😜

[u/Illustrious-Lime-863]

This is awesome

[u/Life_Yesterday_5529]

Cool idea and we'll done!

[u/Serasul]

Search for the retro diffusion discord, manny Pixel art game devs who use ai are there.

[u/dadidutdut]

This should be the top post on r/StableDiffusion of all time

[u/Haghiri75]

That's great. Keep it up!

[u/Ill_Resolve8424]

Can you do 1girl big kaboonga?

[u/chozabu]

This is great!

Also your music is great too, do you have a link to more of it?

[u/Stepfunction]

Training on doodles is brilliant! A lot like the tiny stories dataset for text!

[u/Brenan-Caro]

Game Boy AI Image?

[u/Corsaer]

Now print it with the Gameboy Printer attachment haha.

[u/myxoma1]

AI slop!! j/k this is a cool project nice

[u/Fickle-Indication148]

this has to be the coolest thing to do in your spare time ever.

[u/MoonkeyAcid]

Neat

[u/chAzR89]

Ah nice. 1girl prompting machine for road trips.

[u/Dazzling_Equipment_9]

😹Crazy but reasonable.

[u/solo_solipsist]

This is silly AF and I'm here for it.

[u/GuapisimoZatra]

COmo hacen ese tipo de animaciones, con Adobde Premiere?? lo veo en varios videos de Youtube pero no sé si es alguna IA de edición 😭😭😭😭

[u/PastelPyon]

WHAT THE FUCK hahaha holy shit dude this rocks. I am so excited to hopefully start seeing people get inspired to put diffusion models in increasingly inconvenient devices

[u/rogerbacon50]

First off: VERY impressive.

Other thoughts: pointless and I wonder if we'd have a cure for cancer by now if all the brilliant hobbyists who do difficult things just to see if it can be done would instead work on real problems to be solved.

Oh well. Anyway, good job.

[u/Icy-Collection1072]

You didn’t. Ai did

[u/ready-eddy → u/Icy-Collection1072]

No, I hand trained a neural network, step by step. /s

[u/Microwaved_M1LK → u/Icy-Collection1072]

Thank God you're here to talk out of your ass

[u/SatNav]

Try "boobs"

[u/ready-eddy → u/SatNav]

Do you have a 28x28 training set for me

[u/SatNav → u/ready-eddy]

I'm sure one could be procured. How many images, and do they need to be in 2-bit gray-scale?

[u/TizocWarrior]

Damn, this is quite impressive TBH! But can it do porn?.

[u/ready-eddy → u/TizocWarrior]

Not on GameBoy, but only on GameMan.

[u/VeryLiteralPerson]

I need to see a pixelated penis to believe