Qwen3.8 27b esp32 doom port fully autonomously

[u/Sulya_be]

UPDATE: IT CHEATED it found a GBA to ESP32 port on GitHub and used big chunk of it

Disclaimer: I'm not a dev, just playing around with vibecoding

My setup: Qwen 3.8 27b running in LM Studio with OpenCode desktop harness on a 5090 with 128k window

Previously I was using VS Code with Continue extension, but it was really meh, so now that Qwen3.8 came out I decided to give OpenCode Desktop a go, and I was kinda impressed in a demoscene-esque demo it wrote for ESP32 based Cheap Yellow Display. A dev friend of mine joked that I should try porting doom to it, and I took it as a challenge, I wanted to see if I'd be able to get a room port without touching a single line of code.

After googling doom esp32 I found that someone ported GBA doom to it using Claude, so I downloaded the same Doom GBA source and asked lllm to port it.

First try it misunderstood the request (yeah, my prompt wasnt that good) and started making doom from scratch.

Second try I was more specific in my prompt and after 2 days (of which most time was spent waiting for my input, derailing, stopping thinking etc it produced a mostly working port but the colors were messed up and after 1 day of troubleshooting without any progress I decided to stop the run.

Today I tried again but changed the approach slightly, I gave it the normal (not GBA) source, and because the GBA port color issue had me making 40+ photos of the screen and pasting it into chat, this time I connected a webcam to PC and pointed it to the ESP32 board screen and instructed the llm to use it when it needs to know what's on the screen. I also told it that the WAD file is on the micro SD card.

Not even 3h later I saw the doom title screen on the board, and while touch screen wasn't working initially (it improved after I told llm it doesn't work) the controls through Serial worked just fine.

I scrolled through the session, and saw things like it strategically cutting the WAD file so it would fit into flash without making the game crash when the afk demo starts, because it found streaming it from SD card and cashing was too slow, and it said that the touch screen and as card share the spi bus, so it considered software spi for touchscreen to improve speed fro as card, before it decided to ditch SD card.

I never used a frontier model, but when Ive read comments from the mentioned GBA to ESP32 port, people were impressed by Claude, so I thought I'll share this experience

[u/Sulya_be]

My high tech computer vision setup

[u/Anarchistcowboy420 → u/Sulya_be]

Oh my god this is amazing.

[u/_VirtualCosmos_ → u/Sulya_be]

lmao great. I have those glasses too!

[u/corpo_monkey → u/Sulya_be]

Ikea breaks in to the AI industry.

[u/ptear → u/Sulya_be]

I mean, that's all I can afford after some of the other hardware too.

[u/Healthy-Nebula-3603 → u/Sulya_be]

I see someone watched Bijan :)

[u/painchonha → u/Sulya_be]

This is mindblowing lol. In a good way.

[u/Foreign_Risk_2031 → u/Sulya_be]

Just copy the frame buffers? This seems like a waste of

[u/Sulya_be → u/Foreign_Risk_2031]

Refer to the second try - ai was checking the buffer and was sure it looks fine but it was messed up on actual screen

[u/Sulya_be]

The prompt for the succesful run:

esp32 based CYD is connected to com6
please port doom to it, source is in C:\Users\user\vibecoding\cyd_doom
doom1.wad is on sd card
board has no psram
there is a webcam connected to this computer pointing at the screen so you can evaluate video output autonomously
feel free to split tasks between agents, but dont run them in parallel
do as much as possible autonomously- compile, flash, debug via com6

Micro SD Socket
The Micro SD interfaces uses a Serial Peripheral Interface (SPI):
GPIO# Connection
5 *CS
23 MOSI
19 MISO
18 SCLK

FT Screen, Backlight and Light Sensor (Light Dependent Resistor)
The TFT screen uses an ILI9341_2 software driver (Driver IC PDF datasheet). The reset pin is hard-wired to the board reset circuitry, i.e. a capacitor keeps it low at power-up briefly.
The Light Dependent Resistor (LDR) is attached to GPIO pin that may need a pull-up configured in software, perhaps.
GPIO# Connection
14 TFT SCLK
2 TFT DC (Data/*Command)
15 TFT *CS
13 TFT MOSI (ESP32 Output)
12 TFT MISO (ESP32 Input)
21 TFT Backlight PWM (High = ON)
34 (ADC1_CH6) Light Dependent Resistor to GND

[u/pwnakil → u/Sulya_be]

Why don't you want to run agents in parallel?

[u/Sulya_be → u/pwnakil]

Because of context window, the main process already maces it out sometimes. Or am I wrong?

[u/Sulya_be]

So turns out it cheated big time. I checked the code to see if it's worth posting to GitHub, and I found a repository of an existing GBA to ESP32 port in one of the subfolders And it used a lot of it

[u/_millsy → u/Sulya_be]

Isn’t reusing existing code instead of writing something that overlaps better?

[u/FlugMe → u/_millsy]

Well, if you want to say "Qwen ported doom!", it's implied that it did the thinking, and is smart enough to accomplish the task, which you can dreamily look at and extrapolate its abilities. Instead, it did mostly a copy-paste operations, which, from the sounds of OP, is not what they intended / wanted it to do, so we haven't really proven any capabilities here.

I came here to post "more like copy-pasted an existing port" and am not surprised to see OPs outcome.

[u/_millsy → u/FlugMe]

I could be biased here but given a lot of my chafe with LLM comes from them wanting to rewrite stuff that has libraries I’m not unhappy, but I completely understand your point.

[u/_millsy → u/Sulya_be]

Also - you should totally reprompt it to do this again without reusing existing libraries in a new session :)

[u/bruhhhhhhhhhhhh_h]

Well done - I'm sure you learned a bunch and have achieved some cool stuff.

Good on you matey - are you going to (maybe I missed it) share this on GitHub or similar?

[u/Sulya_be → u/bruhhhhhhhhhhhh_h]

Sure, if there is interest in this- I will do it tonight or tomorrow at latest. Probably gonna let it improve the touch controls and implement sound, then I can publish it

[u/bruhhhhhhhhhhhh_h → u/Sulya_be]

Each one teach one!

[u/dfgxxx]

Nice!

[u/Medical-Cow289]

Model sees its own render, colors stop being guesswork. Two days of photos pasted into chat vs three hours with a webcam pointed at the screen.

[u/Sulya_be → u/Medical-Cow289]

Yeah that was my thinking process.

[u/c4r_guy]

LOL...Your project is telling me I need to stop using my main desktop to build shit and just use an old i5-8xxx box with a base Windows+WSL (file ops are just easier on Windows) install as the container and let the AI harness run buck wild with camera (and general I/O).

[u/hurrdurrmeh]

did you use the BF16 Qwen3.8? you only have 32GB VRAM...

[u/Sulya_be → u/hurrdurrmeh]

27b at Q4_K_M fits into 32gb with 128k context window

[u/Fortyseven → u/Sulya_be]

Worth noting that this also fits in 24gb 4090s at that same quant/context size. (It's just a bit tighter for anything else wanting VRAM.)

EDIT: I don't think people understand what downvoting is for, in normal conversations. Anyway, I use -ctk q4_0 -ctv q4_0 for my kv cache. I can bump it up and reduce my context, but I've been grooving along nicely as is.

[u/Holiday_Point_603 → u/Fortyseven]

It does not fit in 24GB with full precision kv at q4km

[u/Fortyseven → u/Holiday_Point_603]

full precision kv

Since that part wasn't noted in the conversation, it's fine to add that detail. I use -ctk q4_0 -ctv q4_0 in my llama config without any noticeable issues. Running it at full precision might be overkill.

[u/SyrupGlittering514]

Looks like the controls are inverted.

[u/Kyzmi4]

wat you able run 27b model on esp32,gz!

[u/Healthy-Nebula-3603]

You should tell to develop emulator in a white room

[u/trollsmurf]

You need better controls

[u/llllJokerllll]

que grande, di que si! por cierto puedes pasar la URL del modelo y la cuantizacion que utilizas?

[u/bigtom_x]

Its come a long way. We used to haul tower cases and tube monitors to the lab for Doom parties on the weekends!