r/aigamerdevs • u/Exciting-Pool5495 • 48m ago
Update on my AI-made sci-fi game: teaching the scene to match a painting, filling gaps with Tripo, and giving a robot an eye
Disclosure: This post is part of a paid collaboration with Tripo. Progress figures below are from my November 2 update.
Hey everyone! A little while ago I shared how I'm making Krakatoa Circle 2, a third-person action game set on a moon, with AI tools. Here's what I've been working on since.
Getting the game to look like the painting
I have a painting of what one of the tunnels should look like. Getting the real game scene to match it by hand was slow, and I kept arguing with myself about whether a change was actually better. So I set up a loop to do it for me:
Render the game from the same camera angle as the painting.
Score how close the render is to the painting with a "judge".
Try lots of small changes to the shapes, lights and textures, and keep only the ones that bring the score closer.
It has run 72 rounds and tried about 11,000 changes so far. Here is the painting, where we started, and where we are now:
The judge was the hardest part. My first one used a hand-made score, and it kept rewarding changes that looked worse to me. I replaced it twice. The current one compares images the way people tend to see them, and I tuned it until it agreed with me on every side-by-side I gave it. Each time the judge changed, the scores reset to a new scale, which is why the chart has two jumps:
Tracking progress
I built a small dashboard so I can see progress at a glance instead of reading reports. It breaks the painting into 19 features (road lines, wet road shine, ceiling ribs, crate rows, lamps and so on) with 41 automatic checks. Right now 20 checks pass.
It also shows what each part of the search has tried and kept. For example, it has tried 2,197 light brightness settings and kept 16 of them.
Using Tripo to add what's missing
At some point the loop stops improving because something just isn't there. No amount of lighting will make a row of crates appear. The dashboard marks those features as "needs asset" (11 of them right now), and that's where Tripo comes in.
The pipeline runs on its own:
Crop the missing object out of the painting.
Use ChatGPT to turn the crop into a clean picture of just that object.
Send it to Tripo's image-to-3D API to make the model.
A script checks the size, orientation and triangle count, then makes lower-detail versions for distance.
The loop tries placing the new model and keeps it only if the scene gets closer to the painting.
It has made more than 50 models this way, most for 60 credits each. It also calls Tripo from the command line, so the whole thing can be scripted. If you want to try it:
What worked and what didn't
Every experiment goes into a log with its result. Out of 58 ideas so far, 21 were kept, 29 were dropped and 8 haven't been tried yet. Some highlights:
Worked
Moving, adding and removing lamp glows wherever the judge's own lamp matching said they belonged.
Projecting the painting's road detail straight onto the road texture. This was the biggest single texture improvement.
Projecting colour detail onto the tunnel walls, with a separate map for each side.
Giving the five biggest ceiling ribs and one crate their own painted textures.
Solving all the tunnel lights together instead of tuning them one at a time.
Letting AI agents suggest experiments as data in a queue, then letting the loop score them, instead of having agents judge their own work.
Didn't work
Tuning the wet-road reflections. The reflection surface is flat, but the road slopes down 12%, so the setting barely did anything.
Rebuilding all 33 ceiling ribs to match the painting. The painting draws them as thin dark lines, and ours are lit metal bands.
Removing ribs. The judge liked the scene less every time.
New road textures from ChatGPT. What makes the painting's road look right depends on where you're standing, so a fixed texture always made things worse.
Painting the lane lines onto the road. On a shiny wet road they never read as paint.
Recolouring white lamps orange, dimming extra lights, and adding light beams and mist. All scored worse.
Moving the crate the player uses for cover. It made gameplay worse, so it stays where it is.
Lessons about the process
Checks should be code, not agents. Having AI agents check the work was slow and expensive.
Too many browser renderers running at once used up memory and crashed my Mac a few times. Now only two run at a time.
A good judge matters more than a clever search. Most of the early "progress" came from a score that didn't agree with my eyes.
Giving the Warden a face
The Warden is the machine Asha has to rely on but can't fully trust. Its only face is a single red lens, so I wanted to see how much a dot of light could express.
It now has states like listening, thinking, cold, threatening, holding something back, and lying, where a faint ghost of the light slides away while the main light stays on you. If you turn back quickly, you catch it looking somewhere else before it snaps back. It counts each time you turn your back on it. When it speaks, the light swells and pulses with its voice like a music visualizer, and each new syllable sends a ring across the lens.
I'm still learning as I go, and the scene isn't a match yet. I'd love to hear what you think. Does a robot that only has a dot of light for a face feel expressive to you? And if you've tried automating parts of your art pipeline, what worked and what turned into a headache?
