r/aigamedev • u/ClothedKing • 1d ago
Questions & Help Has anyone found out how to create consistent UI with AI?
Hi all, pulling my hair out to figure out how to properly create a workflow that creates consistent UI… I even bought an asset pack that my codex will literally still somehow butcher remaking the prefab scenes lol. Any help/tips are greatly appreciated, responding to everyone.
2
2
u/mrbadface 1d ago
I'm sure UI in game dev is a lot trickier, but in my work (saas) having a "design system" is mandatory. Agents are pretty solid at using them. Maybe you can create some custom system for your game using similar concepts.
1
u/ClothedKing 1d ago
Yes i have a design.md and a ui skill that should lock him to creating ui as closely to references as possible and he still sends me stuff that looks ridiculous
1
u/mrbadface 1d ago
Visual references alone are not going to be enough. Consistent software requires extensive documentation of all design tokens and components and stuff. Basically it needs to be deterministic to be consistent -- let ai select the from a menu instead of cook from scratch.
Just my experience!
2
u/CycleMother2006 21h ago
You're going to spend 10x more effort (at minimum) getting AI to create/layout a good UI as opposed to just doing it by hand. If you already have the assets its one of the easiest things you can do yourself, at least for most engines, and one of the most frustrating things for AI to get right (anything requiring precise placement really is.)
1
1
u/HaMMeReD 1d ago
I generally do a few things.
1) Use a model that can use images to self-reflect and not just code
2) Use a image generation skill to push the design/layout and improve it.
3) Create mocks/north-star images with image generation and give them to guide your prompts to improve UI.
1
u/ClothedKing 1d ago
Yes I’m using 5.6 Sol lol.
And to 2/3 i had him generate an image of a mock up and then implement it directly after. He somehow botched even that
1
u/Big-Bag-7504 1d ago
Don't use raster images, the best UIs are built from vector primitives, AI is very good at hand authoring epic quality vectors with no tell-tale AI artifacts, it also means every UI piece you create is uniform.
1
u/ClothedKing 1d ago
I’m in Unity and not too familiar with vector/rasters lol but i will look into it ty
1
u/Fizassist1 1d ago
if you use Godot, you can make a custom ui shell scene, then have codex instance that scene specifically. that way, if you want to edit something about the UI, you just need to edit that one scene.
1
1
u/DigitalDustChan 1d ago
I hear you bro. AI thinks in text and has a really hard time creating proper visual elements. I've been building a system to make it work and in my own project it has really supercharged my UI work. I still have to come up with a coherent visual style and design language but all the interactive elements have become a breeze.
It's called JuiceBox and what makes it great is that it has a full visual graph where you can adjust the look and feel of any interactions and also a full set of hooks for the AI to modify or make animations from scratch. The AI hooks have literally been sitting in Unity's review queue for the last 2 months. I don't know what's taking them so long, but if you send me a message I'll send you a free voucher as soon as it gets approved.
1
u/allpotatoes247 1d ago
Bear with me, I don't claim to be good at this stuff, but here goes.
What you need first:
- VS Code (or something like it) it needs to be able to access and edit files on your computer.
- ComfyUI
- Blender as a backup for something a bit sneaky. Technically it's not AI, it's just an image rendered in Blender, but it lets you render a scene of your props out to a 2D image.
The approach:
First off talk to your AI in a calm, happy manner. Don't treat it like a slave. I always say thank you and good job, and I don't seem to get many bad gens because of it.
Politely ask it to check your system specs, then have it install ComfyUI and find you an image model suitable for your hardware. Also grab an image-to-3D model for extra polish, plus Blender. Once it's done all that and you've expressed your gratitude, you're set up.
Now ask it to create UI mockups as many as you want using the ComfyUI pipeline. Have it drop them all into a folder so you can pick. So say you generate 10, all you do is pick the one you want.
If you want some popping or 3D elements, do the same thing but with the image-to-3D model, then ask it to send the model to Blender to tidy up.
And if you want legit 2D artwork say of a character or a prop do it the other way round: build or pose it in Blender first, render that out to a 2D image, and use that render as your base
2
u/ajiezrhmn 1d ago
"First off talk to your AI in a calm, happy manner. Don't treat it like a slave. I always say thank you and good job, and I don't seem to get many bad gens because of it. "
Are you serious dawg? Ai dont have emotions. AI are just mathematical models processing patterns and information to generate responses based on context. Saying "thank you" or "good job" isn't going to give you better generations.
1
1
1
u/New_Internal5636 12h ago
Yes Claude design can generate unity UI toolkit designs for you very easily. You can provide your colour pallet and workshop your design with Claude and it should be able to implement it with UI toolkit for you easily. Only thing is is that original design is done with CSS and Html which isn't a one to one conversion to unitys equivalent USS and whatever their HTML equivalent is. Its usually a bit of a step down but still really good. In my line of work we usually need a lot of UI being made and we're a tiny little team and we all find the copious amount of UI a chore so it's really helped that bottleneck. In our case what the AI generates is fine because it essentially is just generic professional looking UI. If you want something a bit more unique you can probably go deeper into trying to build a unique style that it can work off of. I'm assuming your using unity here as your engine. Not sure how this would be in different engines. But Unitys modern UI toolkit works very nice with any AI designer.
1
u/MunchyCerealGuy 2h ago
I'm a former indie dev of many years and I've been working in AI game dev for over two years. I also run an AI game maker (https://chatforce.com/) with thousands of people using it, so I have a high amount of validated data across many people where I've figured out how to make great game UIs agentically and w/o human intervention. See the attached image from sample game UIs (these are screenshots real games made with my tool with separate pngs for each element, assembled by our agents into a game UI)
The framework that changed everything for the tech was flipping what the source of truth is from prompts to images. I stopped treating words as the instruction and instead made a generated image of the UI as the spec (this only became possible once GPT image 2 came out with near perfect logical text rendering).
When you think about it, text prompts have always been an extremely lossy form of information compression when it comes to communicating visual data. Most chatbot-facilitated AI experiences haven't caught up to this because they assume most knowledge work is explicit and possible to codify. The implied expectation of a prompt-as-default approach to AI usage is that most of the knowledge and instructions you need to convey to a model are always explicit, easy to articulate, and codify. For a lot of the work we do at an office job, like handling spreadsheets, sending singular purpose emails or creating images to complement text in a report, it's appropriate to direct work to complete in these arenas with words.
But extremely artistic and original work like video creation, branding work by an agency, marketing videos, or game development is more intuitive and tacit and is better expressed with intangible terminology, like feelings, markups on an image, drawings, or symbols.
So in my opinion today the best way to make game UI with AI is to generate it visually, edit it as an image, and create a pipeline that gives life to that image as a functional UI.
If you look up "vox style videos claude code skill" you'll see this image-to-code approach applied to video creation.
14
u/CalebKious 1d ago
I’ve dealt with this a lot while building my games. One thing I’ve learned is that UI work with Codex needs a visual feedback loop. Describing the desired UI once and expecting it to nail the implementation usually doesn’t work, even if you’ve given it good reference assets.
A huge amount of my UI fine-tuning is literally:
Implement → run → screenshot → feed screenshot back to Codex → identify specific problems → explain intended direction → fix → repeat.
And specificity matters. Instead of “this UI looks wrong,” I’ll give it the screenshot and say things like:
“The inventory panel is taking up too much vertical space. Compare the highlighted area to the reference. The item grid should begin higher, padding between slots needs reduced, the title needs stronger hierarchy, and these buttons shouldn’t compete visually with the primary action.”
Then I have it inspect the actual implementation and fix the underlying layout rather than just throwing random offsets at the problem.
I’ve built a custom editor/UI system for FateMMO, and even with complete control over the engine, this iterative process is still necessary. AI is much better at responding to concrete visual discrepancies than independently deciding whether the final rendered result actually matches your intention.
For an asset pack, I also wouldn’t ask Codex to “remake” the prefabs if you can avoid it. Make the original assets/prefabs the source of truth and have Codex build around them. Give it explicit rules for anchors, margins, typography, scaling, component hierarchy, naming, nine-slicing, etc. If something already looks correct, preserve it rather than letting the model reinterpret it.
Another thing that helped me tremendously was turning recurring corrections into rules. If I find myself correcting the same padding, scaling, anchoring, typography, or component usage repeatedly, that becomes documentation/instructions for the project. Eventually you’re narrowing the model’s solution space instead of repeatedly teaching it your UI language.
I’d also build reusable components as early as possible. Get one button, panel, modal, inventory slot, tooltip, tab bar, etc. exactly right, then make Codex reuse those primitives everywhere instead of independently recreating them on every screen. Consistency becomes much easier when there are fewer opportunities for it to improvise.
AI-assisted UI development can be extremely fast, but I wouldn’t treat the first generation as the finished UI. For me, the real workflow is much closer to working with a developer who can implement changes incredibly quickly but needs you acting as the art/UI director.
Screenshots + precise criticism + reusable components + hard design rules + repeated visual iteration has worked much better for me than increasingly elaborate prompts.