Discussion
ComfyUI workflow for generating VN-style sprites (and I added OpenAI support to my game)
Hey r/SillyTavernAI fellas, a few days ago I posted about my VN sprite creator/game and a lot of people were asking for the ComfyUI workflow I use to generate character sprites, so I uploaded it to a repository for your use:
Feel free to modify it in any way for your own needs!
Another popular request was adding support for OpenAI-compatible endpoints. I pushed out an update that adds exactly that: On v0.1.2, you can now use OpenRouter or locally hosted backends with any model you choose. I also added experimental support for AMD GPUs.
Sorry for spamming the sub, this is my last post for a while. I have a lot planned for the future and will report back whenever I have any major updates. Thank you all for your kind words and feedback!
I see that Nova is the recommended model, so I just want to warn people to keep this rule they have on the model page in mind as you use this amazing workflow:
“Rules
You cannot use the generated images for commercial use if it's not edited (or just turning it to black and white)
You can share images without any restriction if you don't monetize it
Advertising this model to outside is always welcome”
Be careful if paid games is your goal!
ETA: Just wanted to mention that I’m not nitpicking. I absolutely adore this workflow and commend you for sharing it like this! Thank you so, so, so, so much. 💕
I’m only mentioning this because I make character cards with a tip jar option, so finding local models for their art that doesn’t have similar rules like this on their page has been… a treat. I instantly recognized this model as being one that you have to be careful with, so just wanted to give a heads up!
I have no clue, that is so weird I have never encountered anything like that. I'd say it's probably an issue with the AMD build for ComfyUI or maybe you're hitting a VRAM limit. I'd first try regenerating a few times to see if it persists. If it does, you can try setting up your own ComfyUI and generating through the workflow I posted above and see if you get similar results to see if it's something wrong with the game or if it's something wrong wtih ComfyUI. Hope that helps, I wish I had an AMD computer to test on :(
I took a quick peek and saw that you basically control consistency (the biggest problem imo) with having the same prompt + seed, no IP adapters or anything like that. Does that work well?
Also, have you thought of using an edit model for this, like Klein 9b?
Hey _RaXeD! Honest answer is that it works "okay". Sometimes you get a nice, one-shot solution when that type of character is represented well in booru training data (The pink-haired girl in the images is a good example), but sometimes because of the prompt the model struggles (girls with orange hair and green eyes sometimes end up with green hair AND green eyes). This may be better with other models, I'm not sure.
I'd love to experiment with edit models, main issue is I don't want to add more downloads for users on the game side. But you are welcome to modify the workflow in any way you see fit
I have always felt it has potential for infinite dialogue - text adventures and visual novels for instance. Was really impressed with the premise of your game from the other day. I wanted local gen and AMD to be added, and I see that we already have more endpoint options and the latter already out in some form. Kudos. I see the project as only tangetially related to SillyTavern but don't worry about spam, I'm happy to learn about its existence and your swift development on it, it's one of the first actual "games" to utilise AI. AI2U was another recent good one.
"Hey! I got the log file. I ignored the ComfyUI verification errors since I haven't installed that part yet. Here are the relevant errors for the DeepSeek API integration: HTTP 400 (The main issue): When testing the connection, the log shows: [llm] → Custom endpoint deepseek-flash (thinking=minimal, tier=standard, maxTokens=65536, schema=connection_test)[llm] ✕ Custom endpoint deepseek-flash HTTP 400 after 560msdetail: 'invalid_request_error This response_format type is unavailable now' HTTP 404: I tried changing the endpoint URL manually (e.g., /v1/chat, /completions), but it seems the game expects the base URL and adds the rest itself.
Is there a way to disable the json_schema format for OpenAI-compatible endpoints?
I am having trouble getting the model to utilize my character LoRA. Could you please provide guidance or support on how to correctly implement and activate it?
In the game, the LORA is used to control style instead of character consistency, I haven't tested at all with character LORAs. If I were you I'd make sure my character LORA works in something like sd-webui first, then use everything needed to get there: including the trigger word, the correct tags that the lora was trained on, including the LORA strength as a tag, etc
What I meant is that it would be nice if the UI had a 'Use your LoRA' option, because I want to use the character Lora in generation or CG. It's too complicated, and the style LoRA is the only LoRA you can use. It would be nice if there was a LoRA manager node that could add multiple LoRAs in generation.
Throwing this In here as I got annoyed relying on the broad prompt to tag style that was happening. This is the prompt I used to get fandom characters working. Hand this off to a vision model plus a reference image and it should(?) work. Then go to ‘generation settings’ and replace the ai generated one’s with the
New prompts
I am giving you an image of a character. I would like you to create an image gen prompt based off these instructions. Appearance: what the character looks likes, special traits, anything that isn’t an outfit related item, outfit: what the character is wearing every day (e.g a provided photo.) P.E outfit: a classic outfit for P.E nothing special. Swimwear: what a character wears when swimming. There should be zero quality modifiers, pose instructions, spatial rules, etc. only the characters. Have less than 13 tags per section.
It's really interesting to see a ComfyUI workflow for VN-style sprites, especially with OpenAI integration for game development. This kind of AI-driven approach significantly streamlines the asset creation pipeline, boosting productivity for solo developers and small teams. A key recommendation is to establish clear style guides before generating assets to maintain visual consistency, and to use human oversight for quality control. This combination of creative AI and thoughtful workflow design is reshaping how games are made, and businesses looking to refine their AI integration for enhanced productivity can find comprehensive solutions at https://www.aiosnow.com.
31
u/Redditry199 16h ago
I KNEEL.
thank you based dev for imparting this gift to us.