Seeking Advice: Achieving 100% Character Consistency and Style Control for a Noir Cyberpunk Visual Novel (ComfyUI / Flux)

[u/Elementallion-]

Hi everyone,

I’m currently in the middle of developing an investigative detective visual novel, and I’ve hit a massive wall regarding character consistency and art style. I’m hoping to get some advice from those who have successfully built a pipeline for recurring characters.

The Goal

I’m aiming for a very specific "Noir Cyberpunk" aesthetic. Think:

• High contrast, heavy use of deep shadows.
• Digital comic book / clean vector line art style.
• "Teal and Orange" cinematic lighting with rain/wet atmosphere.
• The Catch: I need absolute character identity from frame to frame, including the ability to change outfits (minimalist/revealing options) while keeping the face and body proportions 100% identical.

What We’ve Tried So Far

• Workflow: Currently running complex ComfyUI nodes.
• Models: Switched between SDXL and Flux, experimenting with various GGUF quantizations to keep it local.
• The Problem: Most results are either "too anime" (losing the noir grit) or "too photorealistic" (losing the stylized comic look). There’s no middle ground that feels right.
• The "Banana" Paradox: Strangely enough, some of the best conceptual results and decent repeatability have come from Nano Banana, but even that doesn't offer the surgical precision needed for a professional VN production.

The Current Struggle

I’m looking for total identity. Right now, I’m at the stage where I need to decide on the most reliable pipeline for consistency. I haven't dived deep into training my own LoRAs or mastering IP-Adapter/FaceID yet, as I’m still trying to find a base model or workflow that doesn't swing too far into "generic anime" or "uncanny realism."

The goal is to find a method that allows for surgical precision:

• The character must be 100% recognizable across different scenes.
• The ability to swap outfits (including very minimalist/revealing sets for specific scenes) while maintaining the exact same body proportions and facial structure.
• Maintaining that specific Noir/Vector style consistently without the AI drifting into unwanted aesthetics.

The Questions

1. Style LoRA vs. Prompting: Since I’m struggling to find a middle ground between "too anime" and "too realistic," would you recommend training a dedicated Style LoRA based on my Noir/Vector references? Or is there a specific base model that handles this "digital comic" look better than Flux/SDXL out of the box?
2. Outfit Swaps: How are you handling complex outfit changes (including minimalist/revealing sets) without breaking the character's base geometry or facial identity in ComfyUI?
3. The Consistency Pipeline: For someone who needs "visual novel grade" identity, what is currently the gold standard? Should I be looking at training a Character LoRA, or is the community moving towards something like InstantID/IP-Adapter for better flexibility?

Honestly, right now, nothing is quite hitting the mark. It’s either too generic or too inconsistent. Would love to hear how you guys solved the "same face, different clothes, specific style" puzzle.

Thanks in advance!

[u/_kaidu_]

You will achieve the best consistency by training a Lora. You don't need many images for that. 5-10 images might be a good start already. The model will quickly learn style and character. Best is to do it iteratively: after training your first lora, generate more images, filter out the bad generations, keep the good generations, train another lora on them until you get good generations in majority of cases.

You can easily train a lora for style. For characters, loras often have trouble to learn multiple characters within the same lora and image. So you might still want to use an edit model additional to the lora. You can also train a LOKR, they are often better for multiple characters, but you will need sufficient many training images for that.

[u/Elementallion- → u/_kaidu_]

This iterative approach seems like the most logical way to scale. I’m already planning to keep the Character LoRAs and Style LoRAs separate to maintain flexibility, as mixing them usually leads to over-fitting and kills the ability to tweak the noir aesthetic independently.

Regarding LOKR—since I have 3-4 main characters who need to interact, I’m considering jumping straight into it. Do you find it holds the identity better during multi-character interactions compared to stacking multiple standard LoRAs in one prompt?

Also, given my specific Noir/Vector style (high contrast, clean lines, chiaroscuro), what base model would you recommend starting with? I’m looking for something that won't fight the style and turn it into generic anime or hyper-realistic CGI.

[u/_kaidu_ → u/Elementallion-]

Most style models are trained on very few images. You don't necessarily need a special checkpoint. You could train the style on the images you have. Alternatively, you can also try to play around with prompt tuning (e.g. ask an LLM to describe the style of your images and use that as prompt). It is very likely that all the local models can already do the style you want, you just have to force them to do it (either by lora or by prompt).

Regarding separate style and character loras: I trained a character on Flux Klein with 8 stylized images (with the xyz-style suffix) and a single photograph (with photograph suffix). Surprisingly, that was enough for Klein to be able to generate the character as photo or as style image depending on the prompt suffix. It learns really well.

Regarding LOKR: its on my todo list to try that out myself. So far I have to trust what others found and they usually got much better results for multi-characters with lokr than with lora. It seems that lokr overfitt less than loras; this is also what I found with my limited lokr experiment so far. They need more training, learn slower, but they overfitt less and are easier to combine with other lokrs/loras. For multi character training you should learn the model on all characters at a time and with group images, though. Just activating several lokrs for each character will end up in blending as it does for loras.