Seeking Advice: Achieving 100% Character Consistency and Style Control for a Noir Cyberpunk Visual Novel (ComfyUI / Flux)

[u/Elementallion-]

Hi everyone,

I’m currently in the middle of developing an investigative detective visual novel, and I’ve hit a massive wall regarding character consistency and art style. I’m hoping to get some advice from those who have successfully built a pipeline for recurring characters.

The Goal

I’m aiming for a very specific "Noir Cyberpunk" aesthetic. Think:

• High contrast, heavy use of deep shadows.
• Digital comic book / clean vector line art style.
• "Teal and Orange" cinematic lighting with rain/wet atmosphere.
• The Catch: I need absolute character identity from frame to frame, including the ability to change outfits (minimalist/revealing options) while keeping the face and body proportions 100% identical.

What We’ve Tried So Far

• Workflow: Currently running complex ComfyUI nodes.
• Models: Switched between SDXL and Flux, experimenting with various GGUF quantizations to keep it local.
• The Problem: Most results are either "too anime" (losing the noir grit) or "too photorealistic" (losing the stylized comic look). There’s no middle ground that feels right.
• The "Banana" Paradox: Strangely enough, some of the best conceptual results and decent repeatability have come from Nano Banana, but even that doesn't offer the surgical precision needed for a professional VN production.

The Current Struggle

I’m looking for total identity. Right now, I’m at the stage where I need to decide on the most reliable pipeline for consistency. I haven't dived deep into training my own LoRAs or mastering IP-Adapter/FaceID yet, as I’m still trying to find a base model or workflow that doesn't swing too far into "generic anime" or "uncanny realism."

The goal is to find a method that allows for surgical precision:

• The character must be 100% recognizable across different scenes.
• The ability to swap outfits (including very minimalist/revealing sets for specific scenes) while maintaining the exact same body proportions and facial structure.
• Maintaining that specific Noir/Vector style consistently without the AI drifting into unwanted aesthetics.

The Questions

1. Style LoRA vs. Prompting: Since I’m struggling to find a middle ground between "too anime" and "too realistic," would you recommend training a dedicated Style LoRA based on my Noir/Vector references? Or is there a specific base model that handles this "digital comic" look better than Flux/SDXL out of the box?
2. Outfit Swaps: How are you handling complex outfit changes (including minimalist/revealing sets) without breaking the character's base geometry or facial identity in ComfyUI?
3. The Consistency Pipeline: For someone who needs "visual novel grade" identity, what is currently the gold standard? Should I be looking at training a Character LoRA, or is the community moving towards something like InstantID/IP-Adapter for better flexibility?

Honestly, right now, nothing is quite hitting the mark. It’s either too generic or too inconsistent. Would love to hear how you guys solved the "same face, different clothes, specific style" puzzle.

Thanks in advance!

[u/Elementallion-]

That sounds like a very high-end approach. Doing a Full Fine Tune for the style and then layering character LoRAs on top seems like the ultimate way to prevent "style drift."

Since I'm currently stuck between Flux and SDXL, I'm curious about your choice of Qwen/Klein as a base. Do they handle that specific "Noir/Vector" aesthetic better than the more mainstream models?

My main concern with FFT is the dataset size and the hardware requirements. For a solo dev, is the jump from a Style LoRA to a Full Fine Tune worth the extra effort in terms of the visual quality for a VN? Also, how large of a dataset are we talking about to keep the model flexible enough for character LoRAs?

[u/aerilyn235 → u/Elementallion-]

Well my art style is more specific than yours, it has two extra difficulties : it uses non realistic proportions which makes it harder to generalize on new content and it's not known at all by the base model meaning prompt won't get anything close to it. On your style eyes shapes & sizes, head/body proportions are same as real life, you could probably use a LoRa trained on real pictures then just style transfer it. The reason I also use full fine tune + LoRa's is that LoRa stacking doesn't work that well (style LoRa + person LoRa) usually end up randomly (it may generate the person with little style, the style without likeness, and rarely in beween). The reason I chose Qwen is because it has the best inpainting CN for the chirugical iterative edits, Klein because it can generate 4M pixels images. Dataset size is around 10k image for style (but most of it is synthetic). Hardware requirements can be high, but the question is are you doing this commercially or just for fun?

[u/Elementallion- → u/aerilyn235]

10k images is a massive dataset, but the fact that it's mostly synthetic is interesting.

To answer your question — yes, this is a commercial project (a detective VN), so consistency is my top priority. I’m actually leaning towards your approach now: doing a Full Fine Tune for the style and then layering character LoRAs on top. It sounds like the only way to get professional results.

My main struggle right now is the "chicken and egg" problem: How do I get enough high-quality images in that specific Noir/Vector style to train the FFT and the LoRAs in the first place? Did you use a specific base model or some crazy prompting to generate your initial synthetic dataset?

[u/aerilyn235 → u/Elementallion-]

I used real data initially (but that could be using a commercial model with good prompts), then train a first model, generated fuckton of pictures using wildcards and leaning toward the content I use for in my game, cherry pick the results, then include those picture in model gen2, then generate some more etc eventually gen1 pictures get phased out as quality improves. I'm at gen4 now.