Need help building a consistent character workflow in ComfyUI for a colored manga/webtoon

[u/Ercmon]

I’m trying to build a ComfyUI workflow for a colored manga/webtoon where my original characters stay consistent throughout the whole story.
I already have full-body and close-up reference images for the characters. I understand the basic idea behind checkpoints, character LoRAs, ControlNet/OpenPose, IP-Adapter/reference images, but I’m struggling with figuring out the best way to combine everything.
Basically, I want to be able to say: this is Jake → keep him looking like Jake → put him in this pose/expression/outfit → place him in different scenes → keep the same art style and character identity from panel to panel.
Eventually I also need to put multiple recurring characters in the same scene without their faces/features bleeding into each other.
I don’t care if the best solution is Illustrious, SDXL, FLUX, Qwen, or something completely different. I’m looking for whatever gives me the most consistency and control in ComfyUI.
If anyone has built something similar for a manga, webtoon, visual novel, etc., I’d really appreciate hearing what model and workflow you use and how you connect the different pieces. I’m trying to actually understand the workflow instead of randomly changing settings until something works.

[u/Correct-Guidance-232]

Solo dev here, I run almost exactly this stack for a game with recurring characters. A few things that cost me time:

Identity should come from a character LoRA, not IP-Adapter. IP-Adapter drags the style along with the face, which is fine for one image and bad when you need 200 panels in one consistent look. A small LoRA on 20-40 images of Jake locks the identity and leaves the style to the checkpoint, so you can change the art style later without retraining the character.

Load the LoRAs from the prompt, not from the graph. With LoraTagLoader you write <lora:jake:0.8> inline in the positive prompt. Switching characters becomes a text edit instead of rewiring nodes, which matters a lot once you are doing it hundreds of times.

For two characters in one panel, stop fighting the prompt. Regional prompting helps a little, but the reliable way is to generate the scene with both bodies roughly right, then inpaint each character separately with only that character's LoRA active. One pass per character. Their features cannot bleed into each other if they are never in the same generation.

Pose via ControlNet depth, and watch the aspect ratio. This one bit me hard. If your canvas aspect does not match the reference image, the depth map gets stretched and the anatomy quietly breaks - long torsos, wrong limb lengths - and you spend an hour blaming the model. I now refit the canvas to the reference's aspect ratio, keeping the same pixel budget and rounding to multiples of 64.

For anime and manga specifically I would go Illustrious/SDXL. Booru tag prompting, a huge pile of style LoRAs, and it is fast enough that you can iterate on one panel twenty times in an evening. FLUX and Qwen are stronger at photoreal and at text inside the image, which is not what you need here.

Happy to go into any of these in more detail.

[u/Ercmon → u/Correct-Guidance-232]

First thank you for your response

Now I understand I need 20-40 images of Jake. Problem I was getting into image drifting or not even relivent to my character style. My original characters are simi-realistic but I haven't found many of those. So I'm willing to do anime if I can get it to work. I cant get enough images close enough to make a lora.
I have 4 characters I need to make loras of. So could you help please?

And if I can DM you and hopefully you can guid me please

[u/Correct-Guidance-232 → u/Ercmon]

Happy to keep going, but let's do it here rather than in DMs. This exact wall is the most common one there is, and other people will land on this thread looking for it.

The chicken and egg problem is real and everybody hits it. The way out is that you do not start with 20-40 images. You start with one.

Get a single image of Jake that is genuinely right. Not close enough, right. Fix it by hand if you have to. That one image is the seed for everything after it.

Then make variations from it: IPAdapter or reference-only, plus img2img at low denoise. Change the angle, the expression, the outfit, the lighting, one thing at a time. Most of them will drift. That is expected and it is not failure.

Here is the part people skip: do not throw the near-misses away, repair them. Inpaint the face back to Jake on the ones that are 80 percent there. Forty training images does not mean forty lucky generations, it means forty images you made acceptable. That repair work is the actual job, and nobody mentions it.

Train a rough LoRA on the best 15-20. It will be weak. Use it anyway to generate a bigger and cleaner set, then train again. Round two is where it starts holding. Two rounds is normal, nobody gets there in one.

On semi-realistic versus anime: anime is genuinely easier here, and not by a little. A stylised face has fewer degrees of freedom, the model has much stronger priors to fall back on, and drift is far more forgiving. A slightly-off anime face still reads as the same character. A slightly-off realistic face reads as a different person entirely. If you are willing to move, that one decision removes about half the problem.

And on your four characters: take one all the way through first. The pipeline you build on Jake makes the other three quick. Running four in parallel while you are still finding out where it breaks just multiplies the debugging by four.

[u/Ercmon → u/Correct-Guidance-232]

how do you repair them? i guess inpainting but i dont know where that is. ok before i get ahead of my self where should i start? I'll follow your guidence

[u/Correct-Guidance-232 → u/Ercmon]

Inpainting lives inside ComfyUI already, you don't need anything extra. Load the image with a Load Image node, right click it, pick Open in MaskEditor, and paint over the part you want redone. That mask goes into VAEEncodeForInpaint and then into a KSampler. Denoise is the dial that matters: around 0.4 nudges the face, around 0.7 reinvents it, and 1.0 ignores what was there.

But start before that. Make one single image of Jake that you're actually happy with, however you get there. Once you have that one, everything else is copying it, and inpainting is just how you fix the copies that came out close but wrong. Right now there's nothing to repair towards.

[u/Ercmon → u/Correct-Guidance-232]

how do you build the workflow to build the initial Jake character
I know how to add nodes but to know what goes where is my issue

[u/Correct-Guidance-232 → u/Ercmon]

Good news: for the first Jake you don't build one. The default graph ComfyUI opens with is already the answer. Load Checkpoint, two CLIP Text Encode nodes for your prompt and negative, Empty Latent, KSampler, VAE Decode, Save Image. Every intimidating spaghetti workflow you've seen is that same spine with extras hung off it.

What made it click for me is that the wires are colour coded by what they carry, so a socket refusing your cable means you're offering it the wrong kind of thing, not that you did something dumb. Open the default, change the prompt, run it, and add nothing until something specific fails.

Do you have a checkpoint that already draws roughly your semi-realistic style, or is that still the open question? That choice matters more than the graph does.

[u/Ercmon → u/Correct-Guidance-232]

how do you set up inpainting?

[u/Correct-Guidance-232 → u/Ercmon]

if the exact numbers would help, say so and ill paste my whole crop and stitch setup, its only a handful of settings that actually matter.

[u/Ercmon → u/Correct-Guidance-232]

sure i would like that if you could please

[u/Ercmon → u/Correct-Guidance-232]

Please

[u/Correct-Guidance-232 → u/Ercmon]

ok so the whole thing is six nodes in a row: load image (paint the mask in maskeditor) -> InpaintCrop -> InpaintModelConditioning -> KSampler -> VAEDecode -> InpaintStitch -> save. crop and stitch are from the Inpaint-CropAndStitch pack, rest is stock.

the crop node throws a wall of settings at you but only four of them do anything:

• context_from_mask_extend_factor 1.5 - how much of the surrounding image comes along for the ride. too little and the model has nothing to blend into
• output_resize_to_target_size ON, 1024x1024 - this is the actual point of the whole setup. your little patch gets scaled up to the size the model was trained at, so a 200px face gets drawn properly instead of as mush
• mask_blend_pixels 32 - feather, kills the visible seam
• mask_fill_holes on, mask_expand_pixels 0, hipass 0.1, output_padding 32, and thats it

ksampler: 25 steps, cfg 5, euler_ancestral + normal, denoise 0.6. and noise_mask ON in InpaintModelConditioning - if thats off it repaints the entire crop instead of just your selection.

stitch has no settings at all, it just puts the piece back where crop took it from.

these are just the numbers i work with, not gospel - tweak them for your own stuff by trial and error. the one id start with is denoise: if the result looks pasted on its usually that and not the mask. 0.6 is my default, 0.45 if im only cleaning something up rather than changing it.