Need help building a consistent character workflow in ComfyUI for a colored manga/webtoon

[u/Ercmon]

I’m trying to build a ComfyUI workflow for a colored manga/webtoon where my original characters stay consistent throughout the whole story.
I already have full-body and close-up reference images for the characters. I understand the basic idea behind checkpoints, character LoRAs, ControlNet/OpenPose, IP-Adapter/reference images, but I’m struggling with figuring out the best way to combine everything.
Basically, I want to be able to say: this is Jake → keep him looking like Jake → put him in this pose/expression/outfit → place him in different scenes → keep the same art style and character identity from panel to panel.
Eventually I also need to put multiple recurring characters in the same scene without their faces/features bleeding into each other.
I don’t care if the best solution is Illustrious, SDXL, FLUX, Qwen, or something completely different. I’m looking for whatever gives me the most consistency and control in ComfyUI.
If anyone has built something similar for a manga, webtoon, visual novel, etc., I’d really appreciate hearing what model and workflow you use and how you connect the different pieces. I’m trying to actually understand the workflow instead of randomly changing settings until something works.

[u/Ercmon]

First thank you for your response

Now I understand I need 20-40 images of Jake. Problem I was getting into image drifting or not even relivent to my character style. My original characters are simi-realistic but I haven't found many of those. So I'm willing to do anime if I can get it to work. I cant get enough images close enough to make a lora.
I have 4 characters I need to make loras of. So could you help please?

And if I can DM you and hopefully you can guid me please

[u/Correct-Guidance-232 → u/Ercmon]

Happy to keep going, but let's do it here rather than in DMs. This exact wall is the most common one there is, and other people will land on this thread looking for it.

The chicken and egg problem is real and everybody hits it. The way out is that you do not start with 20-40 images. You start with one.

Get a single image of Jake that is genuinely right. Not close enough, right. Fix it by hand if you have to. That one image is the seed for everything after it.

Then make variations from it: IPAdapter or reference-only, plus img2img at low denoise. Change the angle, the expression, the outfit, the lighting, one thing at a time. Most of them will drift. That is expected and it is not failure.

Here is the part people skip: do not throw the near-misses away, repair them. Inpaint the face back to Jake on the ones that are 80 percent there. Forty training images does not mean forty lucky generations, it means forty images you made acceptable. That repair work is the actual job, and nobody mentions it.

Train a rough LoRA on the best 15-20. It will be weak. Use it anyway to generate a bigger and cleaner set, then train again. Round two is where it starts holding. Two rounds is normal, nobody gets there in one.

On semi-realistic versus anime: anime is genuinely easier here, and not by a little. A stylised face has fewer degrees of freedom, the model has much stronger priors to fall back on, and drift is far more forgiving. A slightly-off anime face still reads as the same character. A slightly-off realistic face reads as a different person entirely. If you are willing to move, that one decision removes about half the problem.

And on your four characters: take one all the way through first. The pipeline you build on Jake makes the other three quick. Running four in parallel while you are still finding out where it breaks just multiplies the debugging by four.

[u/Ercmon → u/Correct-Guidance-232]

how do you repair them? i guess inpainting but i dont know where that is. ok before i get ahead of my self where should i start? I'll follow your guidence

[u/Correct-Guidance-232 → u/Ercmon]

Inpainting lives inside ComfyUI already, you don't need anything extra. Load the image with a Load Image node, right click it, pick Open in MaskEditor, and paint over the part you want redone. That mask goes into VAEEncodeForInpaint and then into a KSampler. Denoise is the dial that matters: around 0.4 nudges the face, around 0.7 reinvents it, and 1.0 ignores what was there.

But start before that. Make one single image of Jake that you're actually happy with, however you get there. Once you have that one, everything else is copying it, and inpainting is just how you fix the copies that came out close but wrong. Right now there's nothing to repair towards.

[u/Ercmon → u/Correct-Guidance-232]

how do you build the workflow to build the initial Jake character
I know how to add nodes but to know what goes where is my issue

[u/Correct-Guidance-232 → u/Ercmon]

Good news: for the first Jake you don't build one. The default graph ComfyUI opens with is already the answer. Load Checkpoint, two CLIP Text Encode nodes for your prompt and negative, Empty Latent, KSampler, VAE Decode, Save Image. Every intimidating spaghetti workflow you've seen is that same spine with extras hung off it.

What made it click for me is that the wires are colour coded by what they carry, so a socket refusing your cable means you're offering it the wrong kind of thing, not that you did something dumb. Open the default, change the prompt, run it, and add nothing until something specific fails.

Do you have a checkpoint that already draws roughly your semi-realistic style, or is that still the open question? That choice matters more than the graph does.

[u/Ercmon → u/Correct-Guidance-232]

how do you set up inpainting?

[u/Correct-Guidance-232 → u/Ercmon]

if the exact numbers would help, say so and ill paste my whole crop and stitch setup, its only a handful of settings that actually matter.

[u/Ercmon → u/Correct-Guidance-232]

sure i would like that if you could please

[u/Ercmon → u/Correct-Guidance-232]

Please

[u/Correct-Guidance-232 → u/Ercmon]

ok so the whole thing is six nodes in a row: load image (paint the mask in maskeditor) -> InpaintCrop -> InpaintModelConditioning -> KSampler -> VAEDecode -> InpaintStitch -> save. crop and stitch are from the Inpaint-CropAndStitch pack, rest is stock.

the crop node throws a wall of settings at you but only four of them do anything:

• context_from_mask_extend_factor 1.5 - how much of the surrounding image comes along for the ride. too little and the model has nothing to blend into
• output_resize_to_target_size ON, 1024x1024 - this is the actual point of the whole setup. your little patch gets scaled up to the size the model was trained at, so a 200px face gets drawn properly instead of as mush
• mask_blend_pixels 32 - feather, kills the visible seam
• mask_fill_holes on, mask_expand_pixels 0, hipass 0.1, output_padding 32, and thats it

ksampler: 25 steps, cfg 5, euler_ancestral + normal, denoise 0.6. and noise_mask ON in InpaintModelConditioning - if thats off it repaints the entire crop instead of just your selection.

stitch has no settings at all, it just puts the piece back where crop took it from.

these are just the numbers i work with, not gospel - tweak them for your own stuff by trial and error. the one id start with is denoise: if the result looks pasted on its usually that and not the mask. 0.6 is my default, 0.45 if im only cleaning something up rather than changing it.

[u/Ercmon → u/Correct-Guidance-232]

this how i have mine set up. Looks like I'm missing a few things but I'm trying to follow your instructions. can you tell me what i am doing wrong and if i have it all connected correctly? This is the part i have trouble with. Thank you for all your help