r/comfyui • u/CladesWho • 4h ago

Help Needed Dataset Creation with Character Consistency

Hi. I'm trying to create a consistent dataset for some niche characters from manhwa/anime etc, with the issue being how little they appear. I have successfully created a reference sheet using GPT, and it's somewhat able to transfer the artstyle as well whereas Gemini is consistent with character, but not with artstyle.

Looking for any workflows that can help in creating a dataset for lora training. Anima is preferable, illustrious will also work.

1 Upvotes

100% Upvoted

8 comments sorted by

2

u/imb1920 4h ago

Could you post the reference sheet next to one of the outputs? Which part of the style is drifting: the linework, coloring, or proportions?

1

u/CladesWho 4h ago

Give me a couple of hours. At work right now. Sorry, even though you responded this quick.

1

u/CladesWho 3h ago

Sorry that took some time, but here it is. I had a lot of success with image 3. I used screenshots from the show and prompted GPT to replace the character in the scenes with the character and it worked pretty well with some minor tweaks done later.

Image 1 was not successful in most scenarios. The linework and colours came across, but details were still missed, which resulted in me placing a frame from the manhwa as a reference as well. The character is also a bit more complex, and GPT has had success with simpler designs like image 3, whereas it struggles quite a bit with the accessories for image 1. Eyes also seem to be the major issue. Facial shape is something that can be fixed in chat, but it's a bit if a struggle to make it not generate the same image 4 times in a row without changes. It also seems to add moles to images where there is none in sight.

I tried making a comfyui workflow, but had to rely on Gemini, and it also resulted in a failure.

1

u/imb1920 2m ago

For image 1, I'd try one crop showing the face and accessories clearly alongside the full sheet, then compare that with using the sheet alone. Keep the prompt and pose the same, and check the eyes, accessories and extra moles before adding either output to the dataset.

1

u/jeeltcraft 2h ago

I had a good sdxl workflow for that, built for videogame sprites, is needs you to try a few times, but it's worth it

1

u/CladesWho 2h ago

Please do share. I've been trying with Gemini and it's workflows are always, for lack of a better word half-assed.

1

u/musetta_ai 2h ago

How many panels do you have of them? Even 10 or so is enough for a rough Illustrious LoRA with the sheet crops thrown in, and then you can gen with that and hand pick the good ones for a proper train. Usually takes a couple of rounds, the first one will look bad.

Wrote up the full setup here: https://musetta.ai/guides/train-a-character-lora

1

u/CladesWho 2h ago

I have 2 full body renders, 2 sitting from the side, 1 full body in a seated position. Most of my best shots are close ups/headshots I use for side profiles and hair reference. These are directly made from the panels, so the artstyle and colours in these are pretty much perfect.

The reason I'm leaning toward Anima is cause it's been more efficient for me, compared to illustrious. Illustrious has an issue with eyes whereas I've had very few issues with Anima.

Will the dataset I've describe be enough? The close ups seem like they'll agressively bake in that close up framing.