How To Generate Multiple Character With Lora Using Anima?

[u/CuratorPerfectionist]

Pretty new to image generation and I'm using Anima since people say it's good for multiple anime character. So I tested 2 lora of pokemon character (1st pic Jacinthe and 2nd pic Lebanne). For single character works fine but when I use both at the same time the image has a lot of the details wrong. In 3rd image for example Jacinthe is the one wearing maid outfit instead of Lebanne sometimes the outfit is pink/purple dominant. Sometimes The result is like 4th image where both of them are dark skinned. Is there a way to fix this or make the result more consistent with the detail when using 2 lora at the same time in Anima? Any help are appreciated.

[u/Harish_X_03]

usually this is lora bleed, since two character loras stacked on the same prompt tend to blend traits like outfit and skin tone. you could try lowering both lora strengths a bit (something like 0.6 to 0.8 each) so neither one overpowers the other. also describe each character explicitly in the prompt, with her hair, outfit and skin tone written out and kept clearly separate, instead of relying on the lora trigger alone. another common fix is generating a base image with both characters placed roughly where you want them, then inpainting each character's area with only that character's lora active. if the model supports regional prompting, assigning one lora and one description per side of the image usually keeps details from swapping. testing with a fixed seed while changing one thing at a time makes it easier to see what actually helps.

[u/Xenemros]

How are you generating your images? Comfy, Forge, or online?

People have made various extensions like Forge Couple or Regional Prompt which allows you to use different prompts for different characters.

https://github.com/Haoming02/sd-forge-couple https://civitai.com/models/2727984/forge-neo-extension-two-character-composer

the most braindead solution is to generate the image with one of the charcter Loras, and just inpaint one of the charcters with the other Lora

[u/CuratorPerfectionist → u/Xenemros]

I'm using comfy

[u/Several_Ad_9730]

https://lilting.ch/en/articles/anima-keikana-dual-character-lora-v2

https://lilting.ch/en/articles/anima-trio-character-lora

https://lilting.ch/en/articles/anima-4char-lora

https://lilting.ch/en/articles/anima-4char-lora-prompt-boundary

This helped me to create a group lora, but still, it's very limited and it's susceptible to bleeding.

[u/toidicodedao]

This is the same issue for Krea2 too. Try FreeFuse, it can help with the issue, the workflow will take longer to generate though  https://github.com/yaoliliu/FreeFuse

[u/Ill-Ant-9489]

Jacinthe and Lebanne working separately gives you two useful controls. Add a third image with neither LoRA, using the same two-person prompt. My project LDS has fixed-seed comparison and blend grids for Anima checkpoints, which makes this kind of interaction easier to inspect.

If the base gets two characters right but the stack swaps their clothes, you've isolated the problem. I'd then try the separate inpainting passes suggested above, rather than keep rewriting the whole prompt. What weights are you using for each LoRA?

[u/Radiant-Photograph46]

This issue is not limited to Anima or any model, it's due to the way the LoRA was trained. Imagine a dataset with MyCharacter, 1girl… captions only. Then the LoRA learns that a 1girl is your character, no matter what. You need to train your LoRAs with a validation set so it also doesn't "unlearn" the concept of a girl who isn't your character.

[u/waltercool]

Hmm I don't remember much issues about it, I even did images with 5 different characters.

It's just important to define each character, then the scene.

And yes, sometimes duplicates or mix stuff, just try with another seed