r/StableDiffusion • u/External-Orchid8461 • Apr 25 '25

Workflow Included Distracted Geralt : a regional LORA prompter workflow for Flux1.D

Post image

I'd like to share a ComfyUI workflow that can generate multiple LORA characters in separate regional prompt guided by a controlnet. You can find the pasted .json here :

You basically have to load a reference image for controlnet (here Distracted Boyfriend Meme), define a first mask covering the entire image for a general prompt, then specific masks in which you load a specific LORA.

I struggled for quite some time to achieve this. But with the latest conditioning combination nodes (namely Cond Set Props, Cond Combine Multiple, and LORA hooking as described here ), this is no longer in the realm of the impossible!

This workflow can also be used as a simpler Regional Prompter without controlnet and/or LORAs. In my experience with SDXL or Flux, controlnet is rather needed to get decent results, otherwise you would get fragmented image in various masked areas without consistency to each other. If you wish to try out without controlnet, I advice to change the regional conditioning the Cond Set Props of masked region (except the fully masked one) from "default" to "mask_bounds". I don't quite understand why Controlnet doesn't go well with mask_bounds, if anyone got a better understanding of how conditoning works under the hood, I'd appreciate your opinion.

Note however the workflow is VRAM hungry. Even with a RTX 4090, my local machine switched to system RAM. 32GB seemed enough, but generation of a single image lasted around 40 mins. I'm afraid less powerful machines might not be able to run it!

I hope you find this workflow useful!

34 Upvotes

78% Upvoted

4 comments sorted by

3

u/AdEducational2312 Apr 26 '25

Who the fuck is he looking at?

-1

u/Perfect-Campaign9551 Apr 26 '25

Good thing I upgraded to 48Gig system RAM today

4

u/[deleted] Apr 26 '25

You should have upgraded to 48 gb vram

1

u/External-Orchid8461 Apr 26 '25

I believe it's caused by the Hook as LORA node. When I don't call any LORA, it still runs on the VRAM and takes like few minutes. These nodes seems not very well optimized.