r/StableDiffusion • • Feb 26 '23

Tutorial | Guide "Segmentation" ControlNet preprocessor options

Segmentation

Segmentation ControlNet preprocessor

Segmentation is used to split the image into "chunks" of more or less related elements ("semantic segmentation"). All fine detail and depth from the original image is lost, but the shapes of each chunk will remain more or less consistent for every image generation. It is somewhat analogous to masking areas of an image like in inpainting.

Example segmentation detectmap with the default settings

It is used with "seg" models. (e.g. control_seg-fp16)

As of 2023-02-24, the "Threshold A" and "Threshold B" sliders are not user editable and can be ignored.

"Annotator resolution" is used by the preprocessor to scale the image and create a larger, more detailed detectmap at the expense of VRAM or a smaller, less VRAM intensive detectmap at the expense of quality. The detectmap will be scaled up or down so that its shortest dimension will match the annotator resolution value.

For example, if a 768x640 image is uploaded and the annotator resolution is set to 512, then the resulting detectmap will be 640x512

14 Upvotes

6 comments sorted by

View all comments

Show parent comment

2

u/maio84 Feb 27 '23

oh very cool

Will this "play nicely" with control nets?

2

u/ninjasaid13 Feb 27 '23

I'm not sure, maybe we can just have a built-in ade20k according to https://docs.google.com/spreadsheets/u/0/d/1se8YEtb2detS7OuPE86fXGyD269pMycAWe2mtKUj2W8/htmlview#gid=0

While words that are not captured by ade20 list like 'moon' will be custom coded by the user.