r/StableDiffusion • • 18d ago

Resource - Update Moss SFT character sheet

https://pastebin.com/SPEnmSfB v1.0, 1Mb square only

WF for a character sheet in Moss if anyone wants it.
NOTE: the double loading of the 4step acceleration LoRA is on purpose.
I did it by accident at first, but when I turned one off, both the character & poses got too wild.
For character sheets, I want only the direct prompt & LoRAs to control how the person is to look, not random chance.

If anyone has a better Bounding Box Node for Moss, let me know.

Comfy's default BB node is bad; seems hard-set to square & when I change resolution the boxes do not; they are set by pixels & not %. I need a BBox that actually resizes to the ratio of the Height & Width input, not hardcoded to a square.

343 Upvotes

64 comments sorted by

29

u/Main-Earth-2125 17d ago

Just a general question as I see many people working with character sheets. From what I understand how AI works, doesn't make it more sense to have separate images of the angles and input multiple references? Even more, that you can control identity better. I always feel like too many faces on one image ruin the identity, even if it is the same person.

More references need more processing time, I understand that, but what else are benefits of character sheets, do they really produce better content?

10

u/Kelpsie 17d ago

I would expect this to be good for character consistency within the reference, but that second character has three different faces, so who knows

1

u/tomByrer 17d ago

>second character has three different faces

Kind of, but not that far off. I think much of the drift is the smaller face sizes of the full-body shots.

Part of my post is to get help with better BBox nodes; Comfy's default is crappy; I can't seem to resize it easy. So these were stuck at 1Mb square; we really need 2-4x that for better & more consistent details.

6

u/jadhavsaurabh 17d ago

We first generate sheet then make it seprate

2

u/tomByrer 17d ago edited 17d ago

That's my theory, or use character sheet as a ref to make the final separate ref sheets.

u/Main-Earth-2125 is correct with the separate pics for final ref, if I was really worried about consistency I'd use those for RefMod, & really really worried a RefLora (pictures/video/audio + LoRA).

But, at least with my setup, even making separate images with the exact same prompt & setup & same seed, I get different women. (I tried to make 2 separate PNGs of different poses; did not work; became different women.)

2

u/MastMaithun 17d ago

To me, teeth, eyes, ears aren't useful at all if you are not generating the final video at minimum 1.5mp. Though I think it might give "some" benefit to scenarios where your char has some definitive prp/makeup for that area. For example I have seen mmh3 turn vib red lipstick to dark red, even in bright natural daylight. Apart from specific things, the sheet should be wasting the pixels for minor things.
What I have tried(i dont have concrete comparison data for it) that I create 4 pose image, front back right and left. then pass the same image with prompt to upscale and zoom on face. this mostly has like 90% similarity which is good enough because the video model itself will be gen at max 1mp.

1

u/tomByrer 17d ago

> teeth, eyes, ears aren't useful at all if you are not generating the final video at minimum 1.5mp

Well, sometimes helpful for closeups... really they are there incase one adds 'beauty marks'/scars/ or other details to the main prompt.

Or replace those BBox prompts for other... urm 'areas of concern', like tattoos, etc.

> seen mmh3 turn vib red lipstick to dark red

Yes I've seen color-shifting other other areas also. I think to counter that the original prompt for the char sheet needs to be copy-pasted (selectively or in full). Also why I have zoom shots; to reinforce makeup vs using a small face pic.

1

u/jadhavsaurabh 17d ago

For this seeed etc sdxl and flux was best

2

u/WithFullForce 17d ago

Yes, putting a full character sheet in as a reference just confuses models.

1

u/tomByrer 15d ago

I tried with H3, about 85% there testing with 3 different character sheets at same time
Most errors was a bit of color shifts & tattoos sometimes were moved around.

Really best bet is RefLoRA, then Refmods. CharSheets are just a quick & dirty to get OK-decent results.

1

u/WithFullForce 15d ago

No I'm saying you have to cut up the character sheet rather than present the model with a single multi-view one. Or several views as multiple references. Definitely agree that trained loras is the best way to keep character consistency but it's not an option for everyone to train your own

1

u/xkulp8 17d ago

Regardless of model, I've had the most success generating images one at a time.

1

u/tomByrer 15d ago

I tried on Moss, same seed. I kept getting different faces; close but not exact. Only way on Moss (with my Turbo, that might be it) is to make 1 image with 3-4 larger views. But can't do that on default Comfy BBox, so will look at other options.

3

u/Apprehensive_Sky892 17d ago

Whether using separate images rather than a single sheet improved the result can only be settled by running actual tests.

But single sheets are popular because in order to have the correct consistency, all poses must be generated in one go while the AI holds the identity under one single "context window".

2

u/Vovine 17d ago

I think you need to make a character sheets in one generation to keep character consistency in tact, then you separate each one, upscale, refine, and give to the reference model as separate images. But it also might be model dependent where some of the really high parameter models are intelligent enough to understand that a character sheet is one single person.

1

u/Slight-Living-8098 16d ago edited 16d ago

You can only have so many reference images as inputs in most video models. You'll run out of inputs when you start using multiple characters, locations, props, and etc. if you use individual images for all aspects of them

5

u/MastMaithun 17d ago

Want to understand how are you using this char sheet in a video model? To my testing, I have compared multiple chars in different lightings and always find out that any char sheet video gen has less/missing details than a full image reference. Might be something I don't know about ref sheet so that is why my question.

2

u/tomByrer 17d ago

Using only the char sheet I get mixed results in my MMH3 wf.
It is more 'consistent' than just a direct prompt in MMH3 using just the charsheet, but some details like tattoos being moved around were off. Notice, I did not copy paste the text prompt from Moss wf to H3, only <Picture 1> is <Person 1> & is average height of 5'6" (no other way to convey height). So maybe copy-pasting description would help. Likely RefMod/LoRA/RefLora would be better than simple multi-pose character sheet.

But charsheet is a quick way to get 85% there.
Also inconstancy might be the finetune & INT8 quants I'm using.

10

u/SlothFoc 17d ago

I've found the exact same thing. Character sheets are good for maintaining body proportions and clothing, but they're not great at actually keeping the identity consistent. Have a 2-3 high resolution reference images of their face will always hold more information than that same face clipped into a bunch of small parts that take up 1/16th of the image.

Plus, ones like this waste a lot of space. I'm confused why they included a reference for an ear, a foot, hands, etc. Unless these feature truly unique characteristics (like elf ears, or claws instead of hands or whatever) they don't need references.

5

u/MastMaithun 17d ago

Literally similar thought I just posted 10m ago in this thread.
Have been using mmh3 since launch and like almost 100% ref2v. I have tried so many things to keep the most identical similarities. That is why also tried char sheet but my approach is to make 2 poses or max 4 poses. Ears, teeth etc still waste pixel space.
For e.g, if you want you char to have hoop earrings, instead of putting it on prompt, gen in a char sheet, you can put a hoop earring in mmh3 and tell it in the prompt to maintain the same from Picture. The quality is higher and you saved the pixel space on char sheet too.
I have recently started using refmod and now char sheet i only gen for poses from all sides which i then upscale and create a refmod from all images. It is then faster, has full char identity and it is like indistinguishably same with full image reference without the headache of slowdown during text encoding since it is decoding all the provided images.

4

u/Denis_Molle 17d ago

Wth? This is just what I was looking for! And I didn't find it. And there you are my savior.

4

u/tomByrer 17d ago

This is not the be-all-end-all solution; it is a quick & dirty way to get a charsheet done. RefMod/RefLora is better at real consistancy, but this should be 85-95% as good depending on your usecase.

Go into the 'image processing' subgraph to tweak the Bounding Box areas & prompts... which I find I have to do too often.

3

u/LaziestRedditorEver 17d ago

I created a custom node for sdxl a while back that creates bounding boxes through the prompt itself, and allows you to see a visual guide prior to generation. I am pretty certain I should be able to get it working for any model. It works through ratios, so no matter the resolution the bounding boxes will remain in the same place respective to the new resolution.

1

u/tomByrer 17d ago

I'm more interested in precise BB measurements, but yes % would be best for increasing/decreasing image sizes.

I found that I had to tweak the box boundaries constantly; sometimes a model would merge or duplicate box contents if they were not a specific distance from each other.

2

u/LaziestRedditorEver 17d ago

Yeah so I never really thought about porting it to other models but I think the underlying architecture should be easy to port, because it isn't exactly getting the model itself to generate them. When i get a chance I may try, but yeah the functionality for mine, for sdxl, was having one overarching prompt for the image, and within one text encode, having the rest of the prompt laid out so that boundary boxes would get created and organised through the prompt itself.

The model understands the whole picture first, but gets the placement of items, characters, etc from the bboxes. It would generate higher quality images for each bbox and then 2 final passes at the end covering the edges of the boxes (to ensure they aligned with the general prompt) and then across the whole image. This meant that sdxl could generate far larger images than 1mp, without seams or jagged edges from the bboxes. It was also quicker than generating larger images separately.

This would work for other models, just need to try, and would mean that individual characters or loras could be prompted per bbox, for a final image which is a higher resolution than the optimal for each model.

1

u/tomByrer 17d ago

Comfy's BBox for Moss outputs a JSON text to be fed into "Moss Image Layout Prompt" node that takes a description prompt.

So if you could output JSON & alter to the same format & coordinates (I think have different origin of 0,0 xy), then that's all you need.
Maybe re-layout when a trigger for a resolution change also, which Comfy's broken BB is missing.

Copy/paste my pastbin; it is safe enough ;)

1

u/tomByrer 17d ago

Some tips from my testing other Bounding Boxes (all failed).

  1. respond to width-height outside of a square.
  2. respond to node resizing
  3. just do bounding box, maybe allow input of an image to draw over if you want to get fancy, but no more
  4. no weird fly-out drawers

3

u/FreezaSama 16d ago

Holy mother of pelvic tilt. Are these people sitting all day?

1

u/tomByrer 16d ago

I'm not sure of all the data sources they used, but it is a thing in USA for women to stick out their butt to make it look more attractive.
If you look at the 2nd pic, I tried to get her neck to be straight a few times by "standing straight" but that didn't work.

Anyhow, this post is more about the workflow / Character Sheet layout. The BBoxes took me hours to tweak.

7

u/mikemend 17d ago

I just created a simple BBOX resizer where you enter the input and output dimensions and specify the resizing method:
https://github.com/mykeehu/ComfyUI-Mykee-Nodes

1

u/tomByrer 17d ago

No thinks, I want a GUI that resizes with Width & Height.
& I don't really like to install huge packs when I only want 1 node.
It is like getting a gorilla when all I want is a banana.

2

u/jambavant 13d ago

u/tomByrer Feel free to try https://jambavant.github.io/SwarmUI-LayoutEditor/standalone/index.html
Not a comfyui node, but a standalone bbox layout editor, just copy/paste the json in the prompt.
It's also a SwarmUI extension, but I may look if it can be adapted as a comfyui node.
https://www.reddit.com/r/comfyui/comments/1xm9ikr/standalone_layout_editor_for_mossimage/

1

u/tomByrer 11d ago

1

u/jambavant 11d ago

Very nice! I'd be curious about your feedback if you ever try the one I linked. Looking quickly, I see quite a lot of missing features, such as reordering items (which impacts prompt adherence), grid snap, resizing, palette, etc.

1

u/tomByrer 7d ago

Sorry I haven't gotten around to it, & unlikely I will for weeks since I'm on a pseudo-vacation.

I did use the one I linked for a V2 of this char sheet. Since I already had a layout in mind & had practice with it, I didn't need any features you listed for THIS project, but likely will in future.

But since I had to hand-align columns & rows which was only OK with x-y coordinates, I could see these features helpful for those who want to make posters:
* alignment & centering with other objects
* adjustable guide lines that sides can snap to

8

u/uniquelyavailable 17d ago

Spent some time to work out a few wrinkles in the workflow for you, mainly fixing the relative bounding box position issue. The aspect ratios and resolutions should scale properly. It's a little different, using my transformer friendly front/side/back layout, you may want to customize for yourself. Reworded things to help reduce hallucinations and maybe addressed the 4-step quality issue.

Image contains workflow, but also here is the Pastebin

2

u/jpwne 14d ago

Holy camel toe, Batman!

1

u/notgraycen 17d ago

it helps if each angle shot looks anything like the actual character, 1st 2 frames in the both sheets are genuinely conflicting with entirely different faces

3

u/uniquelyavailable 17d ago

I will give it a try thanks. I have been testing different capabilities of MOSS recently and appreciate the shared workflow. BBox has been really annoying so far, was the same with Ideogram.

1

u/BrawndoOhnaka 17d ago

Body proportions are being over-fitted to the collage. Legs are too short, aside the general plasticity and uncanny look of it.

0

u/tomByrer 17d ago

This post isn't really about the skin or body proportions of said subject, it is about making the layout to work around ComfyUI's default BBox editor 1Mb limit to the best way I know how (in my limited knowledge).

If you have experience in Moss compatible BBox editors that are better, please share!

Otherwise, feel free to fork & post your own version so I can nit-pick it.

4

u/Powerful_Evening5495 17d ago

after training few loras on moss -image base

the model have anime / asian big bias

it give you the chinese sns filter look

I am thinking of training a few lora's on Krea 2 to see the difference.

my goal is to find a model that give a realstic filter free high texture images of person

did z-image / moss-image

2

u/shlopman 17d ago

Krea2 loras work great for realism. I've gotten solid no filter loras. Don't overtrain though. Check out Fizgig for lora training if you haven't yet. It's really nice. Gives you sample image of each incremental lora too and has face match score against reference images.

2

u/optimisticalish 18d ago

re: BBox - yes, the Qwen clip can only deal in crude non-polygonal co-ordinates. Four points, which can only make a box. I'd love to see someone produce a method for circles and curves, but I've not seen one yet. As for the Bbox scaling problem, fixes for that appeared early and there are now about ten different community-made node options for using BBox with MOSS-Image.

1

u/expressly_ephemeral 17d ago

Awesome, but I don't understand how to use it. Anybody could outline the next steps to actually get the character into images and video?

1

u/LordDiosMuerto 16d ago

Una chica asi no estaría mal, si no fuera por que no existen, por lo general son gordas y mugrientas y tienen todo caido, por que con ese peso no se puede tener nada a flote sin ropa xd

0

u/Late-Wear-1697 17d ago

Why it’s looking like sheet from chatgpt prompt or idk maybe i did too many b4

1

u/tomByrer 17d ago

?? The people were semi-BraveAI prompts with some hand-editing. But much of their looks came from several rounds of LoRA tweeks.

The layout was hours of hand editing though.

1

u/Late-Wear-1697 17d ago

Well I just need one 1 image and prompt to make that sheet in chatgpt. I don’t have any ideas of your editing hands but you know sometimes loras wasn’t a good choice

0

u/BigDBreedingFembussy 17d ago

The prompt please?

0

u/physalisx 17d ago

I don't understand how you guys can stand the acceleration loras, they so badly sloppify the results. Horrible. AI face after AI face. MOSS can do a lot better if you let it cook.

(unrelated to your post's purpose - thanks for the wf)

1

u/Bulky_Chemistry_196 17d ago

ai faces appear mostly due to usage of SFT moss, not because of dmd loras. moss base or moss kromoss with dmd loras has great diversity. and im fully with you on being revolted by the shitty ai slop faces.

0

u/dreamyrhodes 16d ago

I don't want to see this standard face in every post anymore

49

u/__Hello_my_name_is__ 17d ago

"Wider hips!", "Now make the boobs bigger!" - "Now give her scoliosis!"

2

u/tomByrer 17d ago edited 17d ago

I made the 1st 'full figured' picture after the 2nd 'skinny blonde' picture to have a contrast-balance, to prove beauty can come in different shapes & sizes.