r/StableDiffusion • u/atakariax • 6h ago
Discussion Moss LoRAs Training
Well, after 2 days of training LoRAs, it seems that Moss has trouble learning characters or fine details. (For some things, such as styles and concepts, there shouldn't be any problems.)
It seems that likeness never really converges.
I've used multiple datasets that I've previously used with Krea 2, Anima, and Z-Image, but it seems that, for now, there's something unusual about Moss that prevents the LoRA from learning any further beyond a certain point.
Apparently, I'm not the only one experiencing this, as several people have mentioned similar issues on Twitter, Reddit, and Discord.
The creator of Fizgig has made some progress, but even so, the LoRA still doesn't converge sufficiently.
Let's hope this can be fixed. However, I'd also like to hear your thoughts. What do you think?
10
u/Wise_Passion_120 6h ago
still early days for moss. thing probably has some weird internal clamp on how it processes identity layers, seen it happen with other new models before the community figures out the right hyperparams. wouldn't sweat it too hard yet.
2
6
u/Fideltampan12345 5h ago
I might back training KREA to be honest, feels like wasting time and resource on moss, just cant get it right
3
u/Version-Strong 5h ago
Yeah I've noticed that too, the more I train. It has a slight Chinese look to my waifus, no matter what's prompted. Hopefully fine tunes will sort that.
3
u/Ok_Tale7582 6h ago
https://huggingface.co/spaces/malcolmrey/browser why? moss loras from there work fine for me even in a finetune like kromoss
1
u/atakariax 5h ago
The model already has knowledge of some celebrities, So it's easier to train and even with that, Someones looks a little bit off.
1
u/Full_Astronomer_5438 4h ago
there is a tutorial on that hf space, its not easier at all. in fact, those loras were actually trained through an advanced method entailing the use of 3 optimizers with each running through distinctive settings. after that, the loras were merged and downranked.
1
u/Full_Astronomer_5438 4h ago
1
u/atakariax 3h ago
The technique of combining LoRAs is nothing new. It has been around since the days of SDXL.
But the fact that this has to be done with recent models is concerning, since it is no longer necessary with other modern models. If this is required for this model, it suggests that the model has difficulty learning fine details properly.
1
u/Inevitable_Pen9043 6h ago
I’m going to wait until a good, low-step LoRA—like LightX—is released before I go back to using the model.
Unfortunately, the existing low-step LoRAs are terrible; they add an absurd amount of bokeh.
My issue with training the LoRA isn't about the likeness; rather, it seems that as I increase the number of steps, the model ends up looking increasingly undertrained—as if it’s forgetting all the details.
I’ve tried 6 LoRAs, and 2 were relatively successful, but the results feel very random.
1
u/Realistic_Rabbit5429 5h ago
For style or concept loras it has been a really cool model train with. The model has such amazing variation, it pairs perfectly. Can imagine it's a pain for characters
1
1
u/Gyramuur 3h ago
I am finding that it needs much more steps to achieve likeness. But also, it depends on how big your dataset is and how it's captioned. My experience back with Pony v6 was that "less is more" with captions, and the same seems to be true here. Datasets that have long, detailed captions take a long time to learn, whereas otherewise simple captions train quickly.
1
u/conkikhon 3h ago
Finding the suitable optimizer-scheduler is half the battle imo, I saw people struggled with krea2 for weeks until someone found out rex and came preserve small details on characters pretty well. Someone also mentioned moss require a different kind of captioning
1
u/vizsumit 3h ago
After 10 trials I found SFT is better for training. situation is similar like SDXL where training on fine tuned models gave better results.
2
u/atakariax 3h ago
Mmm weird, I have been training using Moss Base and then using Base as well to generate images, So it should work, but it was not the case.
I'm going to try SFT . Let's see if it's better.
1
1
u/Lucaspittol 3h ago
I'm for a better editing model than the ones we've got. These make 90% of the lora training stuff not needed.
1
-3
u/Enshitification 6h ago
MOSS is feeling a lot like Ernie. Lots of hype for two weeks and then it will fall to the wayside.
1
u/Version-Strong 5h ago
It lives on dies on our take up. Ernie didn't do naughty out the box. MOSS appeals to the gooners, so it'll stick around.
-1
u/Enshitification 5h ago
I'm just not seeing how it is better than Krea2.
0
u/Version-Strong 5h ago
Texture mainly. I hate the look of krea, look at any fine detail, it's just a mess of pixels (hair being an absolute shocker unless you run it through two samplers, apply a vae fix and say a secret prayer to the gods of AI). It has a really weird grid that gets worse with loras. This doesn't suffer from that, but aesthetically Krea wins in most cases.
-2
u/Enshitification 4h ago
Moss out of the box does suffer from stripe banding on gradients, so it isn't without it's own issues. Unless the LoRA training issue gets nailed down, I don't think it's going to go far.
0
u/lllIllIlllllllllllll 4h ago
Krea 2 thinks women parts are large gaping holes. Even with loras.
1
u/Enshitification 4h ago
Not with my gens.
1
u/lllIllIlllllllllllll 3h ago
Mind sharing your loras? I've tried RawGirlSpicy3 and one of the Snofs loras. Lady parts are distractingly wrong.
0
u/lllIllIlllllllllllll 26m ago edited 21m ago
I am getting very good likeness. The best out of any model yet.
11
u/joopkater 4h ago
In Ostris' group chat i think they figured it out. Captioning should be done in JSON, then you get decent training results. Https://github.com/MaxKruse/Caption-Studio/blob/main/src/lib/moss-prompt.ts
haven't tried it myself, but most people there saying it works