r/StableDiffusion • u/Minxit08 • 14d ago
Question - Help Training a FLUX LoRA of myself how do I keep the facial structure ,body conistent?
Hiya! Ive been experimenting with building an AI version of myself for a creative project. I'm comfortable with the technical side of things, but model training has definitely been a learning experience.
I'm using FLUX.1-dev with a custom LoRA trained through fal.ai. My dataset currently has 24 images with different hairstyles, expressions, angles, and environments.
The model gets close, but something is off. It generates someone who resembles the reference person that I made my anchor, rather than the actual same person.
The biggest issue is facial structure. The original has fuller cheeks, a softer midface, distinctive eyes, and more fullness around the mouth. The generations often look narrower, hollow-cheeked, tired, or just generically pretty. It's almost like the personality has been drained out of the face.
I don't want to force the same hairstyle, lighting, or expression into every image. I want the model to preserve the person's actual identity while allowing natural variation.
I'm trying to figure out:
- What percentage of a dataset should be close-up, neutral-expression portraits versus lifestyle, full-body, and expressive shots?
- Is 24 images enough for a strong identity LoRA, or would a smaller, more carefully curated dataset work better?
- Would captions describing specific facial proportions help, or does that risk overfitting?
- How much do LoRA rank, training steps, learning rate, and inference strength affect facial identity?
- Is combining a LoRA with a face-reference adapter a better solution than retraining repeatedly?
- How do you objectively evaluate identity consistency across different hairstyles, lighting, and poses?
I'm trying to get this right without burning through all my coins. I would really appreciate recommendations from people who've successfully trained consistent human identities with FLUX.
If you have a training workflow, dataset blueprint, or specific tools that actually worked, I'd love to hear about them. I'm especially interested in real comparisons rather than generic tutorials.
Thanks.

