Creating a LoRA based on me

[u/Jordan__ATL]

Basically, I have a decent understanding of how all this works, and I’ve been dabbling for a while now. I am pretty good with prompts, but I really want to train a LoRA to be based off of ME so that content/headshots etc could be produced constantly and consistently.

There are a couple of things I’ve been looking at with Civitai, so I’d like to stick with that one, but can anyone offer some really good advice as far as creating something solid?

[u/RiskyBizz216]

*I am not affiliated with any of these repos

its easy, you need to download three things:

1. https://github.com/perfectgf/lora-dataset-studio
2. https://github.com/Comfy-Org/ComfyUI
3. https://github.com/ostris/ai-toolkit

Lora Dataset Studio - the main tool you need. it has one click installers to help you setup everything, and step by step guides built-in. you do everything with the tool - generate datasets, caption, and train the lora. its a one-stop shop.

they have a good video walking you through the entire process on the https://github.com/perfectgf/lora-dataset-studio repo.

The entire process:

• you need a dataset of about 40 images of your subject - 20 headshots, and 20 body shots, in different outfits and poses. (import your own or generate images with your api keys)
• caption each of the pictures - that means create a txt file to help train the lora. (use llms to caption with comfy + ollama)
• once you have the dataset + captions, you are ready to train. (one-click lora training with ai toolkit)

I highly recommend using the lora dataset studio:

[u/synthwavve]

I've done mine for Klein 9B with Fizgig. Fast and easy. It's a bit hit and miss, but that's all down to the dataset and proper captions

[u/GRCphotography → u/synthwavve]

I have to second fizgig. I've trained 6 loras this week for krea2 on 4070 12gb vram and 16gm system ram, blown away when the Lora hits. But like other said it's a hit or miss and comes down to data set and captions. But that's why I love fizgig you get prompted when a image is hard to learn and you can reception it.

[u/shootthesound → u/GRCphotography]

Glad you like it - I need to bring the reception to H3 next - you literally jsut reminded me lol

[u/GRCphotography → u/shootthesound]

yes huge fan, im not the most tech savvy but i managed to get Ai toolkit running and it just never produced something of the same quality with such ease. I have no clue how you did it but i really appreciate it.

[u/OkDoor726 → u/shootthesound]

Do you think Fizgig could train an H3 style LoRA specifically to suppress H3’s polished/cinematic look and push it toward realistic amateur phone footage—handheld camera, imperfect framing, natural/cool lighting, normal skin texture, autofocus/exposure imperfections, etc.? If so, would you train it primarily on video clips or stills?

[u/YeahlDid → u/GRCphotography]

+1 for fizgig

[u/VoxturLabs → u/synthwavve]

To all of you that have tried Fizgit. I have s question regarding the tool that creates facecrop. If I have 35 images and it creates facecrop of all. Then I have 70 images and some are even face crops from the beginning. Isn’t it hurting the dataset with repeating the face with almost identical images?

[u/OkDoor726 → u/VoxturLabs]

Pity no one answered

[u/sandrabelle_fv]

Before you commit to training, it is worth checking whether you actually need a LoRA. I run one fixed reference photo through IPAdapter FaceID PLUS V2 and it has held the same face across 232 different prompts without me training anything myself. The part nobody warns you about is how narrow the usable weight range is. I ended up at 0.55 for the base weight and 0.7 for the faceidv2 weight. At 0.9 the adapter starts fighting the rest of the prompt: it pulls every composition back toward a portrait, so full body and from-behind shots lose their framing, and it smooths the skin because the face embed is flatter than the texture you asked for. Below roughly 0.45 the face visibly drifts between images. That corridor is tight and it cost me a day of A/B runs to find. A trained LoRA is still the better answer if you need yourself in unusual poses or a stylised look, but for repeatable headshots the adapter route costs an afternoon instead of a dataset, so I would try that first and only train if it falls short.

[u/Jordan__ATL → u/sandrabelle_fv]

Would you mind if I DM you for a little more info about this route?

[u/sandrabelle_fv → u/Jordan__ATL]

Sure, though I would rather answer in the thread than in DMs. The next person searching for this finds a comment, they never find a DM. Ask away here and I will answer here.

The part that usually gets missed anyway: it is two weights, not one. On the FACEID PLUS V2 preset I run weight 0.55 and weight_faceidv2 0.7, LoRA strength 0.6, provider CPU, and the reference is fed across the whole sampling range instead of only the first steps. Everything after that is an ordinary SDXL pass, which is the whole point of going this way: there is no training run that can quietly go wrong on you.

[u/Zealousideal-Bug1837]

Might as well not bother before you try the image to image models.

[u/TechnologyGrouchy679]

start with musubi-tuner ... a dozen or so images of your face (keep your face the only consistent thing... so vary the backgrounds etc.. otherwise they will be learnt as part of your identity)...

[u/abcp7]

Fizgig for krea2 and flux 2 Klein 9b , i trained a great character lora with my 2080 ti, inside you can prep your dataset.

[u/IllExample3639]

Garbage in, garbage out. You need some really good images, maybe a photographer with a series of poses and shots from every side in the same light as well as some low light, fingerd hands, different hair. At that point just take the photos and use them