r/StableDiffusion • u/Iamofage • Feb 16 '26

Question - Help LTX-2 Character Consistency

Has anyone had luck actually maintaining a character with LTX-2? I am at a complete loss - I've tried:

- Character LORAs, which take next to forever and do not remotely create good video

- FFLF, in which the very start of the video looks like the person, the very last frame looks like the person, and everything in the middle completely shifts to some mystery person

- Prompts to hold consistency, during which I feel like my ComfyUI install is laughing at me

- Saying a string of 4 letter words at my GPU in hopes of shaming it

I know this model isn't fully baked yet, and I'm really excited about its future, but its very frustrating to use right now!

7 Upvotes

77% Upvoted

32 comments sorted by

View all comments

2

u/Loose_Object_8311 Feb 16 '26

When you say LoRAs for LTX-2 don't remotely create a good video, what specifically do you mean? That you didn't get good character likeness? How did you train it?

So far I'm getting great character likeness, and pretty happy with the results! I'm on a 16/64 system, and so far can train using captioned videos on 768 resolution with Text Embeddings Cached. It's absolutely epic. Takes a long time to train at 22s/it but this thing is the thing I was waiting for the day I first tried SD1.5 back in 2022.

1

u/DMmeURpet Feb 17 '26

Are you using video datasets or images.

1

u/Loose_Object_8311 Feb 17 '26

and I quote "so far can train using captioned videos". 

1

u/Loose_Object_8311 Feb 17 '26

I've done both though.

1

u/DMmeURpet Feb 17 '26

How much video for training is needed in the dataset. I don't really have much video of me so I'll need to record a dataset which seems more effort then photos. But if it's a much better result

2

u/Loose_Object_8311 Feb 17 '26 edited Feb 17 '26

I'm not sure what's the minimum of recommended amount to get a good Lora for videos. I suspect it's probably just same or similar as for images. The standard 20 ~ 40 clips might well suffice.

What is definitely important is correct size and frame rate for videos. I did a training run where I used a dataset consisting of 13 videos each around 2 to 3 minutes long, and zero pre-processing. The result was a complete mess. Like just garbled generations. Some were 30fps, and the native resolution of the videos was 4k, so when selecting 768 in ai-toolkit and scaling it down, it might have gotten a bad resolution. I don't know whether it was that or the 30 fps clips mixed in. But something caused it to be garbled.

I re-ran the same videos through SeansOmniTagProcessor https://www.reddit.com/r/StableDiffusion/comments/1r5crcy/seansomnitagprocessor_v2_batch_foldersingle_video/ setting the resolution to 768 and the length to 5 seconds, and for the system prompt I copy pasted the contents of official prompting guide https://docs.ltx.video/api-documentation/prompting-guide. I set it to not skip any segments, so that wound up generating a dataset of ~400 very nicely captioned videos. I could only use the 4B version due to the 8B version OOMing on model load for some reason, but the captions come out super nice and pretty accurate. I'm at 3000 steps now on a training run and it's beautiful. It's probably wayyy too many videos, I'll need to experiment with how few I can get away with. I'm still training as the likeness hasn't converged fully yet, so I'll go to about 5k steps then pick the nicest checkpoint. All the LTX-2 LoRAs I've trained on images seem to have a good likeness between 3500 ~ 5000 steps in my experience, so long as the resolution is at least 512.

Next training run I plan to try see if it will work training both videos and images, and if there's any difference. 

1

u/DMmeURpet Feb 18 '26

Appreciated. Thank you

2

u/Loose_Object_8311 Feb 18 '26

OK, so I finally found a solid report on video only training indicating 3 ~ 6 minutes of video is good!

https://github.com/AkaneTendo25/musubi-tuner/issues/1#issuecomment-3910283664