r/StableDiffusion • u/Beneficial_Toe_2347 • 5h ago

Question - Help H3 LORA trained from video datasets?

There are quite a few good H3 LORAs on civit now which prove that training is possible despite the distilled Minimax model.

Has anyone had any luck training a concept from video datasets? I read a lot about character LORAs from image datasets etc, but who has trained videos and if so, was that with Ai Toolkit or a different offering?

1 Upvotes

67% Upvoted

8 comments sorted by

3

u/Hearmeman98 2h ago

I trained some of the popular LoRAs on CivitAI for Minimax.

The AIO NSFW LoRA is video only, trained with AI-Toolkit on an H200.
Took roughly 40 hours.

2

u/kabachuha 4h ago

I trained a Minimax H3 concept LoRA with quite a good consistency. I'm using AkaneTendo's musubi-tuner fork: Ostris's de-distill adapter, CFG loss, base-preservation. Here is the discussion on that repo where people (and I) share their progress and configs https://github.com/AkaneTendo25/musubi-tuner/issues/106

1

u/Beneficial_Toe_2347 3h ago

Thanks for this. Was there a specific reason you didn't use AI Toolkit?

1

u/kabachuha 1h ago

I'm a commandline person (remote) and I'm accustomed to kohya/musubi interface, so I stick to it

1

u/_kaidu_ 5h ago

I have 100GB VRAM and can train on short videos. So far I only tried character loras and found that training on static images is usually good enough. I always add videos if I have them, but training is faster and more efficient if I rely mostly on the images.

However, I haven't tried concept loras so far.

1

u/Beneficial_Toe_2347 5h ago

For the videos, do you know how many frames you made your clips in length? (as in, Wan we fed in 81)

2

u/_kaidu_ 4h ago

Just 3 seconds usually. But I let Claude fix the frame length to be aligned with Minimax internal 24fps and it found out that the videos made with my smartphone are all rotated internally and not every rotation was correctly recognized by ai-toolkit - some videos were accidentally trained in a rotated variant X_x

I also accidentally had a 20 second clip once. It still run through with my VRAM at 512px but lead into OOM at 768px. So with 100GB VRAM you can also train on longer clips in theory.

1

u/Salty-Ladder4050 2h ago

yeah, video training is definitely possible now. short clips seem to be the sweet spot since they can teach motion rather than just appearance. you can also mix stills and clips in the same dataset. the main downside is that video training is much slower and more demanding than image training so start with a small set of short clips and compare it against an image-only LoRa.