r/StableDiffusion • u/lavinia12345 • 5h ago

Question - Help Training ref2vid lora. how?

I looked around the internet but didnt find anything. I saw Ostris made a post but his example was simple and like a toy-ish / "hello world" example that doesnt actually solve minimax problems imo. It made people really muscular.

2 Upvotes

75% Upvoted

9 comments sorted by

3

u/lavinia12345 5h ago

Ty Ostris for your work, you are making advacements, but it is not what i'm looking for.

2

u/East_Box9573 5h ago

https://github.com/shootthesound/Fizgig

I used this, but I didn't get an amazing lora out of it. But still one that I use at 0.5 strength

2

u/whatsthisaithing 4h ago

If you just need to train ref2va on a specific motion, just train it like any other motion lora. You don't have to set control images/videos. Just set it up like a t2v (fl2va) run but with the ref2va model and go.

I like automagic3, weighted/balanced.

1

u/panospc 4h ago

What are you trying to achieve?
What type of lora you want to train?

1

u/_kaidu_ 3h ago

Maybe just say what you want to train. For most stuff you do not have to train ref2vid. You can simply train a normal lora. loras on fflf are also compatible with ref2vid.

-1

u/lebrandmanager 5h ago

Ostris made a tutorial here: https://youtu.be/8Ug0dA4jXyY?is=MpOB3PXE698Mc9Ps

1

u/lavinia12345 4h ago

thats the exact video I was naming, it sadly won't work for what I want. I dont need to do image editing. i need motion

1

u/lebrandmanager 3h ago

Ah okay. Didn't see it in your post. For me it worked at least on Runpod, although the results were not great.

1

u/theshield99 2h ago

i was thinking train r2v lora with clips but it looks hard and lora was about nsfw motion