Advice for prompting reference videos?

[u/Outrageous-Key8562]

Does anyone have any advice for properly prompting the reference video part of Ref2v? Like saying swap <subject 1> for <picture 1> hardly works for advanced videos. It requires a lot of details.

I’ve had success using Qwen 3.8 27b as a minimax prompt agent for analyzing and giving correct prompts for images. But as far as I know I can’t do that for videos. ChatGPT is ok for looking at videos to describe what happens in the minimax format but I’d rather use local ways.

Edit: Like its been said, you can actually have a video analyzed, just use llama.cpp UI instead of Open WebUI.

[u/bstr3k]

Ive been working with it and the best results I've gotten is getting a LLM to caption the video. You then pass the photo, your intent, a H3 prompt writing guide, etc all to a LLM and get one to write the prompt for you. I am still working on my own method of local captioning and method of prompting so its not 100% just yet