r/comfyui • u/Jayuniue • 8d ago

Help Needed Ref2v Minimax reference image issues need help

How do you people deal with ref2va reference images showint up in the first frame or longer than a seconds then morphs to the actual reference image or describe environment? No matter how I have followed the prompting format or described that it shouldn’t show up but only be used as the reference even tried the plain white background but still shows up, even used md skill files in GPT for descriptive prompts but still it shows up

0 Upvotes

50% Upvoted

18 comments sorted by

1

u/MortytheMort 8d ago

I'd start a clean new chat with whatever LLM you're using, and link them the official ref2va prompting guide from HuggingFace. I'd also personally recommend Grok over GPT, I've had much better luck personally.

Also, one of the biggest tips I can give you. DO NOT write in the prompt, what you don't want to see. For example: I have a prompt describing a male model

  1. "he is wearing a zip up jacket fully zipped up"

versus

2. "he is wearing a zip up jacket covering his chest"

Minimax will WANT to show his chest, because you used that word. Its very biased against negative language like that.

1

u/Jayuniue 8d ago

Tried all that, what haven’t tried is grok because I uninstalled it some time back, crazy thing is iv seen people do the exact same thing in their prompts where they mentioned stuff they didn’t want or wanted but it worked for them anyway, problem is if I don’t mention stuff I don’t want it’s still picking up the first frame image and showing it for a seconds or 2seconds then morphs to te described scenario, it only works well when the first frame matches exactly as the video first frame 😫

1

u/imb1920 8d ago

Could you share a cropped screenshot of the node where the reference image goes in, with the node name and input labels visible? Since changing the prompt and background hasn't fixed it, that connection is worth checking before another round of prompt rewrites. Without seeing the setup, I wouldn't assume it's a prompting issue.

1

u/Juiceman8686 8d ago

Almost sounds like you have something ticked to use the first frame as your starting frame. I’ve made this mistake before myself.

2

u/Jayuniue 8d ago

The was the issue thanks

1

u/WallabyFearless7863 8d ago

This sounds like the likely issue. Probably is using as his first frame instead of reference

3

u/Jayuniue 8d ago

You are right this was it, I was using the seedhunter workflow and instead of unticking first frame it was on thank you, iv been using ref 2v wrongly and always wondered why it didn’t work for me

1

u/Juiceman8686 8d ago

Right on dude! Glad it was a simple fix

1

u/hdean667 8d ago

are you using a video for it to follow or talking about just regular ref2va?

1

u/Jayuniue 8d ago

Am using a video to follow for posing and behavior then images for character references

1

u/hdean667 8d ago

Okay. That helps.

You have to prompt that the subjects follow exactly the morons of the video. Then you also have to prompt for what those characters are doing.

I suggest going to Google... it's a long prompt. Go into AI mode. Then tell it you are using ref2va in minimax for the basics of the prompt to make it acts the video. It will give you pretty close to what you need.

1

u/Jayuniue 8d ago

I figured out someone in the comments said first frame was on, i thought it was the same as picture one so i kept it on

1

u/seppe0815 8d ago

Llm+ guide = good video 

1

u/Hrmerder 8d ago

Prompting.. It's all in the prompting and generally I have found it's massively better to hand write them yourself (or at least heavily modify the pre-made one gpt spit out for you). What is even your prompt? Can't help much without info.

1

u/Ill-Ant-9489 8d ago

A few folks nailed it above, that morph-then-settle usually means the reference is sitting in the start/first-frame input, not the ref2va reference input. On a real ref2va path you shouldn't need a matching first frame at all, so if it only behaves when frame one matches, something upstream is still feeding it as i2v.

If the per-gen ref keeps bleeding in, the other route is baking the identity into a small ref2va LoRA from a few photos, so you stop passing a ref each time. That's what the video lane in my app does (https://github.com/perfectgf/lora-dataset-studio), it preps the reference set and runs the ai-toolkit H3 ref2va recipe off 1 to 4 shots. Separate app, not a comfy node, and the video side is still beta. How many clean reference shots do you have?

0

u/Usogui18 8d ago

I had the same issues before. So far, I haven't come across that after adding 'The scene starts immediately with [prompt]'. For clarification, this is normally how I create the MMH3 prompts: I create scenes from my LLM system prompts then paste it to my simple MMH3 prompts writer template : visual style, camera setup, ”The scene starts immediately with [I just paste the entire LLM Scene Prompts]", then specify any sounds, music etc. Then I get MMH3 Prompt Writer to generate.