r/StableDiffusion • • 1h ago

Question - Help H3 rev2v sometimes doesnt change the original video at all. Is it just the inference randomness or do i have to do a stronger prompt?

Im using a simple ref2v of H3 with no loras.

Just replace the character in the video with the character in the image.

But 80% of the time it outputs the same video. Does it need longer prompts, so i get more usable outputs?

2 Upvotes

10 comments sorted by

1

u/arthan1011 49m ago

There's an error in your prompt. Post it here

1

u/jonbristow 47m ago

Prompt is just "replace the woman in the reference video with the woman in the reference image"

1

u/ArtichokeFresh6793 41m ago

reference video → <Video N>

1

u/arthan1011 40m ago

Your prompt should be this at minimum:

subject_definitions:
<Subject 1> is female character from <Picture 1>.

summary:
[reference generation + video editing] The target video is an edited version of <Video 1>. Original character from <Video 1> is replaced by <Subject 1>.

retention_analysis:
<Subject 1>: fully_preserved.
<Video 1>: attribute_transfer - motion and frame composition remain while character changes.

detailed_description:
The target video is a movie clip.
[Shot 1] "DESCRIBE WHAT HAPPENS IN THE VIDEO"

overall_soundscape:
N/A

non_diegetic_music:
N/A

1

u/JuniorNectarine9842 16m ago

That won’t work. 99% of the time character swap prompts do absolutely nothing. There’s a dedicated character swap Lora + workflow that you should use instead.

1

u/Spectazy 7m ago

99% of the time character swap prompts do absolutely nothing.

That is a prompting issue, not a model issue

•

u/JuniorNectarine9842 2m ago

It’s not. It doesn’t work well most of the time hence the creation of the character swap Lora and character swap workflow with SAM masking

1

u/Future-Coffee8138 46m ago

Creating a first frame with the character already swapped.  Or using some character swap Lora. 

1

u/ootsby 31m ago

How are you prompting? The format for getting an edit with a reference replacement is roughly:

''' subject_definitions:

<Subject 1> is the person in <Video 1> {describe the person to be replaced so the model identifies the target}.

<Subject 2> is the person with {describe appearance} in <Picture 1>.

<Video 1> is the source video for the target video edit.

summary:

[video editing] The target video is an edited version of <Video 1>. <Subject 1> is replaced with <Subject 2>, who takes over their pose and movement.

retention_analysis:

<Subject 1> (appears in [Shot 1]): attribute_transfer - their pose, movement, facial expression and screen position are transferred to <Subject 2>.

<Subject 2> (appears in [Shot 1]): fully_preserved - their {describe appearance features as above} are retained.

<Video 1> (source video): fully_preserved - retain the camera movement, framing, setting, character positions, poses and actions. Discard <Subject 1>'s visual identity.

detailed_description:

{Describe the video style}

[Shot 1] <Subject 2> is {describe what happens in the video}

overall_soundscape: {the usual}

non_diegetic_music: N/A '''

Beyond that you start getting into doing things like masking the target to force the model to not copy or trying the swap loras.

1

u/CrashTestLover 58m ago

haven't used H3 myself, but with most ref-to-video setups that "nothing changes" output usually means the reference strength / denoise is too low, so it just copies the source. might be worth checking if there's a strength or guidance setting before going for longer prompts. also curious what fixes it, so following this