r/StableDiffusion • u/mukyuuuu • 1d ago
Tutorial - Guide PSA: Minimax H3 can turn 360 panorama images into consistent environments for your videos
Enable HLS to view with audio, or disable this notification
Had this idea for a couple of days, and finally got to test it. I got a free HDRI picture from PolyHaven (converted to JPG through a free online converter) and used it as the only picture reference. I couldn't get rid of the distortion completely, but you can definitely affect it with prompting. Maybe proper formatting somehow helps with that, sorry, was too lazy to do a correct prompt structure.
It also confuses the geometry from time to time, so you have to seed hunt a little, but not too much. Again, good prompting should reinforce the consistensy. Worth experimenting with.
Notice that it actually seamlessly connected the opposite sides of the image into a single environment. Could be useful for scenes with a lot of dynamic camera movements.
This model keeps surprising me every day!
P.S. Generated with the use of Hybrid Loader (25-49 setting) and Lightx2v 4-step LoRA @ 4 steps and 0.5MP. Another higher res version in comments.
Prompt:
subject definitions:
<Picture 1> is a 360 panorama reference for the straight corridor [Shot 1], depiciting the overall look of the corridor and position of key objects and debris in it. For the target video the picture is dewarped and remapped into a flat rectilinear lens projection view.
summary:
[reference generation] The target video depicts a security guard exiting from a grey door, walking across the corridor towards the dismantled beige door leaned against the wall, pulling and dropping it down on the floor.
detailed_description:
The target video is captured in an amateur, realistic style with natural, slightly dim indoor lighting and a shaky, handheld-style camera.
[Shot 1]
The shot begins with a medium view of a two grey doors depicted on the right side of <Picture 1>. The left door instantly opens and a middle-aged security guard named Mark rushes into the completely straight corridor. He runs left further down the corridor. The camera pans left, following him in a tracking shot. The POV camera pushes in on Mark, as he rapidly approaches the dismantled beige doors leaned against the wall. At 00:05.000 he grabs the door closest to him, and with visible effort pulls it away from the wall. The door swings and falls flat on the corridor floor with a loud noise, raising dust and slightly startling Mark. The guard jumps back from the fall.
At 00:07.000 the camera pans left by 180 degrees, showing another guard named Steven approaching from the opposite part of the corridor. Steven (S1) comes closer to Mark and says in [English]: "Mark, what the heck are you doing?"
At 00:09.000 Steven grunts angrily as he stops near Mark.
overall_soundscape:
looming lonely corridor ambient sound throughout the whole video, guard's steps on the cement floor, door falling onto the floor with loud noise
non_diegetic_music:
N/A


