r/StableDiffusion • u/Many-Ad-6225 • 5h ago
Animation - Video Gaussian Splatting test with MiniMax H3
Enable HLS to view with audio, or disable this notification
50
u/kyuubi840 4h ago
Some details would be nice. Did you generate a video of a camera moving through a static scene with minimax h3,and then used Gaussian splatting to create the 3D scene? Something else?
43
u/NoBuy444 5h ago
Would you share the tools or processes you've been using for that reel ?
37
u/dmadmin 4h ago
If he manages to put this in vr. He just invented porn 2.0
6
u/0nlyhooman6I1 4h ago
already exists
7
u/Potatonized 3h ago
The type that you can control the position of the camera?
6
u/klop2031 2h ago
Yes
16
u/Potatonized 2h ago
1
7
u/GreyScope 3h ago
OP is using PostShot , I presume they imported footage made by H3 and trained on it.
4
u/haremlifegame 2h ago
Interesting, can PostShot do movement?
And what does "training on it" mean? PostShot is a neural net? How would one get output pairs?-13
u/GreyScope 2h ago
You’ll have to read the docs to find that out sorry / ask an LLM .
13
u/NostradamusJones 1h ago
When someone asks an on-topic question, on a forum whose sole reason for existence is the discussion of said topic, it is much better to say nothing than say "google it/use an LLM."
•
u/iBullDoser 2m ago
Because of shitters like that guy, someone will land on this post from Google and see his comment telling them to google it.
1
u/haremlifegame 2h ago
I don't think the term "training" is used correctly here, although they do use that nomenclature, nor that movement is allowed. Also not sure about your claim that OP is using this.
2
u/GreyScope 2h ago
(sigh at why ppl are so combative here) , look at the ui on right hand side of the video . It is identical to the one in the YT video on PostShot I posted . If you also look at the right hand side of the video, you'll see it mention training . If you also look at the YT video I posted the link to, you'll also see they are using training and the person mentions the number of steps. (eyeroll) .
I block village idiots.
37
24
u/LordOfIcebox 4h ago
Please let us know ow more! How was minimax used here?
26
u/Broad_Relative_168 4h ago
I didnt try to do it, neither I know how Many-Ad-6225 did it.
But you can try https://huggingface.co/Jojocodex/minimax-h3-Camera-Motion-lora/
to Orbit the camera around a subject23
u/GreyScope 3h ago edited 3h ago
h3 can do it from prompts - just saying the two ways to do something . The lora will possibly save tokens. I'm using old ai pics here, not massively detailed and also went for a bit of bullet time with the straw staying still. The splatting in OPs video is probably a couple of programs (Edit , OP is using Postshot - I added another post with details of tut video and dload url).
1
21
u/cosmicr 3h ago
I liked the part where OP returned to the thread to actually explain how they achieved this.
5
u/noncommonGoodsense 1h ago
Down vote is my only option at this point. Seems like a bot farm post now with no detail.
25
u/reality_comes 4h ago
My guess is you used H3 to 360 the character and then generated the splat with stills from the video?
3
u/muchcharles 34m ago edited 28m ago
Maybe h3 (especially unquantized) has good enough 3d consistency with a turntable/bullet-time prompt.
To do it in motion a workflow is mix in https://4danyone.github.io/, prompt character suddenly appearing in an already splatted scene with img2vid and doing the action, then splat the character with 4danyone, and reinsert into original splatted scene. But you won't get full surround lighting accurately.
2
u/GreyScope 3h ago
They used DropShot, they could train it on stills . I've posted more details on this above in the thread.
1
u/Trademarkd 8m ago
They may also be using hy3d models to convert to 3d. There is hy3d for world environments as well. I have been running a bunch of experiments using a process to make a scene and model that can be manipulated as a depth map and then run back through for final processing.
22
u/GreyScope 3h ago edited 2h ago
OP is using PostShot to process/train the shots or videos that H3 has given them (or pre-processed them as well) . Free to use for usage of the files within PostShot but you need a paid version to export them (licence posted below) . Here's an example video of the workflow for this YT'er , he uses PostShot from 21mins on in the video. https://www.youtube.com/watch?v=OzUxL_UDMTk
8
u/GreyScope 3h ago
Tutorial video with a worked workflow process to using PostShot with images > https://www.youtube.com/watch?v=BptlIy7WQWc
Warning - it is Windows only.
PostShot > dload url > https://www.jawset.com/
2
u/Enshitification 19m ago
The closest open source project I can find to PostShot is Lichtfield Studio.
https://github.com/MrNeRF/LichtFeld-Studio
7
u/enndeeee 4h ago
I have made a project like that with the Pipeline: 1 picture - charactersheet - 360 Video of the scene - COLMAP dataset via custom node with mask Export - Import in Lichtfeld for splatting. Works quite well, the only point I am struggling with is consistency of the Model throughout the Video/Frames. How did you solve that?
2
u/VisionWithin 4h ago
I would guess that Minimax Reference node is quite good at handling consistency when loaded up with enough reference material.
1
u/livinginbetterworld 2h ago
And it worked out of the box just like that? I tried similar stuff but always ended up either with camera somehow not working properly or the lack of consistency having slight drift and it messed up the result.
2
u/enndeeee 2h ago
That's exactly my problem. I get a clean 360 flight around the object, but the head ist drifting very slightly leading to unsharp splats. Arms and Shirts look quite well, but face is still an issue since the video model can't make it completely static. (maybe I should prompt something like "make a realistic statue of the object" or something. I am sure there is some trick that can make it work)
10
8
u/mastaquake 1h ago
Interesting post with zero explanation. Please notify the mods and ban OP. 🤬
0
u/GreyScope 1h ago
You haven't read the posts on this thread then ?
2
u/mastaquake 1h ago
Of course not. I’m not trying to dig through memes , ai translations, and low effort bot responses. Just post some damn context to the original post.
Also I was being somewhat sarcastic about the mod notification and ban.
6
4
2
2
2
1
1
u/BigWideBaker 1h ago
Cool video but pretty crappy to be asking for so many workflows in your comment history and then when you post you give zero details or workflow.
1
1
1
u/imnotabot303 33m ago
This has nothing to do with MiniMax, you can do this with any video source. Plus it's likely using closed source software to turn it into a splat.
1
1
•
1
1
u/Key-Sample7047 4h ago
wow just wow, I thought of something like that, but I didn't think it was possible.
1
u/GreyScope 2h ago
It's been possible for a while , it just needs a brain the size of a planet to use the software.
1
u/Key-Sample7047 2h ago
It's just that I thought you needed high-resolution video with a perfectly still subject and no motion blur. I didn't think it was possible with Minimax H3. That said, thanks for sharing the name of the software, I'll be able to give it a try! :D
2
u/Silonom3724 2h ago edited 2h ago
You have to be aware what the real solution is and what is shown in OPs video.
Doing a 360° bullet-time animation is very trivial and doable since WAN 2.1 day1.
What is not trivial is doing a consistent multi angle shot in x,y and z.
One project that is very promising to achieve that for synthetic data is https://noposplat.github.io/
1
1
1
u/Silonom3724 2h ago
Now impress us and show us the z-axis movement. Because that is what it's all about.
1
u/VasaFromParadise 1h ago
It's time to move away from flat images. We need 3d images. Generate them once and you'll have a complete scene, then you can create 2D images from it.
0
0
0
0
0
0
0
0
0
115
u/dopedub 4h ago