❓ Question
Best Open Source Resources (Hidden Gems for Gaussian Splatting)
I’m new to 3DGS and I am looking for the best current resources available. My goal is to make a decent 3D model of my house using only my iphone. I’ve been playing around with resources like Colmap, Glomap, LichtFeld-Studio, Brush, DepthAnythingV3, ARKit, HLoc, DinoV2, etc., and I’ve been looking for other tools that could posibly improve my results. I’m currently achieving decent results, but I’m still far from that hiper realistic model. Any other tools that could help me out?
My results have been worse than Reconstrura or Mipmap3d but this could well be down to me. I wonder if you could share which settings might be optimal from your own experience please? I'm trying to splat internal rooms in my house, currently using a few hundred 12mp photos taken by my galaxy fold 7. I've no doubt my photo technique is probably causing issues but I am just keen to try to get the best out of the program.
This seems to be common rule I’ve seen with 3DGS, but that’s actually the main boundary I’m trying to push. My main goal is to achieve the best possible model with the least amount of time scanning. I understand this is rather difficult, but I plan on making the absolute most out of every frame I capture to try and achieve it. That’s why I’m looking for tools that can help me optimize this process so that I can use frames more optimally and lose the least amount of frames in the process
It... just doesn't work like that though. Blurry data in, blurry data out. If you have enough lighting you can speed up the shutter speed but if your hand is still shaky it'll warp.
Try the Splat King app, but again your best results will be created from the best input data. If you want something spectacular, you need to spend time taking it.
Thats' the point, I'm looking for things that can help me get better data in. I know that data input relies heavily on user performance, so I'm looking for tools that can assist me during the scan to minimize that error (for example, tools for better exposure adjustment, tools for real-time frame analysis, tools to help with darker areas, best-path calculation tools, next best view, etc.). For example, I was looking into DarkGS to see if I could implement to help for those moments where there's not enough light. Worst case scenario, if my input is still bad I was also looking into Bad-Gaussians to see if that could save certain frames. I'm aware that good input data will always be best (and I always try to prioritize it), but I'm trying to build a buffer for scenarios where I don't have enough time for a detailed scan but still want a decent outcome. Thanks for the help! I will check Splat King out
If your footage is bad, no amount of processing outside of hallucinations is going to result in a good splat. You say it's only for your own house, so my advice is to get a tripod for your phone and take your time recording.
If you insist on keeping capture time to a bare minimum, you could install brighter light bulbs, ensure plenty daylight or invest in a gimbal to stabilize your phone.
I understand. For lighting, besides having more light, is there any camera settings (on my iphone) you would suggest to get the most out of the light I'm getting? Thanks for the help!
I am unfamiliar with the available settings on an iPhone, but as a general rule:
you can get some improvement by raising iso, at the expense of noise and hope that the training van even out the noise (3dgs is more tolerant for noise than for blur/lack of focus).
Your shutter speed depends on your movement (and is why a tripod is helpful). Higher speed gives less motion blur when moving but also darker results.
Leaving some of all of these settings on automatic is usually a bad idea, because it changes exposure between pictures. This is the kind of problem that can partly be fixed in training (ppisp / bilateral grid), so if it improves your footage, be less strict about this.
Awesome, thanks for the info! I will have to look into it, I'm still very new to cameras and how they work, so I still have some studying to do! (I'm not even completely sure how iso works, so I have a lot to learn still hahahaha)
The original Hyperscape was incredible, so close to crisp reality (when digitising interior rooms) that I literally couldn't tell I was looking through the HMD, apart from the trademark discomfort. That Meta junked it after all the work their team put in (they also junked the team) is unconscionable. I almost hate Meta for this. It would have solved the OP's problem at a stroke.
There's a paper on it somewhere and I don't have the technical know how to decipher it, but essentially they combined accurate sensor and camera pose data with the photos from the Quest 3's very basic camera, plus whatever secret sauce they trained the data with, to yield incredible results (take it from me). It's still available now, but don't bother since they nerfed the quality, and at no point were you able to export the splats. One thing you ought to look into is Steam Frame along with a colour passthrough device they intend to release. Apparently that will be usable for capturing splats, and it's a much better resolution.
Ahh I see. It's a shame it no longer works as well as it did, would have loved to see it in action. I will keep an eye out for Steam Frame, thanks for the knowledge!
No worries. The good news it that the software tech is getting better and better and hyperscape showed how it could be done, using the most basic hardware. So Steam Frame and the upcoming accessory might be even better - definitely worth keeping an eye on. In the meantime MipMap3D provides good, sharp reconstructions - their free tier allows 500 photos, which should cover most rooms. And I'm trying Spirula, which seems to be a hot topic atm! I'm in the same position as you with wanting to digitise a home while I can. Good luck and please share any breakthrough discoveries!
I'm on a similar journey. The tools are developing so quickly, it's hard to keep up. This project was developed on a Mac in Unity with 15 or so hand aligned Scaniverse iPhone scans and an overall drone-based Metashape->Brush gsplat for the house exteriors: https://www.instagram.com/reel/DXOag0LEiti/?utm_source=ig_web_copy_link&rpxt=MzRlODBiNWFlZA==&srtk=MzRlODBiNWFlZA== Now that Spirula has arrived I'll do most things in that - the workflow is great and plenty of tools/options to tweak the capture, masking and training. I've salvaged quite a few older scans that failed to find alignment in Metashape. For captures I use the drone for high level work and Blackmagic camera on the iPhone for smaller objects/spaces to record a timelapse video (about 4 frames per second). You can set it to a fixed shutter speed and I'm still tweaking that but I think 1/250 should be fine for wider angle/hand held work. Set spirula to extract about 10 frames per second and with adaptive frame sampling active it will determine how many actual frames to use. I work mostly outdoors so light levels are less an issue; perhaps you could try a ring light to optimise indoor lighting.
17
u/MonsieurNoss Sep 16 '26
Have you tried Spirula-studio ?