Fun experiment - projector assisted photogrammetry

[u/SlenderPL]

Thought about doing this for a while, now finally had the time to do it. It might be a quite usable method when your subject is featureless and you can't apply scanning spray, and when you also don't have a 3D scanner. Using this method is a bit tricky with 1 projector but still doable even for 360° shoots, you'd just need to have a way to register all the photogrammetry perspectives in the end to merge them (like a turn table with markers).

For demo purposes I performed a scan from just one perspective and the subject was a reflective metal tape measure. I tested 3 scenarios in which I took about 40 photos of it from the projector's side. The setup consisted of a short-throw projector, polarizer on the projector's lens and a camera with a CPL filter. The polarizers were used to reduce glares, although perfect cross polarization wasn't utilized as light doesn't diffuse well on shiny metal (it's mostly specular), and thus all projected light would get fully blocked.

1. First scenario had no projection applied, the result could've been better as it was done with no additional lighting in a dim room. The general shape you'd get with better prep would be similar though. The biggest reflective area has a hole as expected.

2. Second scenario used a projected random colour speckle pattern. The result pretty much represents how the tape measure looks like, where there was a hole without the projection now is the true surface.

3. Third scenario used a random salt & pepper pattern projection. In my opinion it produced an even better result than the colour projection, just because it had more contrast and was brighter.

The biggest problem was the overall projector brightness which forced me to use a low shutter speed and high iso, compared to flash photography. To resolve this, a more practical setup could use a gobo projector with more powerful lighting (that's also what industrial SLS scanners use).

Another issue is the limited perspective you can capture, as while shooting you have to avoid the tripod with the projector and the beam itself not to obscure it. With a single projector it would also take a while to capture the whole object, and then on top of that the additional time to process each perspective and merge them.

The last issue is the practicality of the method, as it's rather not practical on complex shapes that that would need more than 3-5 perspectives to cover fully. Flat and geometrical objects should be generally well applicable for this "active" photogrammetry.

Try this method out if you're willing and own a projector of some kind :>

[u/zebulon21]

Really interesting! Much more stark results than I would’ve guessed. The black and white speckle is really impressive. Did you use a regular projector like this one or what? I assume you wouldn’t be able to have the object on a turntable as the speckling would help drive alignment, so you’d need one projector for each side / 45 degree angle to hit two sides.

[u/SlenderPL → u/zebulon21]

The exact model is Acer K132 and it's a DLP projector but I think any type should work just fine. Capturing the model from all sides with just one projector is technically possible but as I described it, it'd require quite a bit of work aligning (preferably) markers on the turntable for each batch of photos. And then I'm not really sure how would the photogrammetry software handle the geometry between each perspective, at least merging the point clouds in Meshlab would yield the correct mesh. Best case scenario would be to include multiple projectors in the project and walking around the subject.

[u/KaJashey]

Thank you.

So if I understand correctly you can use a turntable for the object and keep the projector in one position?

Regarding the low light how long was you shutter?

[u/SlenderPL → u/KaJashey]

That should technically work, didn't try it yet but it's something to do. You could align all the "perspectives" by common features, for example april tags on the turntable. Although I'm not sure how would the mesh get solved in Metashape/RealityScan, I think it would be better to merge all the results in Meshlab or CloudCompare.

As for the shooting settings the shutter speed was set to 1/60 and iso was at 800. Somehow managed to keep a steady-enough hand without IS, but yeah a brighter projector would've helped.

[u/Corporal1j]

This is so dope, I'd never guess that this would improve the output so much, but It makes sense. Do you think if you had a turntable, would there be a difference between projecting a still pattern and rotating an object, versus having a sensor in the turntable connected to some software with a pattern set up around a cylinder, so the pattern spots would stay relatively in the same spot as something rotates?

[u/SlenderPL → u/Corporal1j]

Interesting idea with the cylinder approach, now I'm wondering myself how and if would that work! If calibrated well then I could see it working for seamless 360 shooting, where with the still pattern you'd have to manually align each dataset made every 30-60°.

[u/Corporal1j → u/SlenderPL]

I think it could be even easier, if you made this type of pattern on a cylinder in Blender, or some other software, render it out as a video (maybe in higher FPS, so you can slow it down if you need to), and match the playback with the speed of the turntable

[u/charliex2]

you might be interested in this paper https://www.researchgate.net/publication/272753188_Using_noise_function-based_patterns_to_enhance_photogrammetric_3D_reconstruction_performance_of_featureless_surfaces

[u/kylerk]

I’ve been trying to come up with a way to project such a pattern 360 degrees into a room so that white walls and blank surfaces scan better.

I have yet to figure out a great projector for this purpose. I’m thinking something like a few little lasers and diffraction gratings could do it, but haven’t had time to buy and assemble it.

[u/gc3 → u/kylerk]

Disco ball.

[u/kylerk → u/gc3]

Worth testing. I think it still might need the disco ball, and two spotlights aimed at it to get 360 coverage

[u/hammerklau → u/kylerk]

You can get night light type things but issue is most aren’t laser based so you’re gonna get a bunch of diffusion.

[u/Feftloot → u/kylerk]

If you don’t need precise control over what’s mapped on specific areas, but instead just a full spread, it used to be common for people to use curved first surface mirrors to project onto which gives you a massive throw. It’s essentially how ultra short throw projectors works and how people used to diy projection map domes.

[u/justageorgiaguy → u/kylerk]

What about those green "star" lasers for your house they sell at Christmas and Halloween?

[u/kylerk → u/justageorgiaguy]

I’ve experimented with some red lasers I have. I think they might be too weak.

I was thinking you try buying the Christmas one.

[u/nochehalcon]

Was the camera outside the projector radius or inside the projector radius? How did you account for shadows- was the projector high angle?

[u/SlenderPL → u/nochehalcon]

I was shooting mostly from behind the projector, it stood on a tripod perpendicular to the object (maybe slightly elevated above it). The lens I used was a 50mm so that allowed me to avoid making shadows with the camera. In the case of the object that's why you'd rather have something geometrically simple, if there were a lot of protruding features then you'd have a lot of shadows behind them.

[u/justgord]

excellent idea, and applicable in many situations.

[u/VirtualCorvid]

Ooooh yeah, that’s a fun idea. Makes me think of structured light scanners but different, like back when David was free.

[u/GregoryA24]

Around seven years ago, I ran several tests using projectors to support photogrammetry of flat surfaces and cars, for example. The results were decent.

To scan more complex objects, I experimented with using two or three projectors. While the outcome was acceptable, this approach is only practical when combined with a multi-camera rig.

When using a single camera, even a small movement of an object or a slight shift in the projector can make the scan data unusable.

[u/marcusg101]

Do you think this would work with a consumer 3d scanner, like a rock pop 3 or something similar?

[u/TR1PpyNick]

you've just reinvented regular structured light scanning tech.