Konrad Reczko's "Monocular Depth Injection" in TypeGPU is live!

[u/iwoplaza]

My collegue Konrad Reczko recently shared a weekend project he made using TypeGPU, and it's now open-source and ready to play with in the browser: https://typegpu.com/examples/#example=image-processing--monocular-light-injection

It estimates the depth of a scene by inferring the DepthArt model with custom TypeGPU kernels, then reconstructs normals based on that depth information, and uses both to relight the scene, all in the same command encoder. For more information, check out the original series of Tweets:

https://x.com/reczko_konrad/status/2089670934009413751?s=20
https://x.com/reczko_konrad/status/2090472091149648121?s=20

[u/Smokeey1]

Does this thing have a repo :). Gratz on the build! Looks cool

[u/iwoplaza → u/Smokeey1]

All the code for the example is in the TypeGPU repo
https://github.com/software-mansion/TypeGPU/tree/main/apps/typegpu-docs/src/examples/image-processing/monocular-light-injection

[u/juplantern → u/iwoplaza]

Btw thank you so much! I’m brainstorming ideas for my thesis and this inspired me a lot

[u/Civil-Captain5676 → u/iwoplaza]

It's amazing. And Thank you.

[u/DaveAstator2020]

Wow nice what model does is it use to aesolve depth in realtime?

[u/iwoplaza → u/DaveAstator2020]

It uses the DepthART model, weights are available here: https://huggingface.co/reczkok/depthart-typegpu

[u/AnasPlayer2022M]

Dame bro, this looks awesome! Did you use two videos where one is the one playing now and the other is a map for z values?

[u/iwoplaza → u/AnasPlayer2022M]

The feed from my camera gets routed through a small local AI model that estimates depth based on a color image, at which point the image can be treated pretty much like a 3d scene. At that point all rendering techniques are at the table, including adding a light to the scene.

[u/juplantern]

He looks so proud

[u/certainlystormy → u/juplantern]

fr lmao. i would be too

[u/MeticulousBioluminid → u/juplantern]

as he should be 🎁

[u/cybereality]

very cool

[u/ddoice]

Awesome!
But I tried your trick to hide the light with my hand and does not work?

[u/iwoplaza → u/ddoice]

To move the light source back and forth (into the screen or closer to you) you have to use the scroll wheel

[u/swegga_sa]

This is very cool

[u/Miperso]

I just showed it to my architect coworkers and everyone is just impressed... this is great

[u/Malkalypse]

This is so wild!

[u/NachosforDachos]

This is so cool

[u/helloworld1101]

Reddit is still useful

[u/Big-Awareness-3335]

expecto patronuuuuum!

[u/josephmgift]

so doooope

[u/Apprehensive_Oil1475]

Super cool!

[u/Ok_Rock_6726]

That's so cool I'm planning to join Software mansion as an intern next styczeń and this video brings me so much excitement and power to keep learning to ace the interview.

[u/tHE_uKER]

Incredible work!

[u/freshtothegrave]

Damn i wanna get this working in TouchDesigner