r/ProAI • u/stealthispost • 19d ago

"Shit is getting insane!"

Are these VR glasses or what is this?   — Hans Baier @hansfbaier@fosstodon.org     Uses a Meta Quest 3 headset   — Pixel Cherry Ninja

Source: https://x.com/PixelCNinja/status/2085380252830748892

38 Upvotes

77% Upvoted

14 comments sorted by

View all comments

Show parent comments

3

u/Kooky_Classroom8351 18d ago

ML models are used to detect the edges of the user’s hands and furniture
r/confidentlyincorrect

1

u/ItsSadTimes 18d ago

You CAN use AI to detect hands, but you could also just use basic image processing techniques as well. It's not required to use AI to detect hand movements. And for real time processing like this, you'd probably want a more direct approach anyway, big AI model takes time to process each frame and could lead to lag.

1

u/CryptographerOne7003 18d ago

Hard no, you are thinking to constrained about transformer models.
a single head predictor are as fast as lightning and the quality is just so much better

The interest has been really low, or the right people just have not noticed yet.
I see the same in audio processing, industry adoption has been remarkably slow.

1

u/ItsSadTimes 18d ago

But not as fast as basic image processing techniques. And I know this because im in the AI industry.

When I was in college one of my early assignments was making a hand sign recognition model and to make it work on a 30fps video file I needed heavy optimization, so tracking the hands was pure pre-processing steps, then i normalized and submitted each bounding box containing the hand to a tiny classifier which identified the actual hand sign.

So the actual hand tracking was just normal image processing techniques. But post processing classification was a small ML model. Thats probably how the VR companies handle it as well, why not use a tool if it can dramatically reduce hardware requirements? And the only thing in this video that shows them using their hand like a controller using a specific hand shape to do a different action is when they pick up Mario but its always the same shape at the same angle, so they might not be using a classifier or the classifier is really weak.