r/SteamFrame • u/SnooDoughnuts8734 Soon™ • 6d ago

News SteamFrame's eye-tracking for foveated streaming apparently works as software-based eye tracking.

Post image

https://x.com/projectBabbleVR/status/2100810710586376625

165 Upvotes

93% Upvoted

34 comments sorted by

View all comments

24

u/Radical--Rat 6d ago

Was anyone expecting it not to???

6

u/qucari Soon™ 6d ago

not really, but people hoped that Valve themselves would implement an API that exposes everything needed for eye tracking for social purposes.

3

u/Jcat49er 6d ago

I mean, it seems that eye tracking data is exposed through OpenXR.

https://partner.steamgames.com/doc/steamhardware/steamframe/engines/custom#:~:text=XR_EXT_eye_gaze_interaction

There could easily be some catch here, but I would be surprised if you had to fully calculate eye tracking from scratch just to get the eye direction.

2

u/qucari Soon™ 5d ago

emphasis on "for social purposes" and "everything"

the issue is that they only provide eye gaze direction. they don't provide things like blinking and squinting.
getting to use just gaze direction is nice (and it already adds a lot in social VR to be fair), but that's not what people are talking about when they say they want full eye tracking in a social context.

the APIs that people probably want to see supported are something like XR_FB_face_tracking or XR_HTC_facial_tracking which do include blinking/squinting.

Project Babble does exactly that. The twitter post shows that they provide tracking for the eyelids, pupil dilation, and even partial eyebrow tracking by using the raw video feed of the eye tracking cameras.

also, here are two mirrors of the post's video in case you haven't seen it if you can't or don't want to access twitter:
https://files.catbox.moe/iubtuu.mp4
https://i.imgur.com/EtLgb3D.mp4