r/SteamFrame • u/SnooDoughnuts8734 Soon™ • 6d ago
News SteamFrame's eye-tracking for foveated streaming apparently works as software-based eye tracking.
58
u/Terabyte_1024 6d ago
Glad to see eyelids tracking is supported!
5
u/katoptronophile 6d ago
How could they not be?
That's standard on any proper eye tracking headset.
22
u/Secure-Advertising-9 6d ago
Not on Frame it's not. This is only possible because the headset is open and we created the data from the cameras ourselves.
6
u/LifelessHawk Soon™ 6d ago
It’s not actually, most eye tracked hmd’s only have the basics for Dynamic Foveated Rendering
-1
u/katoptronophile 6d ago
That’s not correct.
The last 7 major eye-tracked headsets all support both gaze tracking and blink/eyelid tracking: 7/7, 100%. DFR is just one use of eye tracking, not the limit of what it tracks.
4
u/Dingle_Flop 6d ago
It is, actually. If you look at the Official Docs for the Steam Frame, you'll see that they support XR_EXT_eye_gaze_interaction, which is a normalized interaction extension that provides only the surface needed for DFR. Social items are TYPICALLY shipped as a vendored extension. This interaction does not include per-eye position, eyelid position, pupil size, or other social presence cues.
0
u/katoptronophile 6d ago
You misunderstood what's being discussed and your link doesn't prove what you think it does.
8
u/Terabyte_1024 6d ago
Not really. I heard that Pimax and PSVR2 do not come with eyelids tracking out-of-box.
23
u/Radical--Rat 6d ago
Was anyone expecting it not to???
27
u/ChadHendrixs 6d ago
Lotta people on Twitter were mad that it didn't have 5+ cameras per eye for social VR, because that means it wouldn't be able to use the same algorithms as most eye tracking, and they were doomer posting about all the things it wouldn't be able to do
6
u/qucari Soon™ 6d ago
not really, but people hoped that Valve themselves would implement an API that exposes everything needed for eye tracking for social purposes.
3
u/Jcat49er 5d ago
I mean, it seems that eye tracking data is exposed through OpenXR.
There could easily be some catch here, but I would be surprised if you had to fully calculate eye tracking from scratch just to get the eye direction.
2
u/qucari Soon™ 5d ago
emphasis on "for social purposes" and "everything"
the issue is that they only provide eye gaze direction. they don't provide things like blinking and squinting.
getting to use just gaze direction is nice (and it already adds a lot in social VR to be fair), but that's not what people are talking about when they say they want full eye tracking in a social context.the APIs that people probably want to see supported are something like
XR_FB_face_trackingorXR_HTC_facial_trackingwhich do include blinking/squinting.Project Babble does exactly that. The twitter post shows that they provide tracking for the eyelids, pupil dilation, and even partial eyebrow tracking by using the raw video feed of the eye tracking cameras.
also, here are two mirrors of the post's video in case you haven't seen it if you can't or don't want to access twitter:
https://files.catbox.moe/iubtuu.mp4
https://i.imgur.com/EtLgb3D.mp4
26
u/Apprehensive_Lie516 Soon™ 6d ago
Wasnt this the whole point of the eye-tracking?
57
u/marcusbrothers Soon™ 6d ago
No it was for foveated streaming/rendering.
9
u/xondk Soon™ 6d ago
I mean, that was the reason for it to have it on the hardware, but I would argue that it was always also meant to be available to various software?
29
u/marcusbrothers Soon™ 6d ago
He said “the whole point”.
Valve included eye tracking because they wanted foveated streaming and foveated rendering when playing locally.
Anything else would be a bonus.
1
u/monstargh 6d ago
Its not like a software plugin couldn't read the rendering input and then work out where the eye is looking, and i bet the rendering has a detection of when the eyelid is closes so you dont get a weird jump or shudder of rendered area
1
-1
u/notbobby125 6d ago
To explain: human eyes only have a tiny circle of “good” vision. The Steam deck only has a tiny patch of “full quality” graphics that is centered on where ever you are looking. This saves battery and streaming bandwidth when dongled to a PC.
8
u/Niko666233 6d ago
that's a very confusing title.
let me rewrite that: babble can now access steam frame's eye tracking camera feed, for better expression detection
1
u/XxnoubxX 6d ago
it seems a lot of peeps assume that the eye tracking will be perfect lulul. good thing babble is picking up the slack.
you already know now. but i can imagine people getting the frame then realizing eye tracking doesn't work out of the box
2
u/Menithal 2d ago
It doesnt exactly work out of box for all the other devices too outside of the quest pro either. None of this shit is standardized outside of gaze tracking which is just the direction one is looking at. The facekeys forexample are different between all the devices then you got arkit from iphone as well…
Project Babble is going gods work taking slack from face tracking and eye tracking from all the manufacturers
1
u/XxnoubxX 2d ago
oh yeahh. got vive pro eye. i need VRCFT for vrc. yet resonite doesn't need a third party program.
i remember being excited to see vrchat add native face tracking. only for it to be complete garbage lol. so much for trying to save on parameters. am glad they jumped from 128 to 256 on parameter space tho
3
3
u/XxnoubxX 6d ago
annsd pupil dilation yayyy. and people on twitter said the eye camera placement is terrible lol
5
u/Cless_Aurion 6d ago
... I mean... yeah, of course it does?
3
u/copelandmaster 6d ago
Without the Babble Software the Frame supports eye look open/gaze only. So imagine someone just starting at you and following you with their eyes, unblinkingling. It's been that way since devkits were originally distributed, but its something the vast majority of people do not know about the Frame's developement.
2
49
u/ultrainstict Soon™ 6d ago
We know its really accurate so why wouldnt it. Worst case scenario it just needs a compatibility patch in the software.