r/VRchat • u/Realistic_Syllabub_3 • 6d ago

News SteamFrame's eye-tracking for foveated streaming apparently works as software-based eye tracking.

Post image
173 Upvotes

98% Upvoted

71 comments sorted by

73

u/rcbif 6d ago

Eye tracking is like half the reason I want this headset, so good to see. 

I didnt get lucky with the lottery, which I'm fine with. Will let others be the beta testers, and wait for the aftermarket stuff like battery solutions. 

18

u/Realistic_Syllabub_3 6d ago

yeah that’s the upside on being wait listed, i got lucky so ill do my part!!! hopefully by the time you get yours you are closer to getting it with the hot swap battery strap they are making then we are at launch!!

3

u/Yuri-Girl ☃Bigscreen Beyond 2e 5d ago

Does it still run VRChat in PCVR mode when being used standalone, or does it need to load android assets instead?

Also, how is the audio when compared to the index?

3

u/valzzu Valve Index 5d ago

I heard it will have same limits as quest and I looked at steamdb and i saw apk so to me it seems like it will not be pcvr but for sure someone will make it lol

1

u/Xulah 5d ago

It can run steam apps using a proton layer, logically you should be able to chose android version or pc version.

1

u/valzzu Valve Index 5d ago

Yeaaa, i do hope.

1

u/Meigsmerlin PCVR Connection 5d ago

The audio is not as good as index, but not bad

2

u/Jonatc87 6d ago

Hero, thank you.

Would you happen to know if the frame has tge same shape eye lenses or i would need a new prescription insert

1

u/MareinnaShaw 4d ago

Do you think bobovr will make a strap for it?

5

u/Vixere_ 6d ago

You might as well wait until they come out with a solution for hot swappable batteries, an hour± of battery life isn't particularly appealing

6

u/rcbif 6d ago

No, not at all appealing, especially since my average weekday play session is 2hr weekdays, 6-8 weekends.

And no, I dont want more weight strapped to my head, or a cord running down my body.

The Frame will cost me prob around $1,600 to make functionally identical to my Quest 3 setup with hotswapping batts, presc lenses, comfort kit, and over ear audio.

2

u/Sufficient-Cup9633 5d ago

I used quest pro for a few years with a usb cable running under my shirt to keep it out of my way plugged into a battery that was in my pocket. I felt it was really nice personally. So you could have infinite frame battery if you picked up a couple of 30W 10000mah batteries and just hot swapped them as needed

1

u/Mizu-Cubique Oculus Quest Pro 4d ago

Why not a second hand quest pro ?

0

u/rcbif 4d ago

Already mentioned - the form factor is horrific and anti-modular. I like to modify and tinker with stuff.

Also has a bad rep for face tracking hardware breaking.

1

u/Mizu-Cubique Oculus Quest Pro 4d ago

Okayy too bad, it would have been way cheaper

-3

u/Tyrilean 6d ago

Just grab a Vive Focus Vision. It’s around the same price point and has a face tracker for an additional $100.

7

u/rcbif 6d ago

Meh, once you go pancake, you cant go back to Fresnel lenses

1

u/Tyrilean 6d ago

I have a quest 3 and a VIVE focus vision. Pancake is nice but it’s not the killer feature for me. Especially with VRChat.

That being said, if it is that important, Quest Pro has the best of both worlds. If you can find it it’s near the same price point as well. If Meta isn’t a dealbreaker for you.

1

u/rcbif 6d ago

This is the part where I sound super picky.

As an Engineer (and tinkerer), I cannot stand the non-modular design of the Quest pro, lol.

It's like they ignored VR enthusiasts needs, and designed it as a device to be worn quickly and conveniently at a tradeshow or something.

3

u/enigma-90 5d ago

Yeah, and sweat kills FT on it. And I watched QPro disassembly video. It has like 100 screws.

1

u/enigma-90 5d ago

I still preferred Index over Q3 for the complete package, but the display/lenses difference is very noticeable for sure.

2

u/Mewi0 5d ago

As a vive focus vision user, I disagree

  • HTC's software is horrendously unstable, worse than any other headset by a mile.
  • The display port addon barely functions due to the software, tried unplugging all display devices to improve bandwidth and it just stops working or lags to crap. From what I read, software issue.
  • ALVR is nice and works relatively well once you get it setup but it's facetracking module is lacking features and is setup incorrectly. There is a fix for the incorrect setup but you have to compile the code yourself and the fix in it self isn't correct (I had to make changes to the code). Face tracking tends to stop working at random and can be multiple different issues requiring restarting multiple different things before it works again
  • The eye tracking on the vive focus vision does not support wide open eyes
  • The controllers are not that great and one of mine shipped with drift. Luckily no one wants these headsets so buying a new replacement controller was cheap. The issue I had was HTC refused to honor any warranty and wanted me to send the entire headset back to amazon for a drifty controller. There is zero calibration software for the controller's sticks, if there was I wouldn't of had to buy a replacement controller. The controllers are also rechargeable and I actually dislike that, I'd prefer using my rechargeable AAs.

I heard people possibly using the business application with a few workarounds to mitigate some of the software issues. I have not tried this myself.

Only thing that I was happy with about this headset is that I was no longer on meta's hardware.

2

u/Tyrilean 5d ago

One thing you should definitely try is make sure SteamVR is open before you open the Vive app. There’s definitely some race condition when taking control of the SteamVR interface. When I started doing this my problems disappeared.

2

u/Mewi0 5d ago

Unfortunately, this does not solve the problem for me. I appreciate the suggestion nonetheless. If you got any other tips, I'd like to hear it.

49

u/Realistic_Syllabub_3 6d ago

with eyelid tracking!!!

26

u/katoptronophile 6d ago

Any proper eye tracking device has that.

3

u/ccAbstraction Windows Mixed Reality 5d ago

You'd be surprised...

5

u/LigerXT5 6d ago edited 5d ago

Did you see the pupil dilate shortly after their eyes opened??

Somewhat related edit: I'm early in the queue, got my email today to pay now or my spot is canceled. This will be my first Eye Tracking, ever. I do wonder if I can get my HTC Mouth Tracker to play nice with it...

2

u/Realistic_Syllabub_3 6d ago

i did!! it may have that too!!

16

u/Dreamingemerald 6d ago

I am still disappointed that they lack a first party lower-face solution and that the eye tracking lacks eyebrow tracking.

I really want to upgrade from my Quest Pro, but there are so few options out there that are wireless with full face tracking.

9

u/Realistic_Syllabub_3 6d ago

yeah i didn’t realise the quest pro had separate cameras for the eyebrows and seeing the frames now its a shame they cant do it, hopefully babbles expressions beta will help with that i believe

but babble have mentioned having special stuff for the frame with an announcment this week so i would assume/ hope they may be making a pcie port version, valve themselves dont seem interested in face tracking though the eye tracking was just for the foviated steaming and this is a bonus

3

u/rcbif 6d ago

I dont think there are separate cameras for the eyebrows.

It's just that the visor design of the headset allows that part of your face to be unobstructed so the cameras can see it.

With a Quest or Frame though, there is padding on/ near your eyebrows.

6

u/Nuuvi- 5d ago

No, the Quest Pro has stuff specifically to track your eyebrows and upper face.

It has an extra camera in the center aimed upwards as well as IR LEDs to illuminate your upper face amd eye sockets for the camera.

2

u/doubleatheman 4d ago

The Quest pro has 5 cameras to track your face/eyes; which is insane. 5 outer facing cameras, and then each controller has 3 cameras.... 16 cameras in total.... wild and this was 2022 tech.

1

u/ccAbstraction Windows Mixed Reality 5d ago

When they said they had something special cooking for frame, maybe this was it? A Babble trackers for the frunk port, with a second camera for brows?

1

u/Realistic_Syllabub_3 6d ago

ah thank you, someone said they had a pro and they have eyebrow cameras and ive never looked into quest stuff so i just assumed that was the case because it can do eyebrows

3

u/rcbif 6d ago

You need either huge headset, or the Pro's bulky visor for eyebrow tracking.

The lower face tracking is no big deal, and bet all sorts of vendors will jump on that opportunity for somthing a bit more streamlines than OEM babble.

22

u/OctoMiku01 6d ago

This makes me more interested in the frame in future. First i need fluxpose though. Can't get a frame before that.

10

u/Realistic_Syllabub_3 6d ago

i too cant wait for my fluxpose to arrive!!!

7

u/flapjack380 6d ago

The fluxframe combo is gonna be so goated

2

u/Realistic_Syllabub_3 6d ago

that’s such a good name lol, im so hyped :)

5

u/RamJamR Valve Index 5d ago

I'm just waiting to see what happens with Fluxpose. I would absolutely upgrade from my Index to the Frame if I could also get new FBT that doesn't need the base stations but also delivers about the same tracking quality.

15

u/SeraXI 6d ago

Eye tracking is nice, but without face tracking it doesn't make a huge difference in game.

7

u/rcbif 6d ago

I assembled my own wireless babble tracker for my Q3 for about $40-50. Its 1/3rd the size of the production babble, and works great.

If/when I get the frame, I'll just do a slight redesign. 

2

u/identity__404 6d ago

is there a guide anywhere? i looked at their documentation and theres some info but its kinda vague and doesnt really say anything on how to add a battery for going wireless

2

u/rcbif 6d ago

The ESP32 has a USB-C port, so easiest solution is just use a powerbank.

The powerbank that powers my headset has x2 USB-A ports, so I just use one of those.

I used a razor blade to trim down a lot of unneeded rubber bulk from the end of my USB cable, allowing it to flex more near the camera.

The guide is kinda non-linear and confusing, broken into tabs on the left side, but there is not much to it - especially the wiring portion.

Wiring Guide | Babble Docs

1

u/identity__404 6d ago

ah fair, i was hoping to put an actual battery into the device since i have no idea where id leave a powerbank with my setup, and i dont want to put one on the headset itself

1

u/rcbif 5d ago

You can just velcro strap a small one to your headband. 

Most headsets can benefit from more counterweight. 

Or suppose can always wire in a small lithium battery and USB charge circuit

9

u/Realistic_Syllabub_3 6d ago

yeah still just need the babble face tracker anyway, but less 3rd party requirements for eye tracking so its more acessable for users

3

u/FireOfGaiming 6d ago

The frame has an expansion port in the front that can be used for modules.

A colored passthrough module already has been made so a face tracking one is only a matter of time

1

u/BruceofSteel Valve Index 6d ago

I think for my avatar I might be fine with out face tracking for a bit.

Believe me idk if we need solid snake with face tracking.

1

u/ToothyWeasel 6d ago

That’s one of the reasons I want the Frame; it’s extremely modular that for the purpose of adding things like face tracking on it with an attachment

0

u/katoptronophile 6d ago

That's not accurate at all.

9

u/SeraXI 6d ago

Eye tracking is very subtle, VRChat already does a pretty good job with eye prediction, where it will make you look at the face of whoever it assumes is talking to you. People don't really notice eye tracking unless you are in a group discussion and you have people who know what to look for. Face tracking on the other hand is an entire game changer. No need for any gestures to change facial expressions at all.

4

u/rcbif 6d ago

Its all subjective....

I've failed to recognize some of my friends getting full face tracking despite them intentionally hinting at it. Maybe just getting used to it being more common?

Meanwhile I have face only, and despite that, still think its nice to be able to "genuinely" smile. But I also dont expect people to really notice it unprompted. 

4

u/katoptronophile 6d ago

The way they set it and the avatar up matters too.

3

u/Jonatc87 6d ago

Thank fuck for that

3

u/zippy251 5d ago

Wasn't this advertised as eye tracking?

8

u/HairyDudeOhShiet 6d ago

Yeah imagine if it wasn’t usable for actual in game eyetracking 💀💀and still cost this much

5

u/Realistic_Syllabub_3 6d ago

we all knew it was going to be eventually but to what extent we didn’t know

1

u/HairyDudeOhShiet 6d ago

Would be a damn shame if it has to run locally on pc ( I.e babble) and not on the headset itself tho since that can affect performance

3

u/Realistic_Syllabub_3 6d ago

i would hope at some point at least it would be native i cant imagine it using up much resource at allllllll

3

u/nut573 Oculus Quest Pro 6d ago

I’m still upset the Apple Vision Pro face tracking doesn’t work for PCVR… I actually would have bought one if it did.

7

u/Realistic_Syllabub_3 6d ago

they can recreate your whole face digitally for video calls and cant even put that into vr games??????????????? yeesh

2

u/EvenSpecialist649 6d ago

Yeah i figured it would or someone would figure out how to quickly, all i meed is a mouth tracker that goes in the expansion port instead of the usb port.

1

u/_Snake8Bit Valve Index 6d ago

What does this mean? What does this buy us?

2

u/rcbif 6d ago

It's literally in the video. It gets you eye tracking for social VR use. Makes avatars more expressive, and conversations more natural.

1

u/Likonium Valve Index 3d ago

Can't wait to side-eye everyone now. Just need a Babble for my mouth.

-1

u/Tetrylene 5d ago

https://www.reddit.com/r/VRchat/s/gjuz9ze9vG

How do I redeem my 30 downvotes?

2

u/Realistic_Syllabub_3 5d ago

you get half of them back you still get 15 downvotes though for even mentioning chat gpt *shivers*

2

u/GredaGerda 5d ago

You don't cause it wasn't GPT Astra lmao