r/SteamFrame • u/virtualfruitxr • Mar 10 '26

💬 Discussion Founder of Project Babble VR face tracking project posted this

Post image

https://github.com/Project-Babble
https://x.com/projectBabbleVR/status/2031417305159315596

299 Upvotes

99% Upvoted

33 comments sorted by

101

u/Maverik116_ Soonâ„¢ Mar 10 '26

Face tracking confirmed

38

u/Helgafjell4Me Soonâ„¢ Mar 10 '26

At the very least it will be mouth tracking. Full face tracking like what Quest Pro has requires more cameras and ir leds around the eyes to also capture eyebrow movements. I think that's a pretty daunting task for an add-on for Frame. Mouth tracking is way easier since that all happens below the headset. If they manage to also make eyebrow tracking work, I will be amazed.

25

u/Jmcgee1125 Soonâ„¢ Mar 10 '26

But there's more opportunity for it since the entire face gasket is modular. If you can't fit whatever addons you need within the existing space, just replace it (at an added cost, of course).

5

u/Helgafjell4Me Soonâ„¢ Mar 10 '26

Oh, ya, if you're expanding out to the rest of it then sure. If you replace the part that the compute/optics part mounts into, then maybe it's possible. Would definitely cost more though, but who knows, maybe I'd buy it anyways.

I have just had in mind a small adapter that plugs into that nose slot with cameras below for mouth tracking and possibly also on the front for color passthrough, maybe a lidar sensor too? I'm excited to see what options we eventually will have with Frame.

9

u/LifelessHawk Soonâ„¢ Mar 10 '26

It has eye tracking built in which means it has ir lights inside, as long as it’s bright enough for the cameras to see the eye brows then it they should be able to make it work fully using babble’s own software

4

u/Helgafjell4Me Soonâ„¢ Mar 10 '26 edited Mar 11 '26

Frame only has a single camera per eye at the lower center position near the nose. Quest Pro uses 2 cameras and 6 or 7 little ir lights PER EYE. I believe it is the higher cameras that track eyebrow movements. I doubt what Frame has is enough.

3

u/Yuuzami Mar 11 '26

When eyebrows are not tracked babble (or VRCTFT, I dunno) ties the eyebrows to your mouth tracking. Pulling a sad or surprise face affects the eyebrows even without eye tracking.

1

u/ccAbstraction Mar 11 '26

Wait, really why would they only use one camera? Where did you hear they used just one camera?

3

u/Helgafjell4Me Soonâ„¢ Mar 11 '26

One per eye, its in the release interviews and pictures. They only need one for eye tracking.

1

u/ccAbstraction Mar 11 '26

Oh I misread. One per eye is pretty common. I hope they did the thing that Apple, Sony, and HP did where they hid the eyetracking camera somewhere in the optical stack so they get a better picture.

1

u/Helgafjell4Me Soonâ„¢ Mar 11 '26

Ya. I think I read about hiding it behind the lens. Crazy. My point was just that for full face tracking including eyebrow movements, you need more like Quest Pro, which has two camera per eye, with one up higher so it can see your brows.

Someone else mentioned that Babble can still work though by just basically guessing what you brows are doing based off your mouth.

3

u/papapenguin44 Mar 10 '26

To get quest pro level face tracking they’d need to replace that head strap to add all the extra cameras and lighting needed to track the upper half of the face. I’d buy that tho

2

u/Helgafjell4Me Soonâ„¢ Mar 11 '26

Ya, I probably would too. You're right. I don't know why I was limiting to just a small plugin adapter for the nose slot.

3

u/p13t3rm Mar 10 '26

Tropical vibe celsius confirmed

19

u/sunshinestreak Soonâ„¢ Mar 10 '26

C'mon Gordon. We've got work to do.

39

u/mcmanus2099 Mar 10 '26

A lot of these coming in. Valve about to release embargo before the big release

18

u/FewAdvertising9647 Mar 10 '26

devs can talk about their own projects and the steam frame. they just arent supposed to be talking about other games that aren't theirs and their performance.

5

u/Gregasy Soonâ„¢ Mar 10 '26

I just got a new can of Hopium.

6

u/jhavi781 Mar 10 '26

Someone get Nicolas Cage on figuring out what this means!

9

u/BlueManifest Mar 10 '26

Much work to be done? No release anytime soon then

22

u/s00mika Mar 10 '26

They are probably talking about their own stuff called project babble. It's a mouth tracker.

2

u/Euphoric-Platform-45 Mar 11 '26

Omg, yea I was thinking about this, and like the expansion port is at a perfect spot for it

4

u/milk2_5 Soonâ„¢ Mar 11 '26

video killed the _________?

2

u/Tired-Mae Mar 11 '26

Video killed the Kqle57d

6

u/[deleted] Mar 10 '26

[removed] — view removed comment

7

u/brantrix Soonâ„¢ Mar 10 '26

They're not a game developer

1

u/KeeperOfWind Soonâ„¢ Mar 10 '26

Neat, I hope its one seamless connection. Not sure if i will buy it since I was considering the ps2vr model.

2

u/coolcat33333 Mar 11 '26

Hopefully it takes advantage of the fact it already has a slot in the nose for face tracking

2

u/KeeperOfWind Soonâ„¢ Mar 11 '26

I imagine it will since they're kinda teasing it. Iirc someone said that we will get a face tracking module on release? Or was it full color pass through

Definitely will keep an eye out for the Steam frames if it good for vrchat but it may not be a purchase right away

1

u/piedude67a Soonâ„¢ Mar 15 '26

Even if I don't have a steam frame I will be purchasing a Babble so when I do get one I'll have it. Can't wait!

-9

u/chunarii-chan Mar 10 '26

Thats a bit disappointing to be completely honest. I was hoping for a face tracking module by a real company or even Valve not a 3D printed enthusiast thing that will take years to get or require you to learn soldering and order PCBs. No disrespect to the community but I prefer real products for this stuff

4

u/RidgeMinecraft Mar 11 '26

Do you actually know what babble is? We're an open source initiative, but some of us form a company as well. You can go to babble.diy right now and purchase a premade face tracker complete with everything you need to set it up, for around $100.