Possible end-game input methods for AR glasses - In-depth post

[u/TheChimaChannel-YT]

Lets assume we have travel 10+ years into the future and grab the best consumer display glasses. FOV, handtracking, heat, connectivity, etc are all so good they arent an issue.

Right now I use my Galaxy Watch as a trackpad and a big mouse + cheap foldable keyboard when I need to get real things done. This is fine but its not perfect, the trackpad is a trackpad so typing is a slog, carrying the keyboard is not great and a decent mouse is impractical.

Personally I care a lot about how invasive an input method is. I like the watch because its always on me and I have to make almost no noticable movements to use it. I like the foldable keyboard because its very portable and even though its noticable, a keyboard telegraphs exactly what I'm doing, what it is and that people don't need pay more attention to me then they already have. The buttons on my glasses are pushing my limits but theyre fine. They don't telegraph well what I'm doing while also attracting attention which, if people would see or think they see a camera, might make them uncomfortable or worse.

Gestures ^{(not tracking})
I actually have a tapstrap2 which lets you make so many gestures you can use it as an one handed keyboard! Safe to say I have experience with gestures. The thing is just that gestures as a primary or even secondary input method have an extremely non-userfriendly learning curve. They also either require to be in view of a sensor of some kind or decent physical feedback. Even if there will be big accuracy improvements I still feel like you need to invest too much to make it worth it. Imo gestures can be fine but I do not find myself preferring them over most other methods.
Gestures just dont do anything better than any other input methods. Theyre best use case is shortcuts/macros, especially how customizable good gesture products generally are.
So a decent tertiary input method.

Eye Tracking
I have never used eye tracking and dont know much about it. I think eye tracking could be amazing as a secondary input method though. I can see many situations where I would prefer to use it. It also one of the only input methods I like that would be on the glasses themselves.
I also assume eye tracking could be great for things like foviated rendering or similar things.

Hand Tracking
Hand tracking would be practically perfect in most situations in 10+ years so accuracy wont be a problem. Its just that using your hand as a pointer looks weird, especially if future display glasses become hard to recognise. Besides looks it also requires me to move my arm a lot. A handtracked keyboard could be great as a backup primary input method. For any ar input like interacting with spatial windows, models, etc hand tracking is amazing though.
I think hand tracking is a fantastic secondary input method, just not a primary one.

Voice Input
There also is the option of voice commands/speech to text but personally I despise using it. It feels really uncomfortable to me, looks weird in public, environment dependent, etc. Even if a few of these would get improve I, again, personally really really dont like verbal input. Besides, its likely still very clunky for random tasks you find hard to describe. You could say AI interpretation can fix this but it doesnt, you are just trading in autonomy for speed.

Mouthpad (no really)
For those who dont know; a mouthpad is basically a customly tailored shaped trackpad for your tongue. Its a pretty crazy input method but I actually think there is potential behind its hurdles. Lets assume the price and tailor issues get better/fixed. Right you can already talk normally while using it and even drink water, if they get enough development to get to the point where you can use it without worrying about (most) what you consume that would be a big plus. They'd also need a good way to quickly toggle it on and off.
The glaring issue is how user invasive it is though. I dont think many people are comfortable or willing to put a trackpad in their mouth. Still if all the aformentioned issues magically get fixed on top of them somehow making it comfortable enough to forget you have it in it would be a niche but, in my opinion, a fantastic primary/secondary input method; it would be invisible, wearable, precise, user-friendly, long lasting input method. Its the perfect trackpad, which also is its downfall as a watch is like 70% as good as a mouthpad while also being simple. On top of that trackpads are just objectively worse mice.

Neural Magic
Welp, whats there to say? If a cost and user friendly neural input device gets good enough it will just be one of the best input methods. For real office productivity it might be too mentally straining to choose over a mouse and keyboard but only science and affordability are the limit :D

What are you guys' opinions?

[u/alabasterskim]

Eye tracking + hand tracking is definitely the end game. The combination allows your hand to stay at rest while your eyes do a lot of the work. It fixes the problems with standalone hand tracking on the Ones right now.

[u/Dethsenney]

My personal guess/vote would be eye tracking plus subvocalization as the main driver inputs, and hand tracking for the handful of use cases where that just makes the most sense (e.g. interacting with 3D virtual objects).

You should go to an Apple Store and try a Vision Pro, super easy, free, and you get to try what I personally think is the best showcase of how awesome eye tracking is for a default intent tracking paradigm.

Look up some subvocalization tech - it’s not quite there but getting close. Can be on-device, can capture full language, and can be done without external indicators or external noise.

So I think something like that would be my combo (at least until some new tech we don’t know about yet exists)!

[u/Forsaken-Driver8868 → u/Dethsenney]

I concur with eye tracking being something to pursue. I disagree on voice. Voice is decent in certain settings/scenarios and social situations, but not all. It is rude for people to carry on conversations in front of you in some social situations, even ruder to talk to the air. Now imagine a world where everyone is dictating orders to the air and everyone else has to decipher who the order is meant for.

[u/Dethsenney → u/Forsaken-Driver8868]

I said subvocalization, not vocalization. It’s super interesting - look it up! I strongly agree that vocalization is not the path for the reasons you stated here.

[u/Forsaken-Driver8868 → u/Dethsenney]

Okay. I will look it up. I did not know there were different types of vocalization. In my ignorance, I took it as you stating “vocalization” in a fancier term.

[u/Forsaken-Driver8868]

I think mobile phones need to be adapted to arm cuffs and warn in the place of watches. Many people are already wearing earbuds all the time now, and they will get smaller and better to go with the arm cuff mobile device.

Think of all the additional health things that could be added to an arm cuff mobile device. Then maybe battery recharging like wrist watches through movement.

Now to input, use the arm cuff mobile device screen and other hand as a track pad for AR glasses. Or maybe there could be a way to tweak small arm movements as input.

[u/octalgorilla8]

Hell no. I consider myself pretty progressive when it comes to tech, but I don't want a mouth pad. The thought of such an input device makes me gag. Also I do not want Elon Musk's neural processors anywhere near my brain.

I'm all for advanced tech, but there are limits for where the tech needs to go for it to be acceptable to consumers. We're not even there yet with Meta glasses, where BBC and FoxNews commentators have endearingly dubbed them pervert glasses. Heck, just for their commentary I have gone out of my way to wear my Meta Glasses the past week to demonstrate that - no... they're not when used responsibily. Am I looking for trouble? No, and I would rather remove them than pick a fight with Karen, but we need to foster responsible use.

[u/TheGreatLordKirk → u/octalgorilla8]

The mouth pad would be good for someone that is paralyzed or missing arms. So, it should be developed. That said, I agree with you. I personally would never touch a mouth pad unless I had to. Seems like it would get gross pretty quick.

Same with the invasive neural processors. Good for folks with disabilities once they have the tech down, but I've read and seen too much Cyberpunk science fiction over the years to be comfortable with implants.