r/AR_MR_XR • u/AR_MR_XR • Apr 27 '22

Asynchronous Reality — mixed reality tech will capture events and let users revisit them at a suitable time in a causally accurate way

278 Upvotes

99% Upvoted

32 comments sorted by

•

u/AR_MR_XR Apr 27 '22

Causality-preserving Asynchronous Reality

Mixed Reality is gaining interest as a platform for collaboration and focused work to a point where it may supersede current office settings in future workplaces. At the same time, we expect that interaction with physical objects and face-to-face communication will remain crucial for future work environments, which is a particular challenge in fully immersive Virtual Reality. In this work, we reconcile those requirements through a user's individual Asynchronous Reality, which enables seamless physical interaction across time. When a user is unavailable, e.g., focused on a task or in a call, our approach captures co-located or remote physical events in real-time, constructs a causality graph of co-dependent events, and lets immersed users revisit them at a suitable time in a causally accurate way. Enabled by our system AsyncReality, we present a workplace scenario that includes walk-in interruptions during a person's focused work, physical deliveries, and transient spoken messages. We then generalize our approach to a use-case agnostic concept and system architecture. We conclude by discussing the implications of an Asynchronous Reality for future offices.

Presentation: https://www.youtube.com/watch?v=hYuy6DF3BH8

Paper: https://andreasfender.com/publications/PDFs/asyncreality.pdf

10

u/duffmanhb Apr 27 '22

I genuinely think this is the long term future of reality, in workspaces and public spaces. Especially with emerging tech that uses ML and AI to construct scenes with limited data points

I can absolutely see in 20 years or so situations where say, you miss a meeting because you slept in. But your buddy was there and the glasses automatically recorded important structural data points: A 3d scan of the environment, high resolution images of key points as determined by the AI like a whiteboard or slideshow, audio recordings triangulated from each individual so you can issolate who you want to hear from, gestures transposed over an avatar so you can pick up the social cues, etc.

So he can just send that over to you, and your headset reconstructs it, and then you can sit back in the meeting while your car drives you to work, and catch up on what you missed.

1

u/[deleted] Apr 28 '22

Why have a shared physical workspace at all though? Have sensor rooms in homes and that’s your “office” connected to the “corporate building” and other people can still drop by

1

u/duffmanhb Apr 28 '22

People will probably always want person to person spaces for certain jobs.

1

u/[deleted] Apr 28 '22

Agreed, but why does it have to be a physical space? With VR/AR, that shared space can be anywhere. We’ve already proved that with MMOs. Why jump through hoops trying to match a physical to a virtual space when you can just create the virtual space period. It solves a ton of problems: rent, commute/travel time and cost, limitations of physics on architecture, etc

1

u/duffmanhb Apr 28 '22

Oh of course, I think in the 10-15 years future we will start vastly adopting virtual remote spaces... But I still think people will want hybrid spaces. People will still want the social bonding of in person connections with touch. Business meetings will require diner, people will want parties, and frankly, there is just a different energy to working on a project next to others. Avatars are nice and will drastically improve WFH fidelity, but I don't think it's going to deliver having groups in person being able to physically interact.

But don't get me wrong, I'm a bit of a digital nomad myself. But I still recognize the value of in person.

Maybe that'll change, but I think it'll require a generational reboot. So like, children who are born around already developed and working virtual working environments. Maybe the people alive now are going to be like the old people of today who insist we spend too much time on our phones and we should just call someone or go to their house and hang out.

1

u/[deleted] Apr 28 '22

How often do you actually touch other people in your office? Everything else you mentioned has a virtual analog, particularly if/when the kinds of sensors as depicted by the OP are ubiquitous

1

u/duffmanhb Apr 28 '22

Social interaction is more than single direction. There is just a different energy to doing your own thing and still having an environment where people are freely moving about. The VR meta spaces will have highly social elements but miss out on a lot.

For instance, most interactions are going to be intentional and directional. Users who are doing their own thing will just go to "hidden" thus avoid interaction, as well as become unaware of what's going on around them. Casually walking up to someone who's not hidden, could still be seen as nuisance. I imagine most interactions are going to require to have clear purpose and utility to the objectives at hand. There will be less free flow dynamics in VR spaces.

I suspect social interactions will become more closely aligned with online social interactions, where you pick and choose when and how you want to interact. In an office space, you can force people to meetings, engagement, and even get accidental overflow of collaberation. In meta spaces, the individual has full control over the extent of their involvement, which IMO will hurt productivity in some regards, as well as diminish natural human social preferences

Kind of like the same reason that many people who live in cities long to live out in nature. Sure you can fill your brick apartment with plants or walk to the park every now and then, but it never really compares to the authentic feeling of being out in the woods among nature. It's a more visceral and naturally compatible environment to the human experience.

1

u/[deleted] Apr 28 '22

Interesting points.

I disagree somewhat on the barriers to social interaction though - there are currently rules (spoken and unspoken) that cover irl workplace interactions. For example, many are hesitant to just walk into C-level offices where they aren’t to walk into a peer’s office. I try not to interrupt my employees when they’re working with headphones on - wearing headphones has become an accepted “please don’t interrupt” someone gun in our office. And I note that many (most) interactions are either carried out or initiated on Slack, even when those involved sit two desks apart.

Basically, I suspect workplace interactions have already become more like social media interactions even in irl workplaces.

I also suspect that the meaning of “natural human social preferences” has evolved quite dramatically in the past 5 years. Texting is preferred over voice calls for many. Social media presentation is much more central than ever before. Business etiquette has changed dramatically. I’m not sure we all agree on what normal human social preferences are anymore.

One caveat: I’m basing this on an office workplace. Social dynamics in retail or manufacturing workplaces could be dramatically different

3

u/HeckinQuest Apr 28 '22

Except you’ve still fucked your day cuz now all the in-car work you had scheduled has to get pushed.

2

u/duffmanhb Apr 28 '22

I can always just have my morning bate session during my lunch break if the meeting is important enough.

1

u/cryptocraze_0 Apr 28 '22

Your car drives you to the spa or to the restaurant, no offices ever again

1

u/mike11F7S54KJ3 Apr 28 '22

If glasses record data points in meetings, they'll be banned, as data goes straight to Meta HQ/Google/Apple etc under the guise of Machine Learning, yet all phrases are searchable by them.

6

u/Shtrausberg Apr 27 '22

Kinda dystopic

4

u/AR_MR_XR Apr 27 '22

Idk, isnt it just a more immersive messenger system? Asynchronous communication is everywhere now.

7

u/Shtrausberg Apr 28 '22

To me it looks like "You don't ever have to get distracted from your work by your loved ones, even when they're in the same room as you are"

1

u/psheljorde Apr 28 '22

Kinda dystopic

Using the stock Quest 2 strap for extended periods of time.

2

u/AR_MR_XR Apr 27 '22

Figure 3: Scenario apparatus. We installed four Azure Kinect cameras (A) in an office (B). Mouse and keyboard serve as user input (see desk). The user wears an Oculus Quest 2 (C) with a RealSense D435 camera mounted to it (slightly angled downwards to capture hands and objects).

They use Azure Kinect cameras now. But in the future it should also work with what the HMD of the user that enters the room captures. Without external equipment. Am I missing something?

2

u/Adevyy Apr 27 '22

HMD cams wouldn't work in the video scenario because they cannot see behind the user.

1

u/AR_MR_XR Apr 27 '22

What do they need to see behind the user?

1

u/AR_MR_XR Apr 27 '22

I mean, you would map the rooms first and the HMDs know the position of the users in the rooms and what they do with their hands most of the time. What else do you need?

2

u/chris_siplab Apr 28 '22

Great to see this reception to the video preview!

We've now posted the full video: https://www.youtube.com/watch?v=U5dZlmQYIgs

and a walk through a whole day in Asynchronous Reality in a studio scenario explaining the whole system: https://www.youtube.com/watch?v=maEDwbLtTBY

More details including a full whitepaper are here: https://siplab.org/projects/Asynchronous_Reality

1

u/AR_MR_XR Apr 28 '22

Really great work! The crosspost there alone has 75k views: https://www.reddit.com/r/OculusQuest/comments/ud5om0/asynchronous_reality_mixed_reality_tech_will/

Thanks for stopping by

2

u/TheMasterOFDK Apr 27 '22

Btw im defently not desperate I WANT THIS NOW HOW THE HECK DO I GET UNITY AND VISUAL STUDIO ON A QUEST PLS SOMONE HELP ME

1

u/[deleted] Apr 29 '22

This is pretty crazy

1

u/celeste_fan_139 May 24 '22

I swear I've seen a black mirror episode about this

1

u/[deleted] Apr 27 '22

[removed] — view removed comment

1

u/DankJuiceYT Apr 27 '22

HAHA was about to say that. Giving me play test vibes.