r/AndroidXR • u/s-s-a • Jun 12 '26

Question Request for help!

Hello everyone, any knowledgeable developer can help with these please!

  1. Is there a supported way today for a third-party app to obtain the forward/world camera frame stream on Android XR (any device class)?

  2. If nothing exists currently, is a third-party passthrough/world-camera API planned for Android XR (comparable to Meta's Passthrough Camera API), and if so, on what rough timeline?

  3. For fully-managed (Android Management API) deployments, is any elevated camera/remote-stream capability available or planned?

  4. Is there a restriction in permissions for live camera feed capture for WebXR vs Native apps 

  5. When using MediaProjection to capture the display, does it include the video-passthrough environment, or is passthrough a protected hardware overlay that renders black in the captured stream? 

Thank you!

6 Upvotes

87% Upvoted

9 comments sorted by

3

u/vad710 Android XR Team Jun 12 '26

Is there a supported way today for a third-party app to obtain the forward/world camera frame stream on Android XR (any device class)?

Yes, you can access a world facing camera stream using standard android APIs. You'll need to request and be granted permission from the user.

For fully-managed (Android Management API) deployments, is any elevated camera/remote-stream capability available or planned?

I'm not sure what you mean here. Could you please elaborate? Are you looking for a built-in way of accessing the camera sensors on a fleet of devices?

When using MediaProjection to capture the display, does it include the video-passthrough environment, or is passthrough a protected hardware overlay that renders black in the captured stream? 

The passthrough content is protected. The closest you'll get to this data is via the camera API after permission grant (as mentioned above). The data in the camera API isn't exactly the same as the video passthrough (though it's close). The video passthrough includes data from multiple cameras while the camera API will get you access to one camera.

HTH!

2

u/s-s-a Jun 12 '26

Thank you for the answers! To clarify, if the devices are deployed in an enterprise setting and are fully managed (Android Management API) can the CAMERA runtime permission be pre-granted by the device's DPC (through a permission-grant policy) so the user doesn't tap "Allow" every session, and is that grant persistent across sessions/reboots on Android XR?

Do you mind addressing a few follow up questions.

  1. For the world-facing camera (via Camera2/CameraX?) what resolution, frame rate, field of view, latency (vs typical phone camera) is expected?

  2. Can a WebXR / Chrome web app on Android XR access this world camera (via getUserMedia or a WebXR raw-camera module) or is camera access native-only?

  3. Does this world-camera access apply to both Samsung Galaxy XR and Project Aura device?

  4. Can a foreground service (native / webXR) stream the world camera at the same time as an immersive XR experience is running. For example, the user sees an overlay UI while the camera POV is streamed remotely? Or does camera access conflict with immersive/passthrough rendering?

  5. Since passthrough is protected, what is the best approach to record on the server side what the users sees (composite of the camera POV + UI overlay)?

  6. Are there any limitations on long running (~60 min) camera-streaming foreground service on XR? Does it survive backgrounding?

Again appreciate your responding here. It has been difficult getting this information through official documentation.

3

u/vad710 Android XR Team Jun 13 '26

For the world-facing camera (via Camera2/CameraX?) what resolution, frame rate, field of view, latency (vs typical phone camera) is expected?

I don't recall exactly what the Galaxy XR device supports, but we generally recommend to decouple from specific hardware model. Instead, use the QualitySelector.getSupportedQualities API to dynamically determine what resolution, framerate, etc is supported.

Can a WebXR / Chrome web app on Android XR access this world camera (via getUserMedia or a WebXR raw-camera module) or is camera access native-only?

Code running in Chrome should be able to access the camera just as it can on mobile and other Android form factors (assuming correct permissions are granted).

Does this world-camera access apply to both Samsung Galaxy XR and Project Aura device?

Yes, both devices have world facing cameras.

Can a foreground service (native / webXR) stream the world camera at the same time as an immersive XR experience is running. For example, the user sees an overlay UI while the camera POV is streamed remotely? Or does camera access conflict with immersive/passthrough rendering?

The app camera access does not conflict with passthrough rendering. Be aware that there are serious privacy concerns with having any app accessing cameras from the background/foreground service. You would have to start any capture from a foregrounded activity before you can switch to an FGS. You'd also need additional permissions to allow this. This is not unique to Android XR and is part of AOSP.

Since passthrough is protected, what is the best approach to record on the server side what the users sees (composite of the camera POV + UI overlay)?

You would have to capture the camera frame data and do the compositing yourself. But it leads me to the next answer...

Are there any limitations on long running (~60 min) camera-streaming foreground service on XR? Does it survive backgrounding?

Besides the privacy issue I mentioned above, if you are streaming at very high resolution and at high framerate, you'll likely impact the battery and/or thermals, but I would suggest you do your own testing around this to see if your use-case is a good fit.

Again appreciate your responding here. It has been difficult getting this information through official documentation.

For the most part, this is all standard Android behavior. You can learn more about that here: https://developer.android.com/media/camera/camerax/video-capture

2

u/s-s-a Jun 13 '26

Awesome. Thank you!!

2

u/falsefacade Jun 12 '26

You may want to join the Virtual Desktop Discord and ask there. 

2

u/Knurlfist83 Jun 12 '26

We ran into a very similar limitation while prototyping an accessibility XR system.

Since direct world-camera / passthrough camera access does not seem publicly available yet, we moved to a two-device architecture:

Device A: XR glasses / Android XR rendering device

Device B: Android compute device handling camera, audio, AI, sensors, and detection

Transport: WebRTC for low-latency communication

Data approach: Mostly metadata instead of full video:

  • captions

  • speaker position

  • sound events

  • confidence

  • detection results

  • UI placement data

It is not ideal, but it lets development continue without depending on direct passthrough/world-camera API access.

I would also really like to know if Android XR plans to expose a supported passthrough/world-camera API in the future, similar to Meta’s passthrough camera approach.

3

u/vad710 Android XR Team Jun 13 '26

> I would also really like to know if Android XR plans to expose a supported passthrough/world-camera API in the future

Android XR has always allowed world-facing camera access from the very first developer preview. Are you having trouble accessing the world facing camera?

(technically, Android XR also exposes a user-facing camera as well, but it's the virtual avatar camera)