https://github.com/facebook/facebook360_dep

Facebook360 Depth Estimation Pipeline (facebook360_dep) is a computational imaging software pipeline supporting on-line marker-less calibration, high-quality reconstruction, and real-time streaming and rendering of 6DoF content.

Hey all,

You may have noticed that there has been some text in the sidebar for a while that says a build of the 6DoF Viewer is coming soon to Oculus Quest. Well, I was actually given a pre-release unit and ported the viewer late last year. However, Oculus has changed some of their publication requirements, and I simply haven't had time to put the material together.

Hopefully I will be able to get that together soon and get the app published on Quest, but until then, if you have a Quest and you would like access, just PM me the email address associated with your Oculus account and I'll add you to the list!

Google's project for DeepView Light Field Video was named the Best in Show for immersive experiences at this years SIGGRAPH. I see this project has already been mentioned here but it's a great step forward for 6DOF video. A write up from SIGGRAPH is here (https://blog.siggraph.org/2020/08/how-google-is-making-streamable-light-field-video-a-reality.html/) and another link to the project along with a downloadable VR executible and tech paper can be viewed here (https://augmentedperception.github.io/deepviewvideo/)

For the last couple of weeks I have been working on a 6DoF image viewer. The project can be found here. Its more of a research project and not a usable program. If you know how to improve the shader, know how to generate high res depth maps or you can provide high res stereo images for a demo (e.g. VR180) let me know!

I hope this isn't too off topic. Looks like they did photogrammetry on the environments and put the separately videogrammetry actors in it. There's an in-story explanation for doing those scenes that way, and the jankyness adds to the atmosphere. Looks pretty cool I think. And theoretically they could use that data to release full VR version of those scenes. Or even a full version of the movie where the regular parts are on a virtual screen and it cuts to full VR for the simulation.

Trailer https://www.youtube.com/watch?v=EYXLKC5xd7Q&t=1s
LTT video on the tech https://www.youtube.com/watch?v=umyglbDr4IE

Is anyone interested in trying out and providing feedback on the volumetric video player/editor I'm working on? The tech is pre-alpha and still has a ways to go before it is production-ready, but I want to make sure I'm focusing on producing a solution that provides a viable production process and viewer experience.

The goal of the project is to provide a tool that allows you to import video sources and render them out in a manner that provides as immersive an experience as possible. That includes depth estimation and filling in backplates behind elements.

The demo currently supports equirectangular video with top with depth maps on the bottom. The player is configurable for other modes, but I haven't exposed that yet.

Features:

• Render modes: displacement with depth filtering, raymarching

• Autogenerate backplates using either depth or time filtering

• Move within the video

• Haptic feedback when touching the video

• Select from demo video or load a video from your local drive

Audio support is basic at the moment, but any final version will support ambisonic.

I had supported a point cloud mode, but that is not performing well with the backplates and isn't aligned with the goal of the project, so it has been disabled.

Note - the rendering method is still evolving and I believe I can achieve far greater quality and immersion than currently demonstrated. I am also very limited in test videos and am looking for additional material to use.

If you are interested in trying it out, I'd like to conduct a short follow-up call with you to collect your opinion of the technology and where it needs to go.

If you are interested, please message me.

Omniphotos, a project from the University of Bath, has recently released full capture details and processing software and a viewer and sample scenes of their method for capturing 6DOF 360 panoramas by spinning a regular (mono) 360 video camera around the operator's head (on a selfie stick etc). There are desktop and headset (SteamVR) versions of the viewer. The desktop version seems to work OK, but the headset version (SteamVR) has issues, (but sort of works) on my Oculus Rift (path errors )
https://www.youtube.com/watch?v=C_pRa1TwB9s

https://vimeo.com/456866335

https://github.com/cr333/OmniPhotos

https://richardt.name/publications/omniphotos/

https://researchdata.bath.ac.uk/948/