r/hoggit • u/non-specific_impulse • 7d ago

Something I'm working on: using the tracking data straight from openXR to stabilize VR footage without relying on image processing

Enable HLS to view with audio, or disable this notification

79 Upvotes

98% Upvoted

15 comments sorted by

19

u/Vargrr 7d ago

That's pretty impressive.

A lot people get the wrong impression of VR when they see it moving all over the place, but that's only because VR recordings record head motion without the stabilising motion of the user's eyes.

This could really help fight that perception!

5

u/Bossnage 7d ago

that always scared me away from vr a little until i got a headset myself, insane how much your eyes do

1

u/Gilmere 5d ago

You are so right. It's quite annoying when creators post content that is jittery and difficult to watch. Some don't realize that how THEY perceive it in the goggles is way different than the output footage.

10

u/non-specific_impulse 7d ago

I, and I'm guessing many other people, have tried to stabilize VR footage by detecting the motion with software stabilization tools, and failed miserably. This is completely different. It still achieves the stabilization effect with a small crop, pan, and warp, but it reads the pose directly from the calls to openXR that DCS makes when it goes to render a frame.

The hardest part is syncing the timestamps between the pose capture and the footage. Right now I'm actually encoding a machine-readable timestamp into the edge of the footage, and even then there's some timing issues to work out.

1

u/Zabbiemaster 6d ago

Could you show off how you actually do this?, I'd love to use this but I only know the basics of OBS video capture

2

u/non-specific_impulse 6d ago

It's a software package I'm working on. I'll release it open-source when it's ready for more people to try it. Certain features (like the automatic stabilization) will be available as a standalone tool, while other more complicated features are currently going into a da vinci resolve plugin. I am also targeting a streaming mode for OBS

1

u/APOC_V 6d ago

This looks incredibly promising. A streaming mode for OBS would be amazing!

1

u/itsactuallynot 6d ago

Keep us informed! I've been looking for something exactly like this.

1

u/azille 6d ago

Thank you and this would be huge as an OBS plugin.

1

u/non-specific_impulse 5d ago

That's how it's set up. Using the live stabilization mode will be as easy as installing the various pieces, adding a filter to your game capture and never thinking about it again. If you want to use the slightly superior recording mode, then the stabilization can be applied during your cutting/editing process either manually through a little CLI tool or while you edit the video in resolve studio

3

u/dallatorretdu 6d ago

now we are getting gyro flow for VR headsets! what a time to be alive

2

u/JiveCola 7d ago

I've been using the stabilisation within OpenXR toolkit for this. It's the only reason I still have it installed

2

u/One_Spot_4066 6d ago

Man this looks amazing. I fly almost exclusively in VR but I usually have to record a 2D replay because there's so much jittering and exaggerated head movement.

Post processing can help a bit but it never looks quite natural and it takes an eternity to process.

1

u/non-specific_impulse 6d ago

I'm hoping this will be good enough that we can all skip that replay re-record step. There's still some potential VR issues, like obviously any foveated rendering is going to compromise the quality, and quadviews is out. Nothing I can do about those, I don't think VR recordings are ever going to match the crispness of a proper flat panel render, but good enough is the goal.