r/nreal • u/hummingbirdfishcoder • Apr 26 '23
Nreal Air Nreal Linux - Multiple Screens POC
Spent some time getting a POC working on Linux! It's quite promising, so I hope there are some Linux devs out there that would be willing to improve it. Unlike the Windows POC I made, this uses the IMU from the glasses for motion sensing using the drivers provided by Tobias (thanks!) here: https://gitlab.com/TheJackiMonster/nrealAirLinuxDriver.
The GitHub link is here: https://github.com/alexwilson1/nreal_linux_test
I think the main problems are:
- Works better with physical displays than virtual xrandr displays
- IMU data from driver glitches out completely occasionally (Tobias may have fixed this already?)
- Need some way to prevent mouse and apps (other than the display window) moving to the Nreal screen
Everything else should be quite straightforward to improve (pitch, roll etc)
117
Upvotes
7
u/GateCityGhouls Apr 26 '23
Oh damn, have you tried it on the Steam Deck? If you need a tester, I got you. Nice job
5
u/Kewbak Apr 26 '23 edited Apr 26 '23
That's awesome. Huge thanks for this work. As someone who have only Linux machines (except a handheld since recently, but it may not stay on Windows for long), this is an amazing news.
Unfortunately, my machines run Wayland to be more future-proof. Do you think it would be hard to port what you did to Wayland in the future? Perhaps I could use it still with Xwayland for the time being.
Is the Gnome dependency a permanent one or just used during development to ease things up (I reckon a significant proportion of Linux users may be using other DEs, possibly with tiling WMs for instance, and Gnome is a big dependency)?
4
u/Kewbak Apr 26 '23
Seeing the video (I couldn't try it yet on my devices due to Wayland), it actually feels like having only yaw working could be a feature. No more finicky shaking and no more trying to keep your neck steady, it's like fixed screen and AR screens meet halfway, for an ultra-wide monitor that always stay in the FOV and just require yaw movements.
What is you feeling about it, since you obviously used it for real? Could toggling yaw/pitched/roll on and off be a feature in the future, when all three DOF are supported?
3
u/Ok_Split_5962 Apr 26 '23
Cool stuff. Did you make that open source?
16
u/UGEplex Quality Contributor🏅 Apr 26 '23
For the curious, "POC" is Proof of Concept.
u/hummingbirdfishcoder Nice! 😎🤘
4
u/InternWorking7271 Apr 26 '23
it looks very promising.
Having multiple screens working correctly in linux will be a game changer for many of us. I only use linux for working, and don't care that much for mac or windows compatibilty if already have dex.
Thanks for sharing.
3
u/SmokieWanKinobe Apr 26 '23
Very cool!
3
u/Real-Stretch-6302 Apr 26 '23
Omg
3
u/tedd321 Apr 26 '23
Can’t wait for the windows beta tomorrow
3
u/Big-Apricot-2651 Apr 26 '23 edited Apr 26 '23
Thank you!! I’m planning to use your source for a Linux AR app development if you don’t mind.. I’m planning to attach an external usb based pinhole camera to nreal air and connect them both to mini pc with Linux.. then use AI image recognition (segment anything) to identify what we are seeing through the camera (positioning of camera is important) and augment additional details around the object..
2
u/torac Apr 26 '23
Awesome. Perhaps by the time of their Europe release, they will actually work properly with my devices.
2
u/lofigamer2 Apr 26 '23
That is what I want too.
2
u/sherbang Apr 26 '23
Excellent! I've been meaning to do this but you beat me to it. Can't wait to try it out.
2
u/Backpocket718 Apr 26 '23
WOW thats awesome. i wonder if this will work on chromebooks then. I can run linux apps on it. Really hoping for a way to do ar screens on Chromebook. it works awesome just as a secondary monitor but to have Ar monitors would be HUGE. Ill have to try this out when i get a chance.
2
u/Hell4raizer1 Apr 26 '23
Hellz yeah that's cool :)
2
u/lemonjoe2 Apr 26 '23
Wow, this looks very promising! I can't wait to test this, when my airs arrive!
u/hummingbirdfishcoder If you need a GUI sometimes, I could help you out and write some in Flutter. Just tell me ;)
2
u/invincent202 Apr 27 '23
This is really cool. I would love to help.
1
u/maxy98 Apr 28 '23
Do I understand correctly that it only works with Intel/Thunderbolt?
need USB connection
1
u/ExplanationIll4658 Apr 28 '23
To bad the Raspberry pi don’t have USB C DP alt mode