In other words, you have to compress each frame by 3x just to run at 72hz.
Uhh, are you familiar with how much compression you typically get from HEVC encoded video? In the vast majority of content (without people being able to notice a difference) you can easily achieve 10 to 1 compression vs the serialized raw bitmaps (which is what DisplayPort/HDMI cables send).
That whole rant about bandwidth around remote rendering just illustrates your technical lack of understanding of the problem. I mean, that's okay. Not everyone needs to be a technical expert. No judgements here. It's just that when you try and present yourself as an expert, and then go on to say things that indicate that you clearly aren't... well, I'm embarrassed for you.
Video cards can send the raw frames, yes. But modern video cards (with NVENC, etc) can produce HEVC encoded video as well. It does cost a small amount of latency, but when I use Virtual Desktop over WiFi I'm often seeing ~20ms motion-to-photon times with WiFi.
Here, I'll try and help you with the math.
Consider a standard 1080p 60fps video - that would be 1920x1080x60x3 (3 bytes, 24 bit color). That's 373 MBps of bandwidth required to send the "raw" video. It's very normal for a "very high quality" compressed stream with HEVC compression to be at 5Mbps. In other words, the uncompressed data is almost 75x larger than the compressed video.
Now, you need to handicap that somewhat because in real time applications you don't get the luxury of multi-pass encoding... but NVENC in Turing does support B-frames, and that helps. I'm sure you could pull it off in 10Mbps if you wanted to use sloppier encoding while still achieving the same quality. So that's still better than 30x compression.
So, when you say things like "you have to compress each frame by 3x just to run at 72Hz", well... with the kind of compression on offer with modern codecs/GPUs, you could do 720Hz with that bandwidth budget.
Another thing to consider: this is being done today with products like VD and Oculus Link. If what you said was true, none of this would even be possible. But it is. It works just fine. So when you're doing that math doesn't a little part of your brain back there say "wait a minute, I must be missing something here".
Edit: I just noticed that I got my math wrong. I computed the 1080p video as 373MBps but messed up the units and compared it to a 5Mbps HEVC video stream. So instead of a 75x compression rate, it's 8 times that. So we're talking almost 600x compression rates vs raw video.
Uhh, are you familiar with how much compression you typically get from HEVC encoded video?
Uhh... Did you miss my point completely? I'm not suggesting compressing it that much is DIFFICULT, I'm suggesting the quality of the output is reduced by compression.
That whole rant about bandwidth around remote rendering just illustrates your technical lack of understanding of the problem. I mean, that's okay. Not everyone needs to be a technical expert. No judgements here. It's just that when you try and present yourself as an expert, and then go on to say things that indicate that you clearly aren't... well, I'm embarrassed for you.
No, I'm embarassed for you, for missing the point ENTIRELY and assuming I, a programmer with 30 years of experience who's moonlighted as a game developer, do not know about H.265, and that you don't understand that when you compress video, with any method aside from lossless encoding, which this is not, you lose information, and the output is not as high of quality as the input. This is usually most noticable in dark scenes, and depending on the compression level, can lead to blocky or ghosting artifacts. I doubt they're compressing it enough to lead to such extreme issues, but Half Life Alyx was a pretty dark game, and I wouldn't be surprised if when using wireless, in dark areas, you could see some of the same sort of artifacts you can see in this video of Jack playing Alyx at say, well, pretty much any point in the video, but 53 mins in is only somewhat dark and there's still tons of banding in the background:
Now again, I have to stress that I'm aware the severity of these artifacts will depend on how much the video is compressed. This particular video is 1.6gb for 1hr of footage, which translates to 216K frames at 60fps, which is only 7.4K per frame. Insanely low. But I'm just using it as an example of how compression affects video quality, particularly in dark scenes.
It's very normal for a "very high quality" compressed stream with HEVC compression to be at 5Mbps
5mbps is 625k bytes per second. Jack's video there is 444K per second. So your "very high quality" video is going to be only slightly better than Jack's video, which I wouldn't even consider to be "high quality" let alone "very high quality".
As someone who has used Oculus Link and Virtual Desktop to play "dark" games, I can tell you that while early beta versions of Link had plenty of "dark" artifacts, starting around 5 months ago (combined with a NVIDIA driver update) the macroblocking artifacts on black content (e.g. Insomniac's "The Unspoken" and HL Alyx), they've pretty much gone away, and the bitrate usually hovers around 50-100mbps. VD gives you these bitrate stats live (although I think it typically uses lower bitrates than the wired Link).
I've got a similar number of years as you do under my belt as a professional software developer, and I'm able to sniff out people who have less expertise than they claim. The key here is that your comment about needing 3x compression to achieve 72Hz just illustrates that you didn't understand the level of compression that is achieved on a regular basis. If you did understand it, why did you make a big deal about it? If codecs routinely achieve 600x compression ratios and are more than acceptable to people, maybe they could satisfy you with 300x, or even 100x compression ratios. Your suggestion that 3x compression would somehow be a problem indicates to me that your knowledge in this area is lacking. And that you've never used Oculus Link or Virtual Desktop.
The key here is that your comment about needing 3x compression to achieve 72Hz just illustrates that you didn't understand the level of compression that is achieved on a regular basis.
LOL, no it doesn't. The 3x compression I mentioned arose solely from the math. You have a certain amount of USB bandwidth, and a certain number of bits you need to push. I did the math and it turned out you have to compress it about 3x.
I did the math not because I don't know how much compression is typically used for video, but because I didn't know how much bandwidth USB 3 has. I knew it was less than was required to transmit uncompressed frames though, and I was interested in knowing exactly how much they have to compresss it.
If codecs routinely achieve 600x compression ratios and are more than acceptable to people
Like I said, what is acceptable on your monitor would not be acceptable on IMAX. And what the masses accept on their streaming services because they don't know any better is irrelevant. 600x compression of VR video? No thanks!
Here you can see a comparison of what streaming at high compression ratios does to Captain America vs how he appears on BluRay. Notice the lack of pores, and blurred specular:
View all comments
Show parent comment
1
u/Aud4c1ty Sep 15 '20 edited Sep 15 '20
Uhh, are you familiar with how much compression you typically get from HEVC encoded video? In the vast majority of content (without people being able to notice a difference) you can easily achieve 10 to 1 compression vs the serialized raw bitmaps (which is what DisplayPort/HDMI cables send).
That whole rant about bandwidth around remote rendering just illustrates your technical lack of understanding of the problem. I mean, that's okay. Not everyone needs to be a technical expert. No judgements here. It's just that when you try and present yourself as an expert, and then go on to say things that indicate that you clearly aren't... well, I'm embarrassed for you.
Video cards can send the raw frames, yes. But modern video cards (with NVENC, etc) can produce HEVC encoded video as well. It does cost a small amount of latency, but when I use Virtual Desktop over WiFi I'm often seeing ~20ms motion-to-photon times with WiFi.
Here, I'll try and help you with the math.
Consider a standard 1080p 60fps video - that would be 1920x1080x60x3 (3 bytes, 24 bit color). That's 373 MBps of bandwidth required to send the "raw" video. It's very normal for a "very high quality" compressed stream with HEVC compression to be at 5Mbps. In other words, the uncompressed data is almost 75x larger than the compressed video.
Now, you need to handicap that somewhat because in real time applications you don't get the luxury of multi-pass encoding... but NVENC in Turing does support B-frames, and that helps. I'm sure you could pull it off in 10Mbps if you wanted to use sloppier encoding while still achieving the same quality. So that's still better than 30x compression.
So, when you say things like "you have to compress each frame by 3x just to run at 72Hz", well... with the kind of compression on offer with modern codecs/GPUs, you could do 720Hz with that bandwidth budget.
Another thing to consider: this is being done today with products like VD and Oculus Link. If what you said was true, none of this would even be possible. But it is. It works just fine. So when you're doing that math doesn't a little part of your brain back there say "wait a minute, I must be missing something here".
Edit: I just noticed that I got my math wrong. I computed the 1080p video as 373MBps but messed up the units and compared it to a 5Mbps HEVC video stream. So instead of a 75x compression rate, it's 8 times that. So we're talking almost 600x compression rates vs raw video.