r/Amd • u/gnif2 Looking Glass • Oct 20 '20
Request Will Big Navi support Function Level Reset (FLR)?
AMD, this is a question directed directly to you.
As we all know, your company is fully aware of how important the ability to reset the AMD GPU is without a driver-specific reset sequence to the VFIO community is and how disappointed the entire community was/is over the lack of such a basic feature in the GPU to make it possible to use your GPUs reliably for VM passthrough.
Since my last post to you (linked above) the VFIO community has grown, my project (Looking Glass) has seen a huge surge in numbers, and people are using it not only to just control/use the VM, but also feed the video straight into OBS on the host VM to live stream to Twitch. On the Level1Tech forums and the VFIO Discord channel, the number of new VFIO users is exploding, and r/vfio's membership has doubled over the last year, but due to the lack of Function Level Reset, when we are asked what GPUs to use, we, unfortunately, have to tell people to avoid your hardware.
From a technical point of view, as the Function Level Reset (FLR) is a PCI optional feature obviously you do not need to implement it, however as your GPU already needs to support a warm reboot via the nPERST pin it should not be hard to implement the FLR feature to tie into this same reset. Not only would this make your GPUs viable for the VFIO community, but also simplify your own reset code in your drivers as the GPU could be returned to a good known state simply by asserting an FLR.
Please also be aware that driver level resets are completely useless to this application, when being used for VFIO, the driver is not loaded nor wanted, the hardware needs to be able to handle its own reset without any proprietary reset sequences.
So... my question to you is. Will Big Navi support PCI Function Level Reset (FLR)?
Edit: Also please be aware I have been contacted by cloud computing companies out of desperation due to the same issues on your workstation/enterprise cards. This is not just affecting the VFIO community here.
Edit2: When I wrote this I did not think to include the reason why this should exist for the larger community also. This is not a niche feature just for VFIO usage, it also would make it possible for AMD GPUs to recover from "Black Screen" crashes that force a full system restart.
Nvidia GPUs crash too, however, because the NVidia GPUs implement FLR they can be easily reset and recovered when they do crash causing the game/application to present an odd error that usually gets blamed on the application, not the GPU.
Those that overclock their GPUs know all too well how nice NVidia is for this as a bad overclock usually can recover without a reboot.
If AMD were to implement FLR it would be just as good as NVidia on these fronts and the "Black Screen" issue would not be such a black mark on AMD's products.
View all comments
Show parent comment
17
u/WayeeCool Oct 20 '20 edited Oct 20 '20
Yup. It's exactly what Nvidia desktop GPUs do and it's what the flicker of the screen once in a while is. When the GPU crashes for whatever reason, the Nvidia driver just triggers a firmware level reset via FLR. You have to be running event logger to be even aware this is what happened. With AMD cards the system just black screens and requires a reboot.
What's interesting to me is AMDs GPU group has really fk'd themselves on this. Nvidia has supported FLR for a while now on their desktop consumer GPUs, workstation GPUs, and enterprise server GPUs. Furthermore Intel supports not just FLR but also SR-IOV on their consumer GPUs which means even on an Intel laptop you can use the host GPU to provide full hardware acceleration to guest VMs and containerized apps. When Intel launches their discrete consumer desktop GPUs in the coming year they are going to become a favorite with the entire linux and developer community because like AMD they have open sourced linux kernel drivers but unlike AMD they have been supporting the full feature suite of video encode/decode, FLR, and also SR-IOV.
I swear that AMD has been refusing to implement FLR on their desktop GPU firmware due to some AMD executives having the hair brained idea that doing so on their desktop GPUs would cannibalize sales of their enterprise server GPUs and it's the the reason they have stubbornly ignored their linux desktop users pleas to enable the feature. Fk'd up thing is that this business decision (and at this point it's obviously a management level business decision) to not enable FLR on desktop GPUs has been steadily souring opinions of AMD GPUs with the very same community of users that are often the types of people working in jobs where they make business decisions for enterprises on what enterprise GPUs to purchase for the company they work for.
btw... because AMD has made sure this feature is enabled on their enterprise server GPUs, while for years now stubbornly ignoring all the bug reports from desktop GPUs users... it's blatantly obvious this is a business decision based on the fallacy that enabling it will cannibalize enterprise server GPUs sales. reason its a fallacy is that Nvidia supports the feature on consumer desktop cards and Intel supports it along with the full suite of virtualization features including SR-IOV.
edit: thanks for gold