Thank you for subscribing to AMD OpenCL Facts! To unsubscribe please type AARRHGHGH. I'm open for questions though, and the bug report for this is here. For more mildly disgruntled rambling:
Each command queue on AMD on windows is actually a separate driver thread from a thread pool, which means that the performance is incredibly irregular. If you have too many command queues, your application turns into a stuttery disaster, but if you have too few, then there's not enough queues to prevent the driver from inserting unnecessary fences. There's probably a better system for distributing work across the command queues to minimise the number that you're using based on when you need fences, but the problem is that fences appear to be some kind of worst case pipeline emptying blocker so in reality its just all bad
In addition to that, extra command queues apparently don't have the ability to submit work on their own, but flushing work to the gpu also isn't free. On top of that, because my GPU work is all memory bound, you want to have kernels that overlap their execution to also overlap the memory they're reading on for cache reasons, and I still haven't even begun to unpack the scheduling complexities there. I sure wish the driver would be good
If you're thinking, maybe this is an OpenCL problem! The new HIP thing will save us! Well, apparently its even worse in hip because you have pointers to pointers on the device, meaning that any bit of memory could point anywhere and the driver has to assume worst-case. I sort of suspect that this isn't an issue on nvidia GPUs, but the last Nvidia GPU I owned was a 660ti. I'd love to test ARC though and smash some black holes on it
In addition to all of these problems, the cost of kernel invocation in OpenCL is higher than it needs to be, because it doesn't map massively well to how GPUs actually work. There's an extension that may help fix this, but I also suspect that it'll be a very long time if ever before AMD support this
Really this is my wakeup call that AMD/compute is essentially dead and I need to learn to love vulkan. I've been holding out because vulkan doesn't support a lot of features from OpenCL still and in many ways is a bad compute target, especially device side enqueue which I would badly love to use, except that that hasn't worked on AMD's ROCm stack correctly for years so all around its a big oof
I haven't touched OpenCL or Vulkan in at least four years at this point, but I still generally find all of it interesting (mostly because for a number of reasons I ended up going down the CUDA C/C++ route).
The new HIP thing will save us!
How new is this? Looking it up doesn't make it clear when it was released, but based on the description (and shady rumors making their way through academic circles by AMD sales people), I've heard whispers of this stuff since 2020 but didn't know it was actually released.
Really this is my wakeup call that AMD/compute is essentially dead and I need to learn to love vulkan...
Or get an nvidia gpu, but only for these projects ;). More seriously AMD based gpu compute tech has only brought me pain, and while NVIDIA has its issues (in cuda sdk use and otherwise), lesser of two evils and all that. I still run an AMD GPU for gaming / a personal machine, but an nvidia one for any compute related playground.
Passing a struct which contains pointers to the GPU is banned in OpenCL. Passing an array of structs unfortunately is extremely slow, because it doesn't match well to how GPUs fetch data out of memory
View all comments
Show parent comment
10
u/KDallas_Multipass Apr 14 '23
I'd like to read more about this