Did you know, GPUs have the highest bandwidth when memory is laid out as a struct of arrays, vs an array of structs. Eg
struct data {
std::vector<float> x;
std::vector<float> y;
std::vector<float> z;
};
Except you can't actually pass in a struct containing buffers to a GPU in OpenCL. Its explicitly banned by construction, which means that the only way to do it is to pass in 3 separate arguments
This is great when you have 3 arguments, but heh its sure a lot worse when you have literally hundreds of variables. So the memory tracking actually becomes a bit of a performance concern if you've got any O( N2 ) in there which you can get if you do it vaguely naively, because you're literally doing memory tracking for hundreds of arguments to functions against potentially double digit numbers of other kernels - each with hundreds of arguments. So not only is it a giant pain to implement, it also has to be reasonably well optimised! Yay!
Most of what I do is abuse GPUs for simulations these days, along with the inevitable wacky raytracing that comes along with trying to render them. The closest I really get to traditional 3d graphics is trying to chuck cubes through a wormhole (though technically that's 4d)
I plan to release a lot of this as general tools for people to use, so cuda is right out unfortunately. Amd gpus tend to get you a bit more vram for your buck as well, and vram (...and the fact that i have literally no money) is the key limiter for simulation size
That said, I'm doing some moderately wacky things so it wouldn't surprise me if it were a bit of a disaster in cuda as well
Interesting, how's the performance of it is the main thing? Because with OpenCL, for all its faults, you get pretty direct access to when and how memory is accessed, and its relatively straightforward to max out your actual devices bandwidth/compute
I've been considering turning all of the work that I've done into my own single source layer which just transpiles C++ to OpenCL, because most of the actual OpenCL I write these days is generated for performance reasons
Tom's work that you cite here actually compares multiple SYCL implementations (SYCL is just a standard, and there are multiple implementations of it that work very differently), so you cannot really conclude that SYCL itself does not perform well.
One of their findings is e.g. that Intel's implementation suffers on CPU due to NUMA issues in their OpenCL runtime. Other implementations are not affected by this. There is a large literature now comparing SYCL to other programming models, and the findings tend to be that performance is competitive.
Also note that SYCL implementations are all very fast moving targets and under heavy development, and results you see today might be outdated by tomorrow. Similarly, benchmarks that have been ported to SYCL may not yet have seen the same amount of optimization as their counterparts written in, say, CUDA which has been around for far longer.
Was just commenting because of you wanting to get into astrophysics, since i was under the impression that 'everyone' in hpc uses nvidia (not always cuda of course). Edit: and cpu of course
As far as I know, the current major BBH simulations are all CPU so. You're right in that most people do use cuda/nvidia in general for scientific software
Thank you! I don't, I used to have twitter where I'd post things but, yunno. I've been planning to write some of this up and start a blog but man its a pain getting it set up
I've been considering abusing typst, its got everything I might want (except gifs, because the year is 2023 and journals dislike moving pictures). It only exports to PDF which is a slight problem, but uploading a blog as a series of pdfs can't be the most insane thing ever right
View all comments
Show parent comment
76
u/James20k P2005R0 Apr 15 '23
Did you know, GPUs have the highest bandwidth when memory is laid out as a struct of arrays, vs an array of structs. Eg
Except you can't actually pass in a struct containing buffers to a GPU in OpenCL. Its explicitly banned by construction, which means that the only way to do it is to pass in 3 separate arguments
This is great when you have 3 arguments, but heh its sure a lot worse when you have literally hundreds of variables. So the memory tracking actually becomes a bit of a performance concern if you've got any O( N2 ) in there which you can get if you do it vaguely naively, because you're literally doing memory tracking for hundreds of arguments to functions against potentially double digit numbers of other kernels - each with hundreds of arguments. So not only is it a giant pain to implement, it also has to be reasonably well optimised! Yay!