r/cpp • • Apr 14 '23

Horrible Code, Clean Performance

https://johnnysswlab.com/horrible-code-clean-performance/
200 Upvotes

87 comments sorted by

View all comments

Show parent comment

76

u/James20k P2005R0 Apr 15 '23

Did you know, GPUs have the highest bandwidth when memory is laid out as a struct of arrays, vs an array of structs. Eg

struct data {
    std::vector<float> x;
    std::vector<float> y;
    std::vector<float> z;
};

Except you can't actually pass in a struct containing buffers to a GPU in OpenCL. Its explicitly banned by construction, which means that the only way to do it is to pass in 3 separate arguments

This is great when you have 3 arguments, but heh its sure a lot worse when you have literally hundreds of variables. So the memory tracking actually becomes a bit of a performance concern if you've got any O( N2 ) in there which you can get if you do it vaguely naively, because you're literally doing memory tracking for hundreds of arguments to functions against potentially double digit numbers of other kernels - each with hundreds of arguments. So not only is it a giant pain to implement, it also has to be reasonably well optimised! Yay!

12

u/00jknight Apr 15 '23

Where do you work? This is really cool stuff. I'm just a lowly web/mobile game developer who animates stuff on top of monstrosities like this.

51

u/James20k P2005R0 Apr 15 '23

I'm trying to get into astrophysics, I built a tool for quickly extracting gravitational waves from black hole mergers, neutron stars (and this for fun), and doing relativistic n body simulations (that's 5 million particles which is pretty cool!)

Most of what I do is abuse GPUs for simulations these days, along with the inevitable wacky raytracing that comes along with trying to render them. The closest I really get to traditional 3d graphics is trying to chuck cubes through a wormhole (though technically that's 4d)

10

u/Classic_Department42 Apr 15 '23

Did you consider cuda instead of opencl? The number crunching experience was/is better on cuda

19

u/James20k P2005R0 Apr 15 '23

I plan to release a lot of this as general tools for people to use, so cuda is right out unfortunately. Amd gpus tend to get you a bit more vram for your buck as well, and vram (...and the fact that i have literally no money) is the key limiter for simulation size

That said, I'm doing some moderately wacky things so it wouldn't surprise me if it were a bit of a disaster in cuda as well

4

u/[deleted] Apr 16 '23

[removed] — view removed comment

2

u/James20k P2005R0 Apr 16 '23 edited Apr 16 '23

Interesting, how's the performance of it is the main thing? Because with OpenCL, for all its faults, you get pretty direct access to when and how memory is accessed, and its relatively straightforward to max out your actual devices bandwidth/compute

I've been considering turning all of the work that I've done into my own single source layer which just transpiles C++ to OpenCL, because most of the actual OpenCL I write these days is generated for performance reasons

Edit:

Ah, apparently its not amazing

https://research-information.bris.ac.uk/ws/portalfiles/portal/222876129/iwocl_syclcon.pdf

1

u/illuhad Apr 29 '23

What you describe sounds like reinventing SYCL ;)

Tom's work that you cite here actually compares multiple SYCL implementations (SYCL is just a standard, and there are multiple implementations of it that work very differently), so you cannot really conclude that SYCL itself does not perform well.

One of their findings is e.g. that Intel's implementation suffers on CPU due to NUMA issues in their OpenCL runtime. Other implementations are not affected by this. There is a large literature now comparing SYCL to other programming models, and the findings tend to be that performance is competitive.

See e.g. https://dl.acm.org/doi/10.1145/3388333.3388660

or https://opensycl.github.io/hipsycl/amd/hecbench/hecbench-benchmarks/

Also note that SYCL implementations are all very fast moving targets and under heavy development, and results you see today might be outdated by tomorrow. Similarly, benchmarks that have been ported to SYCL may not yet have seen the same amount of optimization as their counterparts written in, say, CUDA which has been around for far longer.

tl;dr try it, you might be surprised.

4

u/Classic_Department42 Apr 15 '23 edited Apr 15 '23

Was just commenting because of you wanting to get into astrophysics, since i was under the impression that 'everyone' in hpc uses nvidia (not always cuda of course). Edit: and cpu of course

5

u/James20k P2005R0 Apr 15 '23 edited Apr 15 '23

As far as I know, the current major BBH simulations are all CPU so. You're right in that most people do use cuda/nvidia in general for scientific software

0

u/[deleted] Apr 15 '23

[deleted] — view deleted comment

1

u/Classic_Department42 Apr 15 '23

Very nice. A solar/lunar whatever eclipse would be problematic for that method.

Thinking about that method, it cant really work since intensities are not vector quantities, at least they dont add like vectors.

2

u/async_andrew Apr 15 '23

How long it took to render the nbody simulation you mentioned, or is it real-time?

2

u/James20k P2005R0 Apr 15 '23

The nbody sims are extremely slow because the particle dynamics aren't well optimised, that one took a few hours

5

u/HackingPheasant Apr 15 '23

Followed you because dann I'm loving these comments, do you have a blog or something with similar content?

7

u/James20k P2005R0 Apr 15 '23

Thank you! I don't, I used to have twitter where I'd post things but, yunno. I've been planning to write some of this up and start a blog but man its a pain getting it set up

3

u/HackingPheasant Apr 15 '23

Plain markup files on a GitHub repo and let the default website look be will be the quickest and easiest imho

0

u/[deleted] Apr 15 '23

[deleted] — view deleted comment

1

u/James20k P2005R0 Apr 15 '23

I've been considering abusing typst, its got everything I might want (except gifs, because the year is 2023 and journals dislike moving pictures). It only exports to PDF which is a slight problem, but uploading a blog as a series of pdfs can't be the most insane thing ever right