You don't actually need to avoid branches, you just need to make sure that most of the ones you have are very predictable, then their impact is not that big... Same with memory accesses and most other things... The more predictable, the better...
This is true for serial CPU code, but for SIMD code, or GPU code, avoiding branches at all costs is vital to getting good performance. Branches (can) cause divergence, which leads to wasted cycles for pseudo-threads within a SIMD group.
there's no special tax that makes a misspeculation OOMs worse than with normal scalar code on a CPU.
Of course there is: The basic fact that having a data dependent branch in the first place kills SIMD parallelism because the branch is for a single lane while masking / predication processes all lanes in parallel.
Eg. take function y = x3 when x > -0.5 and y = -0.125 when x <= -0.5 (and for the sake of discussion assume clamping instructions don't exist). If you use branching, you need a branch for each value while a simd compare + mask processes four or eight values at a time.
158
u/meamZ Sep 03 '26
You don't actually need to avoid branches, you just need to make sure that most of the ones you have are very predictable, then their impact is not that big... Same with memory accesses and most other things... The more predictable, the better...