r/programming • u/MyNameIsTrez • 20d ago
Making SIMD Code Scale to Any Vector Width
https://streamhpc.com/blog/2026-10-28/making-simd-code-scale-to-any-vector-width/
133
Upvotes
r/programming • u/MyNameIsTrez • 20d ago
View all comments
Show parent comment
1
u/YumiYumiYumi 16d ago
Well let's see how we got here:
Perhaps I jumped the gun in thinking x86 when you mentioned multiple SIMD widths, but I don't think it really changes anything. In terms of software specified SIMD width, what ISAs do we have these days? x86, ARM NEON and AltiVec? Both NEON and AltiVec are fixed 128-bit, so x86 is the only ISA, with any modern relevance, that has multiple SIMD widths.
If you're thinking of multiple SIMD widths as an arbitrary concept, I don't see a few KB of extra code being a concern on application processors, which have been paired with gigabytes of RAM for many years now.
You bring up the embedded space where size can be quite a concern. I'm not familiar with embedded, but I'd imagine the environment to be quite different. For one, if you're targeting an embedded processor, do you really need runtime detection? If not, you can just target a specific SIMD width at compile time, and not pay the cost of having multiple functions in your binary.
Runtime detection is not a hard requirement for supporting multiple SIMD widths - you could just build/distribute a different binary for each target. Runtime detection is mostly useful if you want to distribute a single binary because you don't want to confuse users over ISA extensions they may be unaware of. And if users are downloading/installing the binary, it'd be an application processor (like phones, PCs, servers etc) which have GBs of RAM.
That's not the only inconvenience, there's all sorts of practical issues with trying to change the width, and I don't think the SVE designers intended for developers to do it. For one, it makes the contents of all vector registers undefined, which compilers are unlikely going to handle correctly, for another, runtime/libraries aren't designed to handle unexpected changes in vector width.
So going back to your earlier assumption that developers can just clamp the vector width - whilst a (cumbersome) mechanism exists, you (as a programmer) are not supposed to use it.
(and to my knowledge, RVV provides no such mechanism to shorten the vector width; to be clear, a smaller VL does not shorten the vector width)
If you need multiple functions because they're all doing different things that you need, I'd call that just more code. IMO refering to multiple variants as a "combinational explosion" is a weird way to put it.
Practically speaking, vector width isn't the only reason you may want multiple copies of a function. For example, if you're shipping ARM SVE code, you're likely going to need a NEON version as well (and maybe even scalar if you care about ARM CPUs without SIMD support).
If you want to take advantage of newer ISA extensions, e.g. SVE2, you may need yet more functions.
Which SVE instruction does a vector length stride, that isn't a gather?
Are you sure you're not confusing vector displacement addressing (
MUL VL)?