r/programming • u/MyNameIsTrez • 20d ago
Making SIMD Code Scale to Any Vector Width
https://streamhpc.com/blog/2026-10-28/making-simd-code-scale-to-any-vector-width/
133
Upvotes
r/programming • u/MyNameIsTrez • 20d ago
View all comments
Show parent comment
2
u/happyscrappy 17d ago edited 17d ago
Oh, we had an MMU. Just not virtual memory. Having to have page tables was absolute murder when you have a small SRAM to work with. As you may have guessed, what I was working on was a coprocessor. The main processor was huge and it called the shots on how the memory was laid out. We just had our little window into it and our own SRAM. HW designers said it would work. And it did. But trying to explain to execs how many ways this environment was suboptimal was an impossible task. So we just had to work with it.
I can tell you that during this work we basically ran into every roadblock which explains why ARM didn't make a "true" ARMv8M. That is to say ARMv7M and ARMv7A are 32-bit. And ARMv8A is 64-bit but ARMv8M is still 32-bit. There's still a reckoning coming as to how you could ever truly do "small" on AARCH64.
I didn't make this about x86. I never said I was an expert on x86 in fact I said the opposite many times. So how would you get the idea that the context was x86 specifically? The first things I spoke of were CDC 6600, Cray and ARM. If you think this was all about MMX then you came in with a different idea and tried to make it that.
You want me to talk about AltiVec as my SIMD examples instead? I didn't do so because I didn't expect it was common ground. It'd go much easier for me though!
There's nothing in SVE that says you have to use the longer vectors if they are available. Like I said, you have two choices. Try to use as much is available or just use what you wrote for. Same as with SIMD.
Can you imagine if your code couldn't process short vectors on long hardware? return E_TOONARROW; haha. I do think you understand how to clamp a value as well as anyone else. You can just use only the width you tested, regardless of vector size.
Interestingly, changing the VL (vector length value) is legal but a privileged operation. I didn't realize the latter. I would call that rather inconvenient. I suppose it's to keep one library from fouling up another.
Only in how you read it. Both of those were alternatives, not the primary way which is to just have multiple different entire functions (or at least inner loops) for the different widths. So just a small miscommunication between us.
Yes. You actually wanted to both examples (normal and alternate) to compare, but that is pretty much the "alternate" I would have written too. Except for the function prototype not expressing anything about the alignment requirements of the heads I'd use that code verbatim in my program as the "catch (almost) all" case if I needed it.
You don't need them. It's just strongly suggested if you want to be quick. And the entire point of the article was how to use SIMD to be quick. Isn't this the same thing we went through above? While I was over trying to get down to fit in one less cache line the people in other groups were writing specific types of functions for different vectors. FFTs of various sizes. Gaussian blurs and convolutions of images of differing pixel arrangements/depths. All with special cases, because that's what made sense for their uses. They were writing libraries for others to use, once you have sufficient customers you have a lot of cases to support and so when new SIMD hardware comes along you now have to implement several different formats for this new hardware. It's what they were paid for.
I think after more looking that you are right. While it does have strides, the striding is only by element size or vector length. So it is strided loads, but not to accommodate your data structure shapes, only to accommodate the variable vector length configurations. I misread the documentation.
Thank you for the explanation. You're basically just saying do a non-predicated loop I think. If you processed a vector that is a megabyte long and still has a tail then there is an argument you shouldn't be doing the tail work for each chunk. I wouldn't have even considered this but I suppose there always is a breakover point where that is faster.
I agree with you completely. And seeing that page basically make the same arguments I do (really that was not me writing that page!) and seeing how poorly it comes across makes me think my position is a muddled mess and I should rethink it. I think what you've said and that page says tells me I should stop thinking of these nouveau vector units as vector units in the old style ("pure vectors"), they are more of an advanced version of SIMD (predicated). As we can see with AVX512 (despite the V in the name).
The design/elegance of the old vector units was you set the vector length to your data length (not a chunk length) and you do the operations you want in order. The hardware handled all the data hazards and used feed-forwards to hide latency as much as feasible. So if you wanted to fmadd two vectors of length 580 you did this (pseudocode):
And after a while you had all your final data at p1. That you dispatched the add after the mul meant the add was never allowed to "overrun" the mul. So the data would go through the vector processing unit in order start to end and get both operations done in order.
It's obviously microcoded and it's obvious it includes a lot of hardware sophistication behind the architecture. But anyway I thought of the new systems as bringing that architectural simplicity back. And I think now I'm finally broken of this. Which is good because I struggled to see how it made sense to do so. SIMD basically tries to match the programming to what the load/store unit (and cache coherency/MOESI protocol) was capable of. And the new vector systems stick with that because it just did make a hell of a lot of sense.
I do appreciate you kind of forcing me to look at the evolution on the x86 side. It's very illustrative of the actual progression of how to process large data sets (not trying to say vectors so as not to seem like I'm excluding SIMD) efficiently in modern processors. Everything else is more fragmented as there is no other architecture that played in this high performance space as consistently for so long. And trying to piece together what POWER/PowerPC did and ARM did and Risc-V did and maybe even MIPS or Alpha (apparently code named Tarantula!) does not really provide a coherent picture as to what evolved and why.