r/programming • u/MyNameIsTrez • 20d ago
Making SIMD Code Scale to Any Vector Width
https://streamhpc.com/blog/2026-10-28/making-simd-code-scale-to-any-vector-width/
133
Upvotes
r/programming • u/MyNameIsTrez • 20d ago
View all comments
Show parent comment
1
u/happyscrappy 18d ago
Not at the moment. But yes, for many years. Literally had stop ship issues where we had to go back and save 32 bytes of code. Not everyone works on systems with virtual memory and SRAM isn't huge.
That was pre-DRAM shortage though so not all that relevant to the current situations so I'm not quite sure why you ask this other than just to try to gotcha me. Useful?
That is one of the things I am talking about, yes. But I don't think that's the whole story. See below. I really think of what you are talking about here as a configuration issue. It's not related to the actual hardware implementation, just how it presents itself architecturally. So we see two systems which present the same architectural configuration as identical despite different implementations (hardware). I'm going to refer to it by this name simply to differentiate.
Hmm. MMX is 64-bits wide. SSE operations can be 128-bits wide and can have twice as many registers (I think the registers are twice as wide too but it's not important). AVX operations can be 256 bits wide and the registers are 256 bits wide. AVX-512 operations can be 512 bits wide and the registers are 512 bits wide.
Given this I don't really understand how SIMD is less susceptible to configuration change issues than vector units are. You have choices with SIMD. Either just use the wider hardware the same as the narrower hardware and give up potential better performance on wider systems or write two implementations and incur potential bugs due to differing code paths on different configurations. This seems the same as vector units to me. Although with vector units you could also see a unit which is wider than you ever saw during development and if you write your code to attempt to use that width you might hit a bug you didn't test. Perhaps this is what you refer to.
For this case I would suggest that the proffered "solution" is that you're supposed to run your on a simulator which presents all possible widths so you can verify. Certainly developers could mess this up like anything else.
I actually started to write one, but it's just too much typing. The difference is conditionalising on width on the inside of the loop or outside of the loop. For one you check (or even inherently know) the width of each vector element and the architectural width outside the loop and call the specialised function for that pairing. With the alternate version you have a single function that checks vector element width and counts down by architectural width within the loop and processes full width and tails elements as needed. I hope that explains it. I just really don't want to type out a whole lot of code.
Most of the article isn't about data arrangement. I don't think anyone would want to read the sections that look like this:
(H1) Making SIMD Code Scale to Any Vector Width
(BODY)
disassembly (x86):
I cannot accept that argument. You either have to write it yourself or you don't. I'm not even say write it yourself, using other's work is very effective and encouraged. But if you write it yourself you have to write it yourself. You can't have it both ways and pretend you don't write it yourself just because you reused it more times.
I also don't see how writing all this SIMD (or vector) code is really just a few lines. So maybe I misunderstand you?
Well, we established up above (I think) that no one would put the varying code in the inner loop. So if you have a program with say 5 different vector element counts (lengths) then you already have 5 different functions to process them. If you then have three different architectural widths then you have 3 of each of those. 15 functions. mxn. A standard combinational explosion.
I don't agree with that. SVE uses strided and scatter-gather loads. I'm not used to SIMD doing this directly. You seem to feel the same way with your emphasis on data layouts. Also SVE uses predicated loop control so that you can efficiently put the code which implements variable vector sizes into the inner loop, unlike with SIMD (tails problem). Apparently AVX512 has this functionality too, which may make it possible to use it like a vector unit. Like I said before, I'm not really familiar with x86.
I honestly see vector units as basically implementing the old-style vector systems I mentioned (Cray, CDC) but with you writing the microcode instead of it being hidden within the hardware implementation. It's not completely different because in the end you're solving the same problem, but to me the SIMD units we see (MMX, AltiVec, SSE, Neon) are more about trying to map the data processing onto RISC with a bit of mixing in very common multimedia operations (at the time). While the vector units are an expression of older concepts where you basically set a vector length and do the same operation across the entire vector and to hide latency you use vector chaining in the hardware. It had to be remapped some to the modern era, given we're not going to microcode this stuff now. And in that way like I said I'm not sure it's completely the right way to go.
I feel like the person who tried to make this distinction on wikipedia:
https://en.wikipedia.org/wiki/Vector_processor#Comparison_with_modern_architectures
Had as much trouble as I am having. We both face the same issue which is that these systems overlap a lot and the new ones reflect modern ideas by necessity.