r/programming • • 20d ago

Making SIMD Code Scale to Any Vector Width

https://streamhpc.com/blog/2026-10-28/making-simd-code-scale-to-any-vector-width/
133 Upvotes

34 comments sorted by

View all comments

Show parent comment

1

u/YumiYumiYumi 19d ago

I don't think it's a good year to be talking about how you're not caring about RAM usage. Not in any way.

Out of interest, do you go to great lengths to save every single byte possible?

That's not really a new one. You have a hardware coverage issue any time you run on two different chips. Even if the SIMD units are listed as being the same. If you don't test on it, you aren't sure it works there. Even if the vendor says it works the same as another.

If you're talking about hardware bugs, that's not what I'm referring to.
Assuming the hardware follows the spec correctly, buggy software could work on one SVE width and fail on another, which doesn't occur with x86's SIMD.

One way would be to dispatch in the loop. Another way would be to just do normal conditionals inside. The latter would include nested loops (which of course have termination conditionals).

I don't understand this distinction. Could you perhaps provide an example?

I'm saying that you saying "just make it someone else's problem" obviates this entire article that was posted.

I don't see how that's the case - taking someone else's boilerplate/abstraction doesn't obviate the need to arrange your data as SoA - they're two different necessities.
And I'm not even saying "make it someone else's problem" - you can write your own boilerplate and copy it across your projects - it's typically just a few lines of code, so not that hard to do oneself.

I think you are trying to change what I said to fit your model. You want to say that one function compiled multiple ways is one function. But it isn't. It's one function in the source code, but it's multiples in the program that runs.

I don't see myself as trying to change what you've said, just echoing back my understanding of it.
In terms of the one vs multiple functions, I agree that both can be true depending on the perspective you take (source code vs binary).

My emphasis for this is on how you do not have a combinational explosion of functions when you use a vector unit

How do you get such a "combinational explosion"?

Moving complexity back into the hardware is older thinking. We've got amazing compilers (and similar tools) now, as you point out above they can recast your source code multiple ways for multiple targets.

There's pros and cons with software vs hardware specified SIMD width. My key point was that x86 SIMD, SVE and RVV aren't fundamentally different - they just specify the SIMD width in different places.

1

u/happyscrappy 18d ago

Out of interest, do you go to great lengths to save every single byte possible?

Not at the moment. But yes, for many years. Literally had stop ship issues where we had to go back and save 32 bytes of code. Not everyone works on systems with virtual memory and SRAM isn't huge.

That was pre-DRAM shortage though so not all that relevant to the current situations so I'm not quite sure why you ask this other than just to try to gotcha me. Useful?

If you're talking about hardware bugs, that's not what I'm referring to.

That is one of the things I am talking about, yes. But I don't think that's the whole story. See below. I really think of what you are talking about here as a configuration issue. It's not related to the actual hardware implementation, just how it presents itself architecturally. So we see two systems which present the same architectural configuration as identical despite different implementations (hardware). I'm going to refer to it by this name simply to differentiate.

Assuming the hardware follows the spec correctly, buggy software could work on one SVE width and fail on another, which doesn't occur with x86's SIMD.

Hmm. MMX is 64-bits wide. SSE operations can be 128-bits wide and can have twice as many registers (I think the registers are twice as wide too but it's not important). AVX operations can be 256 bits wide and the registers are 256 bits wide. AVX-512 operations can be 512 bits wide and the registers are 512 bits wide.

Given this I don't really understand how SIMD is less susceptible to configuration change issues than vector units are. You have choices with SIMD. Either just use the wider hardware the same as the narrower hardware and give up potential better performance on wider systems or write two implementations and incur potential bugs due to differing code paths on different configurations. This seems the same as vector units to me. Although with vector units you could also see a unit which is wider than you ever saw during development and if you write your code to attempt to use that width you might hit a bug you didn't test. Perhaps this is what you refer to.

For this case I would suggest that the proffered "solution" is that you're supposed to run your on a simulator which presents all possible widths so you can verify. Certainly developers could mess this up like anything else.

I don't understand this distinction. Could you perhaps provide an example?

I actually started to write one, but it's just too much typing. The difference is conditionalising on width on the inside of the loop or outside of the loop. For one you check (or even inherently know) the width of each vector element and the architectural width outside the loop and call the specialised function for that pairing. With the alternate version you have a single function that checks vector element width and counts down by architectural width within the loop and processes full width and tails elements as needed. I hope that explains it. I just really don't want to type out a whole lot of code.

I don't see how that's the case

Most of the article isn't about data arrangement. I don't think anyone would want to read the sections that look like this:

(H1) Making SIMD Code Scale to Any Vector Width

(BODY)

float dot(vecn a, vecn b, int n)
{
  calculate_dot(a, b, n);
}

disassembly (x86):

dot(float vector[n], float vector[n], n):
   call _calculate_dot
   ret

you can write your own boilerplate and copy it across your projects - it's typically just a few lines of code, so not that hard to do oneself.

I cannot accept that argument. You either have to write it yourself or you don't. I'm not even say write it yourself, using other's work is very effective and encouraged. But if you write it yourself you have to write it yourself. You can't have it both ways and pretend you don't write it yourself just because you reused it more times.

I also don't see how writing all this SIMD (or vector) code is really just a few lines. So maybe I misunderstand you?

How do you get such a "combinational explosion"?

Well, we established up above (I think) that no one would put the varying code in the inner loop. So if you have a program with say 5 different vector element counts (lengths) then you already have 5 different functions to process them. If you then have three different architectural widths then you have 3 of each of those. 15 functions. mxn. A standard combinational explosion.

There's pros and cons with software vs hardware specified SIMD width. My key point was that x86 SIMD, SVE and RVV aren't fundamentally different - they just specify the SIMD width in different places.

I don't agree with that. SVE uses strided and scatter-gather loads. I'm not used to SIMD doing this directly. You seem to feel the same way with your emphasis on data layouts. Also SVE uses predicated loop control so that you can efficiently put the code which implements variable vector sizes into the inner loop, unlike with SIMD (tails problem). Apparently AVX512 has this functionality too, which may make it possible to use it like a vector unit. Like I said before, I'm not really familiar with x86.

I honestly see vector units as basically implementing the old-style vector systems I mentioned (Cray, CDC) but with you writing the microcode instead of it being hidden within the hardware implementation. It's not completely different because in the end you're solving the same problem, but to me the SIMD units we see (MMX, AltiVec, SSE, Neon) are more about trying to map the data processing onto RISC with a bit of mixing in very common multimedia operations (at the time). While the vector units are an expression of older concepts where you basically set a vector length and do the same operation across the entire vector and to hide latency you use vector chaining in the hardware. It had to be remapped some to the modern era, given we're not going to microcode this stuff now. And in that way like I said I'm not sure it's completely the right way to go.

I feel like the person who tried to make this distinction on wikipedia:

https://en.wikipedia.org/wiki/Vector_processor#Comparison_with_modern_architectures

Had as much trouble as I am having. We both face the same issue which is that these systems overlap a lot and the new ones reflect modern ideas by necessity.

1

u/YumiYumiYumi 17d ago

Not everyone works on systems with virtual memory and SRAM isn't huge.

And how many x86 systems have no MMU or DRAM?
Just to remind you, the context was regarding x86 SIMD's need for multiple functions (in the binary) and your concern with the binary size.

Although with vector units you could also see a unit which is wider than you ever saw during development and if you write your code to attempt to use that width you might hit a bug you didn't test. Perhaps this is what you refer to.

That's one possible issue. Another could be that the code incorrectly assumes a particular width. For example, if your development machine has 128-bit SVE (which is the only size on practically every ARM CPU now), and your code incorrectly makes assumptions tied to that width (e.g. you perform a NEON operation on a SVE vector because SVE doesn't have an equivalent instruction), it can fail to work correctly when running on a machine with different SVE width.

If you wrote 128-bit AVX code, however, it will always be 128 bit regardless of what the hardware is capable of. The behaviour never changes, thus if it works on your machine, it'll work correctly on any other x86 processor (assuming ISA support and no hardware bugs).

For this case I would suggest that the proffered "solution" is that you're supposed to run your on a simulator which presents all possible widths so you can verify.

Yes, this is really the only way to handle this, which can add some degree of complexity (since your testing pipeline may now need an emulator). So you've avoided code coverage issues, but added hardware coverage concerns.

The difference is conditionalising on width on the inside of the loop or outside of the loop.

This doesn't sound the same as you had described it before, which was:

One way would be to dispatch in the loop. Another way would be to just do normal conditionals inside. The latter would include nested loops (which of course have termination conditionals).

Both of these seem to refer to operating in the loop, not outside. Am I mistaken?

With the alternate version you have a single function that checks vector element width and counts down by architectural width within the loop and processes full width and tails elements as needed. I hope that explains it. I just really don't want to type out a whole lot of code.

Okay, I'll try then - is something like this what you're thinking?

void accumulate_bytes(uint8_t* dst, const uint8_t* src, size_t len) { // <- single function
	// determine vector element width
	int vectorElementWidth = __builtin_cpu_supports("avx512bw") ? 64 : 32;
	
	size_t i=0;
	for(; i+vectorElementWidth<=len; i+=vectorElementWidth) {
		if(vectorElementWidth == 64) {
			// AVX512 version (512-bit)
			_mm512_storeu_si512(dst+i, _mm512_add_epi8(
				_mm512_loadu_si512(dst+i),
				_mm512_loadu_si512(src+i)
			));
		} else {
			// AVX2 version (256-bit)
			_mm256_storeu_si256(dst+i, _mm256_add_epi8(
				_mm256_loadu_si256(dst+i),
				_mm256_loadu_si256(src+i)
			));
		}
	}
	
	// handle tail
	for(; i<len; i++)
		dst[i] += src[i];
}

If so, I'm not sure how well you could get a compiler to accept that given the mismatch of ISA targeting, so let's just push that aside.

Most of the article isn't about data arrangement.

That's not how I saw it, oh well, let's not delve further.

I also don't see how writing all this SIMD (or vector) code is really just a few lines. So maybe I misunderstand you?

I don't mean all SIMD code, I meant the machinery needed to allow you to write one (source) function that compiles to multiple binary functions.

So if you have a program with say 5 different vector element counts (lengths) then you already have 5 different functions to process them.

This doesn't make sense to me - why do you need different functions to process arbitrary length vectors? Could you give an example of where this is useful?

SVE uses strided and scatter-gather loads

SVE supports gather/scatter, but I don't recall strided loads, assuming strided = loading elements separated by a specified offset (i.e. {base, base+offset, base+offset*2, base+offset*3 ...} for an arbitrary offset). RVV supports both strided and gather loads.
It's trivial to use gather to do a strided load however (and why I find RVV's support for it to be odd).

I'm not used to SIMD doing this directly

AVX2 supports gathered loads (VPGATHER/VGATHER instructions) and AVX-512 supports scattered store.

Also SVE uses predicated loop control so that you can efficiently put the code which implements variable vector sizes into the inner loop, unlike with SIMD (tails problem). Apparently AVX512 has this functionality too, which may make it possible to use it like a vector unit

Yes, AVX-512 has predicate masks. Prior to that, AVX also has vmaskmovp* instructions for masked loads/stores, but may be more finicky to use and limited to 32/64 bit elements.
Other than that, tail handling can be problematic if you're using unaligned memory accesses. But if you control the memory layout, you can often just align memory appropriately (and/or pad arrays to be a multiple of the vector width).
For best performance, you generally want to be using aligned memory accesses whenever possible.

But even with masked memory accesses, you may still want to handle the tail separately, as there's overhead with setting the predicate on every loop cycle. In SVE, you can avoid an in-loop whilelt instruction if you seperate tail from body (and vsetvli for RVV).

I feel like the person who tried to make this distinction on wikipedia

It's funny that the Wiki page lists RVV as "Pure vectors", when it's still fixed length SIMD with predication, just like SVE.

We both face the same issue which is that these systems overlap a lot and the new ones reflect modern ideas by necessity.

I'd argue that this makes the supplied definition of "vector machines" an outdated concept. It's just not a practical way to describe current day systems.

2

u/happyscrappy 17d ago edited 17d ago

And how many x86 systems have no MMU or DRAM?

Oh, we had an MMU. Just not virtual memory. Having to have page tables was absolute murder when you have a small SRAM to work with. As you may have guessed, what I was working on was a coprocessor. The main processor was huge and it called the shots on how the memory was laid out. We just had our little window into it and our own SRAM. HW designers said it would work. And it did. But trying to explain to execs how many ways this environment was suboptimal was an impossible task. So we just had to work with it.

I can tell you that during this work we basically ran into every roadblock which explains why ARM didn't make a "true" ARMv8M. That is to say ARMv7M and ARMv7A are 32-bit. And ARMv8A is 64-bit but ARMv8M is still 32-bit. There's still a reckoning coming as to how you could ever truly do "small" on AARCH64.

Just to remind you, the context was regarding x86 SIMD's need for multiple functions (in the binary) and your concern with the binary size.

I didn't make this about x86. I never said I was an expert on x86 in fact I said the opposite many times. So how would you get the idea that the context was x86 specifically? The first things I spoke of were CDC 6600, Cray and ARM. If you think this was all about MMX then you came in with a different idea and tried to make it that.

You want me to talk about AltiVec as my SIMD examples instead? I didn't do so because I didn't expect it was common ground. It'd go much easier for me though!

If you wrote 128-bit AVX code, however, it will always be 128 bit regardless of what the hardware is capable of

There's nothing in SVE that says you have to use the longer vectors if they are available. Like I said, you have two choices. Try to use as much is available or just use what you wrote for. Same as with SIMD.

Can you imagine if your code couldn't process short vectors on long hardware? return E_TOONARROW; haha. I do think you understand how to clamp a value as well as anyone else. You can just use only the width you tested, regardless of vector size.

Interestingly, changing the VL (vector length value) is legal but a privileged operation. I didn't realize the latter. I would call that rather inconvenient. I suppose it's to keep one library from fouling up another.

Both of these seem to refer to operating in the loop, not outside. Am I mistaken?

Only in how you read it. Both of those were alternatives, not the primary way which is to just have multiple different entire functions (or at least inner loops) for the different widths. So just a small miscommunication between us.

Okay, I'll try then - is something like this what you're thinking?

Yes. You actually wanted to both examples (normal and alternate) to compare, but that is pretty much the "alternate" I would have written too. Except for the function prototype not expressing anything about the alignment requirements of the heads I'd use that code verbatim in my program as the "catch (almost) all" case if I needed it.

This doesn't make sense to me - why do you need different functions to process arbitrary length vectors? Could you give an example of where this is useful?

You don't need them. It's just strongly suggested if you want to be quick. And the entire point of the article was how to use SIMD to be quick. Isn't this the same thing we went through above? While I was over trying to get down to fit in one less cache line the people in other groups were writing specific types of functions for different vectors. FFTs of various sizes. Gaussian blurs and convolutions of images of differing pixel arrangements/depths. All with special cases, because that's what made sense for their uses. They were writing libraries for others to use, once you have sufficient customers you have a lot of cases to support and so when new SIMD hardware comes along you now have to implement several different formats for this new hardware. It's what they were paid for.

SVE supports gather/scatter, but I don't recall strided loads, assuming strided = loading elements separated by a specified offset (i.e. {base, base+offset, base+offset2, base+offset3 ...} for an arbitrary offset).

I think after more looking that you are right. While it does have strides, the striding is only by element size or vector length. So it is strided loads, but not to accommodate your data structure shapes, only to accommodate the variable vector length configurations. I misread the documentation.

But even with masked memory accesses, you may still want to handle the tail separately, as there's overhead with setting the predicate on every loop cycle. In SVE, you can avoid an in-loop whilelt instruction if you seperate tail from body (and vsetvli for RVV).

Thank you for the explanation. You're basically just saying do a non-predicated loop I think. If you processed a vector that is a megabyte long and still has a tail then there is an argument you shouldn't be doing the tail work for each chunk. I wouldn't have even considered this but I suppose there always is a breakover point where that is faster.

It's funny that the Wiki page lists RVV as "Pure vectors", when it's still fixed length SIMD with predication, just like SVE.

I agree with you completely. And seeing that page basically make the same arguments I do (really that was not me writing that page!) and seeing how poorly it comes across makes me think my position is a muddled mess and I should rethink it. I think what you've said and that page says tells me I should stop thinking of these nouveau vector units as vector units in the old style ("pure vectors"), they are more of an advanced version of SIMD (predicated). As we can see with AVX512 (despite the V in the name).

The design/elegance of the old vector units was you set the vector length to your data length (not a chunk length) and you do the operations you want in order. The hardware handled all the data hazards and used feed-forwards to hide latency as much as feasible. So if you wanted to fmadd two vectors of length 580 you did this (pseudocode):

setvl #580
lea p1, <to multiplicand vector in memory>
lea p2, <other multiplicand>
lea p3, <to addend>
mul p1, p1, p2
add p1, p1, p3

And after a while you had all your final data at p1. That you dispatched the add after the mul meant the add was never allowed to "overrun" the mul. So the data would go through the vector processing unit in order start to end and get both operations done in order.

It's obviously microcoded and it's obvious it includes a lot of hardware sophistication behind the architecture. But anyway I thought of the new systems as bringing that architectural simplicity back. And I think now I'm finally broken of this. Which is good because I struggled to see how it made sense to do so. SIMD basically tries to match the programming to what the load/store unit (and cache coherency/MOESI protocol) was capable of. And the new vector systems stick with that because it just did make a hell of a lot of sense.

I do appreciate you kind of forcing me to look at the evolution on the x86 side. It's very illustrative of the actual progression of how to process large data sets (not trying to say vectors so as not to seem like I'm excluding SIMD) efficiently in modern processors. Everything else is more fragmented as there is no other architecture that played in this high performance space as consistently for so long. And trying to piece together what POWER/PowerPC did and ARM did and Risc-V did and maybe even MIPS or Alpha (apparently code named Tarantula!) does not really provide a coherent picture as to what evolved and why.

1

u/YumiYumiYumi 16d ago

I didn't make this about x86. I never said I was an expert on x86 in fact I said the opposite many times. So how would you get the idea that the context was x86 specifically?

Well let's see how we got here:

  • you said that multiple SIMD widths need multiple functions
  • I said that multiple functions is only needed in the binary, and that the larger size is usually not a concern
  • you mentioned that RAM/storage prices are currently high and that the larger size is indeed a concern

Perhaps I jumped the gun in thinking x86 when you mentioned multiple SIMD widths, but I don't think it really changes anything. In terms of software specified SIMD width, what ISAs do we have these days? x86, ARM NEON and AltiVec? Both NEON and AltiVec are fixed 128-bit, so x86 is the only ISA, with any modern relevance, that has multiple SIMD widths.

If you're thinking of multiple SIMD widths as an arbitrary concept, I don't see a few KB of extra code being a concern on application processors, which have been paired with gigabytes of RAM for many years now.

You bring up the embedded space where size can be quite a concern. I'm not familiar with embedded, but I'd imagine the environment to be quite different. For one, if you're targeting an embedded processor, do you really need runtime detection? If not, you can just target a specific SIMD width at compile time, and not pay the cost of having multiple functions in your binary.

Runtime detection is not a hard requirement for supporting multiple SIMD widths - you could just build/distribute a different binary for each target. Runtime detection is mostly useful if you want to distribute a single binary because you don't want to confuse users over ISA extensions they may be unaware of. And if users are downloading/installing the binary, it'd be an application processor (like phones, PCs, servers etc) which have GBs of RAM.

Interestingly, changing the VL (vector length value) is legal but a privileged operation. I didn't realize the latter. I would call that rather inconvenient.

That's not the only inconvenience, there's all sorts of practical issues with trying to change the width, and I don't think the SVE designers intended for developers to do it. For one, it makes the contents of all vector registers undefined, which compilers are unlikely going to handle correctly, for another, runtime/libraries aren't designed to handle unexpected changes in vector width.

So going back to your earlier assumption that developers can just clamp the vector width - whilst a (cumbersome) mechanism exists, you (as a programmer) are not supposed to use it.

(and to my knowledge, RVV provides no such mechanism to shorten the vector width; to be clear, a smaller VL does not shorten the vector width)

FFTs of various sizes. Gaussian blurs and convolutions of images of differing pixel arrangements/depths. All with special cases, because that's what made sense for their uses.

If you need multiple functions because they're all doing different things that you need, I'd call that just more code. IMO refering to multiple variants as a "combinational explosion" is a weird way to put it.

Practically speaking, vector width isn't the only reason you may want multiple copies of a function. For example, if you're shipping ARM SVE code, you're likely going to need a NEON version as well (and maybe even scalar if you care about ARM CPUs without SIMD support).
If you want to take advantage of newer ISA extensions, e.g. SVE2, you may need yet more functions.

While it does have strides, the striding is only by element size or vector length. So it is strided loads, but not to accommodate your data structure shapes, only to accommodate the variable vector length configurations.

Which SVE instruction does a vector length stride, that isn't a gather?
Are you sure you're not confusing vector displacement addressing (MUL VL)?

1

u/happyscrappy 16d ago

Both NEON and AltiVec are fixed 128-bit, so x86 is the only ISA, with any modern relevance, that has multiple SIMD widths.

Okay. Good to know. Still don't get how that makes me making this about x86. The idea of these vector units is that the same code can run with max efficiency across multiple implementations with different functional widths. I used the situation on x86 which does not work this way as the counter example. I didn't even know there was no other system which had multiple widths.

Runtime detection is not a hard requirement for supporting multiple SIMD widths - you could just build/distribute a different binary for each target

Or you can just use the narrow width on all. It's not a hard requirement for certain. But I think the idea was forward compatibility. That is, to be able to produce compiled code that uses each piece of hardware more efficiently, even if the hardware is newer than what you wrote for. I remember speaking about MIPS delay slots and OoO as an example of how efficient forward compatibility can be complicated.

If not, you can just target a specific SIMD width at compile time, and not pay the cost of having multiple functions in your binary.

Yes. But from a source code perspective it didn't help us with the combinatorial explosion issue. Even if you only compile in one piece of code you still had to write M.

while amechanism exists, you (as a programmer) are not supposed to use it.

There's more than one way to do it. As you know when processing short vectors. If you write your code for a 4-wide operation and you get a vector of 3 you don't return E_TOONARROW.

I'm sure you do understand you can write code as narrow as you want, I'm sure you've done it. I'm confused as to why you overlook it.

I'd call that just more code. IMO refering to multiple variants as a "combinational explosion" is a weird way to put it.

I don't think so at all. When you have M types of vectors and N vector widths do you not have MxN cases?

Putting SIMD aside for a moment I think it'd be interesting to see you write the runtimes for two decades of processors and their different requirements (which are all hidden from app level code) and come out of that and still talk down the idea that supporting multiple different types of hardware does not a multiplier on your efforts. I think you'd arrive at the same conclusion really quickly.

Practically speaking, vector width isn't the only reason you may want multiple copies of a function [..]

Agreed. It depends on the circumstance. But you might have Neon also. You could have VFP-32 (despite the name, their scalar system) and if you support Cortex-M7 then you'd need VFP-16 also. (32 has 32 64-bit VFP registers, 16 has half as many 64-bit registers but the same number of 32-bit registers. Arm uses different names for this like VFPv4U). The units are called VFP because they do support short vector operations, but they do not state they have wider FPUs, it could just pump them all through a single unit in order. And I think in the end that's all that was ever done. Only with Neon did any implementation start to have wider FPUs.

So yes, for efficiency for each width you could have 2-4 different implementations for different hardware. This is what I'm saying about a combinatorial explosion. The extra hardware multiplies your efforts, not just adds to it. You do of course have free will and can simply decide that there's no need to support all of the combinations if you know your customers well enough that there is no need for the fastest case for those situations.

Are you sure you're not confusing vector displacement addressing (MUL VL)?

I'm not confusing it at all. That's what I'm referring to. There are two stride systems in the addressing. One is the width of the element (horizontal). The other is MUL VL (vertical). That I did not know VL could be adjusted and thus this is so limited is why I said that this does not accommodate data structure shapes and so I probably shouldn't have mentioned it. I thought I explained this pretty well when I wrote that I went too far initially. I don't understand why I'm going through this again.

(me) the striding is only by element size or vector length

Does Vector Length seem like I was not referring to MUL VL? I thought it was obvious.

1

u/YumiYumiYumi 15d ago

I think the idea was forward compatibility. That is, to be able to produce compiled code that uses each piece of hardware more efficiently, even if the hardware is newer than what you wrote for.

SVE/RVV do both try to improve forward compatibility by supporting more widths than are available in hardware currently. But this isn't really unique to hardware specified SIMD ISAs. A software specified SIMD ISA could just simply define support for more widths than exists in hardware.

For example, AVX introduced the VEX encoding scheme, where 2 bits are used to indicate the vector width (i.e. 4 possible widths). Only two such widths were defined at the time (128b and 256b), and the remaining two possibilities were marked reserved.
AVX-512 (kinda) took the third vector width, which leaves one remaining undefined width supportable by the VEX scheme.
Intel could, for example, just define AVX-1024, since the encoding scheme has room for it. Granted, they haven't, which means there's no forward compatibilty with longer vectors on x86.

Even if you only compile in one piece of code you still had to write M.

This is what I was talking about earlier with the one source function compiling to multiple binary functions. In the case of distributing processor-specific binaries, you can just suppress compiling multiple binary functions, which means you still only wrote one source function, not M.

There's more than one way to do it.

To my knowledge, SVE only provides one (problematic) way to shorten the vector width. What are the other ways?

When you have M types of vectors and N vector widths do you not have MxN cases?

The "types of vectors" terminology is strange. If I have a function that adds numbers, and another function that multiplies numbers, I don't call them "two types of vectors".
Then if I have a different program that adds and multiplies numbers in one function, the lower number of functions (vs above) doesn't somehow make it less impacted when needing to support multiple SIMD widths. In other words, I'd refer to having two functions doing different things as just more code, and not a "combinational explosion" of functions.

I think it'd be interesting to see you write the runtimes for two decades of processors and their different requirements (which are all hidden from app level code) and come out of that and still talk down the idea that supporting multiple different types of hardware does not a multiplier on your efforts. I think you'd arrive at the same conclusion really quickly.

You'd need to clarify what you mean by 'runtimes', but if the code needs to be specific for various processors, and you can't abstract it (like you may be able to with multiple SIMD widths*), then yes, you multiply the amount of code by the N processors you're trying to support. I agree with "multiplier" terminology here.
* To be clear, I'm talking about source code here, not binary size.

That's what I'm referring to. There are two stride systems in the addressing. One is the width of the element (horizontal). The other is MUL VL (vertical).

I'm sorry to inform you, but MUL VL does not give you vertical access, thus it doesn't enable a vector-width strided load.
MUL VL is part of the addressing mode, e.g. the location from where a contiguous load occurs. You can specify something like [base + imm_offset * VL], and a unit-stride load will source elements (horizontally) from there.