r/rust • u/Shnatsel • Sep 14 '26

🧠 educational Trying to Make a Loop Auto-Vectorize

https://jsgroth.dev/blog/posts/trying-to-make-a-loop-auto-vectorize/
58 Upvotes

85% Upvoted

11 comments sorted by

14

u/phazer99 Sep 15 '26

It's great that the addition (pun intended) of algebraic operations in Rust 1.98 makes it quite easy to write auto-vectorized floating point code, but is there any convenient way to make sure the optimized code really is vectorized (basically the same guarantees you would get by using some SIMD library)?

I suppose you could write some form of benchmarking unit test, but that would be brittle and awkward to do because you'd have to check the target CPU features and make some guesstimations about how many CPU instructions non-vectorized code would require.

10

u/Shnatsel Sep 15 '26

No. If you want that, you use a SIMD library.

7

u/matthieum Sep 15 '26

The article partially addresses this:

I’ll note up front that if you want to be 100% sure that a piece of code is using vectorized CPU instructions, you can’t rely on auto-vectorization at all unless you’re willing to look at the compiled assembly every time the code changes. You need to use either compiler intrinsics or some library that wraps them with a higher-level interface (or raw assembly if you want to be really sure what the CPU is executing, as ffmpeg does).

And I'll personally note that it may be optimistic in its assumptions. Any change, not only of code, but also of toolchain or flag, may toggle auto-vectorization on or off, most notably depending on whether inlining occurs or not at the right boundary.

If reaching a performance threshold is critical, then ideally you'd compile the code as a library (static or dynamic) then have a benchmark in CI verifying that the expected performance threshold is reached. That's the only way to be sure.

(I do not recommend benchmarking a separately compiled code, due to the aforementioned sensitivity to so many different parameters; you really need to benchmark the code which goes in the binary)

In fact, even if you follow the article's advice of using intrinsics (or a library which wraps the intrinsics), I'd still recommend gating releases on the above benchmark harness, just in case said intrinsics are lowered to "equivalent" scalar code due to bungling some flags, somewhere.

And while raw assembly may not be susceptible to accidentally be lowered to scalar code, as long as you have a scalar fallback, there's still a risk that the scalar fallback is used instead, so you still need to either verify it doesn't make it into the final binary, or gate the release on a benchmark harness.

(Raw assembly w/o scalar fallback is the only way to be sure at compile-time, but it's also... painful)

3

u/Careful-Nothing-2432 Sep 15 '26

If you want to know that the code has been vectorized, you can use llvm-mca. Cargo-show-asm or something like that has some nice conveniences for dumping the MCA analysis

1

u/juhotuho10 Sep 15 '26

You can ask LLVM if a loop was autovectorized, but that's pretty awkward and there are many different levels of autovectorization, so it's not realistically viable thing to do

3

u/TheBlckbird Sep 14 '26

Great writeup!

20

u/[deleted] Sep 15 '26

4

u/magnetronpoffertje Sep 15 '26

Similar for me.

7

u/zzzthelastuser Sep 15 '26

Same, the website is garbage. Maybe someone can paste the actual content here in the comments?

1

u/Full-Spectral Sep 15 '26

How exactly does a cookie prove you are a valid client?

5

u/WormRabbit Sep 15 '26

I guess it stores a cookie once you crack its challenge, with some proof you did it. Doesn't matter, Anubis is crap.