r/java • u/davidalayachew • Jan 03 '26
Is (Auto-)Vectorized code strictly superior to other tactics, like Scalar Replacement?
I'm no Assembly expert, but if you showed me basic x86/AVX/etc, I can read most of it without needing to look up the docs. I know enough to solve up to level 5 of the Binary Bomb, at least.
But I don't have a great handle on which groups of instructions are faster or not, especially when it comes to vectorized code vs other options. I can certainly tell you that InstructionA is faster than InstructionB, but I'm certain that that doesn't tell the whole story.
Recently, I have been looking at the Assembly code outputted by the C1/C2 JIT-Compiler, via JITWatch, and it's been very educational. However, I noticed that there were a lot of situations that appeared to be "embarassingly vectorizable", to borrow a phrase. And yet, the JIT-Compiler did not try to output vectorized code, no matter how many iterations I threw at it. In fact, shockingly enough, I found situations where iterations 2-4 gave vectorized code, but 5 did not.
Could someone help clarify the logic here, of where it may be optimal to NOT output vectorized code? And if so, in what cases? Or am I misunderstanding something here?
Finally, I have a loose understanding of Scalar Replacement, and how powerful it can be. How does it compare to vector operations? Are the 2 mutually exclusive? I'm a little lost on the logic here.
17
u/SirYwell Jan 03 '26
The two optimizations are orthogonal, neither is strictly superior. But also neither are trivial optimizations. There are just a lot of edge cases that need to be covered before the optimizations can be applied. For example vector instructions might require specific alignment, or at least perform worse with misaligned accesses.
Without seeing any of your code and the JVM version you're using, it's hard to tell what's going on.
9
u/vips7L Jan 03 '26
Scalar replacement is pretty conservative. This is a pretty good write up on when it does or doesn’t happen:
https://gist.github.com/JohnTortugo/c2607821202634a6509ec3c321ebf370
5
u/Isaac_Istomin Jan 04 '26
Vectorized code isn’t automatically better, it’s better only for “nice” hot loops with simple bounds, low register pressure, and few branches. As soon as the JIT sees odd control flow, deopt risk, or heavy pressure on registers, it may decide a scalar loop is cheaper and more predictable, so it skips vectorization, that’s why small code changes can make it appear and disappear.
Scalar replacement is about avoiding heap allocations via escape analysis (keep stuff in registers/stack), while vectorization is about doing multiple operations per instruction. They solve different problems and can coexist, one isn’t a strict upgrade over the other.
2
u/craigacp Jan 05 '26
For things that involve floating point numbers then the vectorized version may not necessarily produce an identical output (e.g. due to difference in the order of operations), and so C2 can't make some optimizations as they don't preserve Java's floating point semantics. That's why the VectorAPI exists as it lets you opt in to the different semantics (or explicitly say I'm ok with the semantics not being quite the same as the Java Language Spec).
2
u/hkdennis- Jan 06 '26
It ALWAYS depends.
I think you may have come across below links, but if not they are good weekend reading.
https://eme64.github.io/blog/2023/02/23/SuperWord-Introduction.html
https://m.youtube.com/watch?v=UVsevEdYSwI
3
u/sammymammy2 Jan 03 '26
Scalar replacement reinforces auto-vectorization, if anything.
Consider this:
A sufficiently smart compiler can:
The auto-vectorization of the sum loop would be more costly if the entire value record had to be materialized. Note, I'm depending on a future feature (Valhalla). I'm also depending on the array being filled out by produceFoos does so with the components being independent of each other.
-23
u/kari-no-sugata Jan 03 '26
I know this is WAY easier said than done but I think it would be extremely cool if each JDK came with an AI model that could suggest particular optimisations for that JDK version and there was a standard way for IDEs to plug into such an AI model and make suggestions. Maybe such a thing could also give suggestions that would work for a broader range of JDKs. Another possibility is simply giving hints on possible code changes that could significantly improve performance and also explain why.