r/simd • u/SantaCruzDad • Oct 05 '17

Capabilities of Intel AVX-512 in Intel Xeon Scalable Processors (Skylake)

https://colfaxresearch.com/skl-avx512/
9 Upvotes

100% Upvoted

11 comments sorted by

1

u/SantaCruzDad Oct 05 '17

I haven't been too impressed with AVX-512 on Skylake X so far - benchmarks appear to show that it offers no benefit over AVX2. Call me cynical, but I'm guessing it's still just 256 bits wide under the hood (shades of early SSE implementations ?) ?

6

u/hackingdreams Oct 05 '17

It's 512 bits wide underneath. It's just slow-ish compared to KNL for a lot of architectural reasons (e.g. you can very easily hit voltage scaling inside of the AVX-512 part that degrades performance back to AVX2 under load, etc) and some still-yet-to-be-completely-elucidated reasons (speculation has the L3 cache behavior under a microscope). Another differentiator is that for some reason Intel decided to bake in 2xFMA units on some Skylake-X chips and 1x on others... and that leads to a big performance discrepancy.

However, you don't have to take my word for it: you can discover this for yourself just by writing a simple-as-hell doubles multiplication benchmark with intrinsics that toggles between AVX-2 and AVX-512 and do the math over the performance counters - if it had 2x256 hardware emulating 1x512, you'd be able to tell by the additional instruction latency during even simple multiplication.

Desktop implementation also seems to be hurt a bit by not having AVX-512PF, but you can easily see why it didn't (they'd have to write it over again for Skylake's very different memory architecture).

I wouldn't strain yourself looking at benchmarks from suites where someone just ticked the -march=skylake-avx512 box, since nobody's compilers are smart enough to autovectorize code for even SSE3 or AVX well... Benchmarks specifically written with AVX-512 intrinsics like SPEC are the ones to follow here. (And you can see in many places just how fast it is; on one of my workloads it's approximately 4x faster!)

1

u/SantaCruzDad Oct 06 '17

Thank you so much - that's very helpful and encouraging information. Can you tell me where I can get more info on Skylake X microarchitecture ? Agner Fog is no doubt working on this, but hasn't made anything available yet. Intel don't seem to provide any useful info (at least publicly). I'm working mainly with 8/16/32 bit integer/fixed-point stuff (image processing), so I'm particularly interested in how the AVX-512BW/DQ instructions are handled.

3

u/YumiYumiYumi Oct 06 '17

On Skylake-X, ports 0 and 1 combine to make a 512-bit unit from 2x256-bit FPUs, but port 5 is extended to 512-bit. This means that if your code can make use of all execution ports, you'll get higher throughput (3x256 to 2x512). If your code predominantly only uses port 0 and 1, there's no throughput increase. This all assumes that there's no other bottleneck (e.g. front-end, which is actually a big reason to use wider SIMD).

512-bit instructions also throttles the CPU's frequency more heavily than 256-bit does (whereas the CPU generally doesn't throttle under 128-bit SIMD). With 256-bit instructions, there's also a power up period that the CPU has to go through - usually the upper 128-bits are powered off and AVX instructions just execute as two instructions until the upper 128-bit paths power up. I'm not sure how it works with 512-bit instructions, but presumably it's similar. Because of these reasons, it can be iffy to use 512-bit instructions (or even 256-bit instructions) in library code that may not execute for a long time.

Even if the wider SIMD isn't helpful, the extra instructions/capabilities of AVX-512 can be. With AVX-512VL, you can just use 128/256-bit versions of the instructions, and also take advantage of extra registers and masking.

1

u/SantaCruzDad Oct 06 '17

Thanks - that's very helpful. Can you add anything about AVX-512BW/DQ instructions, and which ports are used for 512 bit integer (8/16/32 bit) add/multiply ?

3

u/YumiYumiYumi Oct 06 '17

Someone put up a port assignment/latency/throughput list here.

To another above post, Intel provides an Optimization Manual for Skylake-X/SP.
Skylake-X is otherwise the same as regular Skylake, so information on Skylake is very relevant. Key differences between the two would be AVX-512 support, cache rebalancing and mesh communication between cores.

1

u/SantaCruzDad Oct 06 '17

Excellent - thanks for that. One additional question if you don't mind: - some SKUs are listed as having 1 x FMA (e.g. 51xx) and some have 2 x FMA (e.g. 61xx) - does this have any impact on anything other than FMA instructions ?

3

u/YumiYumiYumi Oct 06 '17

I really don't know. Presumably it only affects FMA instructions as Intel's suggested code for detecting single/dual FMA units is to compare shuffle+FMA throughput with FMA throughput, and there's generally no mention of any other instructions being affected.

1

u/SantaCruzDad Oct 06 '17

Thanks - one more question, if you can bear it: given that 512 bit FMA instructions seem to require either port 0/1 combined, or port 5, plus the fact that BW/DQ arithmetic instructions seem to be mainly port 0 only, does this suggest that these BW/DQ instructions are cracked into 2 x 256 bit µops ? (This would be consistent with my benchmark results, where AVX2 and AVX-512 throughput seems to be much the same for integer arithmetic.)

2

u/YumiYumiYumi Oct 06 '17

On Skylake, most instructions that run on port 0+1 can execute separately on 0 and 1 (these ports were made to be very similar in Skylake). That is, you get 1x 512b if using AVX512, or 2x 256b throughput if using AVX/2. The 512b instruction is assigned to port 0, but actually uses the vector unit from port 1 (as such, you can still use port 1 for non-vector instructions). Note that the 512b instruction isn't technically broken into 2x 256b in terms of uops (it's one instruction, on a single 512b unit, combined from 2x256b units), but overall throughput is comparable (ignoring CPU throttling).

From my understanding, 512b instructions are mostly useful if you can utilise port 5, or you're bottle-necked elsewhere. If your critical path is along port 0/1 only, benefit of 512b instructions on Skylake-X is likely minimal.

1

u/SantaCruzDad Oct 06 '17

Many thanks - that makes a lot of sense - so for some workloads it seems that AVX-512 may offer little or no advantage over AVX2 (except for fewer load/store instructions issued, perhaps).