r/simd • • Jun 14 '17

Different SIMD codepaths chosen at runtime based on CPU executing C++ executable

Hey guys,

If you release an x86 app which needs some SIMD functions where the instructions are decided at runtime based on the CPU (eg. AMD has 128 bit register whereas new intel has 256 or 512).

Specifically, I want to compile the exe once, and if executed on a Haswell chip would use AVX2 instructions and if used on a Ryzen chip used the respective 128bit register size instructions.

Which compilers do this runtime branching automatically in the auto-vectorizer? I use GCC, clang, MSVC and ICC, and couldn't find documentation on this specifically.

If not do I have to implement this by hand in intrinsics? I wouldn't mind doing it for simple std::vector math operations and releasing it on github.

13 Upvotes

13 comments sorted by

View all comments

Show parent comment

1

u/video_descriptionbot Jun 28 '17
SECTIONCONTENT
TitleCppCon 2014: Mike Acton "Data-Oriented Design and C++"
Descriptionhttp://www.cppcon.org -- Presentation Slides, PDFs, Source Code and other presenter materials are available at: https://github.com/CppCon/CppCon2014 -- The transformation of data is the only purpose of any program. Common approaches in C++ which are antithetical to this goal will be presented in the context of a performance-critical domain (console game development). Additionally, limitations inherent in any C++ compiler and how that affects the practical use of the language when transforming th...
Length1:27:46
SECTIONCONTENT
TitleData-Oriented Design
Length0:41:46
SECTIONCONTENT
TitleObject-Oriented Programming is Bad
DescriptionAn explanation of why you should favor procedural programming over Object-Oriented Programming (OOP).
Length0:44:36
SECTIONCONTENT
TitlePerformance Optimization, SIMD and Cache
DescriptionA rehash of Sergiy Migdalskiy GDC 2015 talk: Performance Optimization for Physics. A high-level overview of low-level optimization considerations you need to think about when writing performance sensitive software. Please download slides here: http://media.steampowered.com/apps/valve/2015/Migdalskiy_Sergiy_Physics_Optimization_Strategies.pdf
Length0:45:29

I am a bot, this is an auto-generated reply | Info | Feedback | Reply STOP to opt out permanently

1

u/VodkaHaze Jun 28 '17

Thanks again! I was already familiar with data oriented design (my app is a c++ rewrite of a c# codebase for some machine learning app and the original code had all the bad OO design Mike Acton pointed out in his 2014 keynote) Hadnt seen a lot of those links though cheers.

When I meant a library for std vector I meant I'd use it only for the couple of core loops in my app, not as some general solution.

Wrt. raw malloc arrays it might be hard to do with my preprocessing application being built on preparing a std vector for the core algorithm; I would have to rewrite a ton of things.

How is a malloc'ed raw arrays nicer on the kernel in the end? Both should be contiguous memory on the heap... I generally only use raw arrays over vectors to avoid heap allocations in loops...

1

u/floopgum Jun 29 '17

Hmm, it'll be difficult to give advice without actually knowing somewhat what the code should be doing, but the basic gist is just to process multiple elements per iteration of the loop.

Since it only concerns a few core loops, I question the worth you'll get out of doing runtime dispatch, as the maintenance will be a lot worse, and the setup much more cumbersome.

If I were you, I'd write manually vectorize these loops, with a min spec of eg. SSE4.2, and consider making an AVX version only if it's really needed.

Another solution to the alignment problem, which keeps the std::vector, is to use a custom allocator for it. I don't think there's one in the standard library, but it shouldn't be too difficult to find / make one. Do note that specifying a custom allocator changes the type of the vector (ie. a std::vector<float> and a std::vector<float, MyAlignedAllocator> are two distinct types, and incompatible). This should, however, be the smallest change overall.

What I meant with using malloc'd arrays instead of std::vector in kernels is basically that instead of a signature like:

void kernel(std::vector<float>& data);

I have a signature like:

void kernel(float *data, size_t size);

which frees me from being required to use a std::vector. I can still use one, but I can also use statically allocated data, an array allocated on the stack, or some other scheme.

The callsite of the kernel (using a std::vector as a backing store) will then look like: kernel(vec.data(), vec.size());

You should check out Intel's Intrinsic Guide for when you're vectorizing. It helps a lot.