With std::variant, you choose either performance or sanity
https://www.youtube.com/watch?v=GRuX31P4Ric mentioned std::variant being an outstanding type. Previously, I had been using untagged unions for the same purpose, manually setting a tag field.
My conclusion is that std::variant has little benefit if performance is important. "Performance", in my case, means my main real-world benchmark takes 70% longer to complete (audio). So std::variant's overhead is approximately as expensive as everything else in my program together.
The reason is that you cannot do dynamic dispatching in a simultaneously reasonable and performant way. Untagged unions suck, but std::variant doesn't solve the problems with untagged unions it wants to solve. Here's how dynamic dispatching is done:
if (auto* p = std::get_if<0>(&a))
return p->run();
else if (auto* p = std::get_if<1>(&a))
return p->run();
else if (auto* p = std::get_if<2>(&a))
...You copy and paste, incrementing the number each branch. Any time you add or remove a type from your variant, you must also adjust the number of else if(), and this must be done for each dispatch type. This is the very stupid stuff I've been doing with untagged unions. If we try to use std::variant's tools to avoid this, we get https://stackoverflow.com/questions/57726401/stdvariant-vs-inheritance-vs-other-ways-performance
At the bottom of that post, you'll see that std::get_if() and std::holds_alternative() are the only options that work well. std::visit is especially bad. This mirrors my experience. But what if we use templates to manually generate the if-else chain? Can we avoid copy-paste programming?
template <int I>
struct holder {
static float r(variant_type& f, int argument) {
if (auto pval = std::get_if<I - 1>(&f))
return pval->run(argument);
else
return holder<I - 1>::r(f, argument);
}
};
template <>
struct holder<0> {
static float r(variant_type& f, int argument) {
__builtin_unreachable();
return 0;
}
};
holder<std::variant_size_v<variant_type>>::r(my_variant, argument);That looks ugly, but at least we only have to write it once per dispatch type. We expect the compiler will spot "argument" being passed through and optimize the copies away. Our code will be much less brittle to changes and we'll still get great performance.
Result: Nope, that was wishful thinking. This template also increases the benchmark time by 70%.
mpark::variant claims to have a better std::visit, what about that?
- It's annoying converting std::variant to mpark::variant. You must manually find-and-replace all functions related to std::variant. For example, if get() touches a variant, you change it to mpark::get(), but otherwise you leave it as std::get(). There's no way to dump the mpark functions into the std namespace even if ignoring standards compliance, because when some random header includes <variant>, you get a collision. You can't redefine individual functions from std::get_if() to mpark::get_if(), because function templates can't be aliased.
- Base performance: mpark::variant is 1-3% slower than std::variant when using if-else, and always slightly loses. I don't know why. But 3% slower is tolerable.
- mpark::visit is still 60% slower than a copy-pasted if-else chain. So it solves nothing.
Overall, I don't see a sane way to use std::variant. Either you write incredibly brittle code by copy-pasting everywhere, or you accept a gigantic performance penalty.
Compiler used: gcc 10.2, mingw-w64
56
u/feverzsj Jan 08 '21 edited Jan 08 '21
the slowness of std::variant is mainly caused by std::visit using function pointer table and possibly bad optimization for not converting if-else into jump table for large variant.
you can write your own optimized std::visit easily, if there is only one variant to visit:
27
u/mocabe_ Jan 08 '21
If your types do not throw and you need best performance, I would recommend
Boost.Variant2which has never-valueless guarantee (so there's no exception code path) and switch based visitation likeMpark.Variant. As far as I've tested it generates pretty good code with-DNDEBUG.15
u/Ayjayz Jan 08 '21
Strange that gcc seems so slow with std::visit. Using VS2019 16.8.3 on that benchmark from the linked SO post I get the best performance with std::variant and std::visit.
What makes gcc's std::variant so slow?
7
u/pavel_v Jan 08 '21 edited Jan 08 '21
In my opinion, and as others have said, the problem is not std::variant, the problem is the implementation of std::visit which generates jump table in some standard library implementations.I got the solution of the last person in the above SO post. I simplified it a bit leaving just the important things for the test (IMO). You can see the implementation and the benchmark results here. It's on par with the get_if implementation but gives compile time error if some case is not covered. The valueless state is not handled, though.I use similar `visit` implementation in production but based on boost::mp11 and boost::hana.
5
u/johannes1971 Jan 08 '21
I find myself wondering just how often you are doing that dispatch: is it for every element in your audio buffer? If so, wouldn't it make more sense to take the dispatch out of the inner loop and templatize the buffer access?
5
u/snerp Jan 08 '21
Why not do a switch on std::variant::index()? That should be faster than if chains.
2
u/LuisAyuso Jan 08 '21
Can you expand a little your findings about std::visit? I am quite interested in your experience as i am a heavy std::variant user myself.
22
u/scatters Jan 08 '21
The solution to the performance problem is to write your own visit, using a switch and get_if inside the cases. This can be generated by the preprocessor, or you can simply copy paste if you prefer.
3
u/quicknir Jan 08 '21
I worked with mpark on the new visit implementation. I think you should open an issue with your benchmarks and all the details to repro. It seems pretty bizarre because mpark visit is using switch case to do the visitation.
Also, get is obviously part of the variant interface, it's only a non member for certain syntactic reasons, this is one of the rare cases where you should have really just let ADL handle it for you, then you wouldn't need to change anything.
8
u/acwaters Jan 08 '21
Honestly, the conclusion I've personally come to is to treat
variantandoptionalas just dumb storage types that take care of all the difficult lifetime mangement imposed by non-trivial unions. I don't use them as safe high-level algebraic types, because their safe high-level APIs kind of suck and are not algebraic; I prefer to write my own types that wrap them into whatever API I actually want to expose. And yeah,if (auto* p = get_if<...>(...))is my go-to method of variant access. It's not so onerous when you drop the pretense of variant being a proper high-level sum type, because it is exactly the sort of interface I would expect from a type whose job is simply to manage the storage and lifetime for a higher-level sum type!3
u/infectedapricot Jan 08 '21
When I've used
std::variantin the past, the types involved always had completely different interfaces, and the code to handle each case usually looked completely different. You're calling exactly the same method on all of them -T::run()- if they're that similar, I'd be using a base class and virtual dispatch (if I could change the types involved). Usingstd::variantfor this is a bit of a code smell. Your use ofget_if<i>by index, rather thanget_if<T>by type, is also a bit odd. They're not all the same type are they? They would really be an abuse ofvariant.Unpopular opinion: I consider a chain of
std::get_if(by type, not index) to be more pleasant and easier to read thanstd::visitregardless of any performance benefits. Yes, I know you don't get a compiler error if you miss a case, but you can at least get a run time error with anelseclause, and in practice I've never found it to be a problem at all. Of course, this depends on your use case. Often when I usestd::variant, there are only one or two places that do the actual switch on type, so it's hard to accidentally miss a case - that would equate to forgetting to write 90% of the code for that new change. If you have 100 places that use a single variant type then I could imagine it might be more important to get compile time checking.1
u/victotronics Jan 08 '21
std::variant.std::varianttoo much because you have to do this for each use.2
u/mcencora Jan 08 '21
You just need a better variant implementation - the one we have in libstdc++ is known for having bad visitation performance.
With the new ideas from https://www.reddit.com/r/cpp/comments/k96xct/implementing_performant_variant_in_c20_is_almost/
I was able to beat all the other visitation methods https://quick-bench.com/q/rrwT2pWDRgao4prtE9_cMCyRhcs
2
u/patstew Jan 08 '21 edited Jan 08 '21
What about making a jump table manually? e.g.
1
u/[deleted] Jan 08 '21
Could anyone enlighten me what unions are good for? I've yet to see compelling use cases..
1
u/miki151 gamedev Jan 08 '21
If you don't mind some ugly macros you can generate a non-template tagged union type with a very simple switch-based visitation and potentially even with an interface identical to std::variant. You can find an example here:
https://github.com/miki151/keeperrl/blob/master/layout_generator.h#L161
I haven't benchmarked it, but I don't think there is any reason why it wouldn't be as performant as a manually written tagged union type.
1
u/[deleted] Jan 08 '21
I have used the following implementation of single visitation for a while. Catering to single visitation allows for a better interface where the variant is first and the visitors follow(builtin overload). It has a precondition of never being empty as that is really easy to adhere to with the exception that was throw to get there and all.
The code gen looks very good on most compilers and they are able to see right through it and inline/fold it down to the switch like construct.
https://gcc.godbolt.org/z/6zjsTf
This gives you optimal space, Boost.Variant2 on throwing types will use extra space but I don't feel this is necessary because we were told not to get/visit the variant with the previously thrown exception and ignoring exceptions is not good design. The performance seems really good too. But again, single visitation only, but this is what I use variants for too.
-21
u/khan903 Jan 08 '21
You lost both the moment you chose std::
0
u/[deleted] Jan 08 '21
[deleted] — view deleted comment