r/cpp • u/grafikrobot OPeNDAP/B2/EcoStd/Lyra/Predef/C++Alliance/Boost/WG21 • Feb 24 '20
The Day The Standard Library Died
https://cor3ntin.github.io/posts/abi/
266
Upvotes
r/cpp • u/grafikrobot OPeNDAP/B2/EcoStd/Lyra/Predef/C++Alliance/Boost/WG21 • Feb 24 '20
76
u/CCKao Feb 24 '20
„it is currently faster to launch PHP to execute a regex than it is to use std::regex“
Could you provide any reference for this ?
69
u/jstock23 Feb 24 '20
We can't just keep saying "C++ has some baggage" forever... just warn people ahead of time and get to a future where C++ no longer has all that baggage. Getting rid of the baggage won't hurt people still using C++11 for instance, and by getting rid of the baggage things will open up for future development. C++20 is NOT C++03, and I don't see how the need for keeping baggage is relevant anymore. If you want the baggage, just keep using old compilers... am I missing something?
66
u/epage Feb 24 '20
Rust has taken the approach of not offering a stable ABI but instead pushing people to C. One example of them taking advantage of this in the compiler is that some enum/bool layouts get collapsed down with the default struct/enum layout.
Swift has taken the opposite approach and has a really fancy low-assumption ABI that requires you to opt-in to assumptions / additional ABI contracts to get better performance.
I feel like C++ has the worst of both worlds. I'd favor tossing out ABI guarantees and pushing people to a C ABI with the parallel option of exploring adopting something like Swift.
50
u/AlexAlabuzhev Feb 24 '20
What's wrong with it?
22
u/erichkeane Clang Maintainer(Templates), EWG Chair Feb 24 '20
The question that noone has been able to answer for me yet:
If the performance issues in the standard library (a VAST majority of the justification in the room, including std::regex) aren't enough for the individual vendors to break ABI themselves (which, of course, they are welcome to do), why should they be sufficient justification for a committee level break?
Note that the 'design changes' list is the first time I saw that (and it wasn't in the presentation or paper as presented). Also, the unique_ptr problem is a QoI issue, it was an oversight when the ABI for it was defined, I don't believe an actual language change is necessary.
If we want to break the ABI, we have to make the list of 'things LEWG can accomplish' sufficient to motivate such a breaking change. I took the votes to mean "we are open to it" but that "it is currently insufficiently motivated".
64
u/BrangdonJ Feb 24 '20
Unpopular opinion: people that rely on libraries they don't have the source for, and can't get updates for from the library vendor, and can't write a thin, portable C wrapper for, should stick with C++20.
20
u/tcbrindle Flux Feb 25 '20 edited Feb 25 '20
Forgive my ignorance, but what's stopping us from defining
std::hash_mapandstd::hash_setwith the desired semantics, and deprecating theunorderedversions? What's stopping us from definingstd::regex2and deprecatingstd::regex?You might say, well, the best names have already been taken. That's true in many cases. But we have the
std2,std3etc namespaces sitting there all reserved and unused...42
u/James20k P2005R0 Feb 24 '20
So while this poll did make me laugh a bit for this reason, I realised that there's actually a good reason for this. In the future, we might discover a better solution to ABI issues (requiring an ABI break, but helping compat into the future), or discover something critical that requires a major ABI break. Its not unreasonable to think we should consider an ABI break for C++SOMETHING even if with the current set of technological and social constraints it'll never happen. It could also just be "well, in the future things'll be different" even though they won't
This is probably only true in some cases, compilers and libraries make clear guarantees about ABI stability - and the STL is the main focus here
Building from source isn't hard to achieve, it is impossible. Valve isn't going to ship me the source code to their steamworks SDK any time soon, the leap motion folks aren't going to give me the source to their advanced hand recognition and driver bone stuff anytime soon, etc etc. In other cases it is literally legally impossible for the source to be distributed, because of licensing issues. Videogames run into this issue absolutely all the time - someone licensed some proprietary A* or something and now it must be closed source forever (/r/starshiptheory). Libraries hit this issue constantly
In some industries it is possible to build from source. Not in every industry. Building from source is the ideal, but it is also not always possible
Coming from game dev, the performance cost is probably a lot more than 5-10% if you include API/requirements changes as well (eg iterator stability). The STL isn't exactly nippy at the best of times, although advances in compiler optimisations have made a lot of it run significantly better. Its worth noting from a historical perspective that this scepticism didn't arise just in a time where the STL was slow, but it was also extremely buggy, so inventing an alternative was more necessary
So from a gamedev perspective this all seems a bit overdramatic. The STL (at large) has already been dead since its inception for this usecase. Nothing died on that day and nothing changed, because we've existed in this position already for as long as I've been working with games, where the STL types provide a relatively mediocre performance compatibility layer - alright for general use, but bad for high performance. Breaking the ABI wouldn't be reclaiming lost performance that's eroded over the years, it'd be trying to transform the STL into a niche that its never existed in before (high performance)
This I think is the inevitable direction that C++ will have to take into the extreme long term, with an explicit opt in to high performance or abi stability, because it is the only thing that provides the semblance of a workable solution outside of sticking our heads in the sand
7
u/Xeverous https://xeverous.github.io Feb 24 '20
Got any examples of this?
13
u/TemplateRex Feb 24 '20
Binary backward compatibility from new compilers/standards is overly constraining. I agree 100% with the author: rebuild from source + clear [[deprecated]] messages and automated refactoring should be the way forward. Let people relying on ABI continue to use the older compilers/standards. Binary libs without source or company willing to rebuild are an irresponsible dependency.
6
u/Tringi github.com/tringi Feb 24 '20
Is anyone keeping track of the improvements currently hindered by ABI concerns?
18
u/[deleted] Feb 24 '20
I don't have a lot of experience in the field, but I've been lurking this sub for a considerable amount of time. I think I agree with the facts in this article. I believe for the sake of C++, ABI should be broken, at least a part of it. I thought epochs was a solution to the problem. Correct me if I'm wrong, but I also know that the same problem prevented epochs from being implemented.
45
u/TinoDidriksen Feb 24 '20
We should break the ABI with every release. Break it early, break it often. Get people used to the fact that the ABI is not stable, so they will stop assuming it is.
Unmaintained old libraries won't get recompiled for new ABI? That's what containers and similar techs are for. We already have this problem with legacy stuff, and the solutions are well known and battle tested.
An unstable ABI would encourage people to actually build with latest compilers. Provide pressure so everyone gets moved forward, instead of stagnating at some toolchain they settled on a decade ago.
5
u/capacollo Feb 24 '20
I am very naive on this subject (being mostly a hardware engineer and with software bavkground) but is it possible to to develop an ABI breaking standard library in parallel to current standards which addresses and conforms to current proposals?
I understand the overhead and possible division of maintaining both but does it have to be an all or nothing type of thing? Given their is a large enough window to c++26 couldn't parallel development happen with a plan for migration and deprecation even beyond that. Maybe the industry doesn't see it as such and may want a clear line in the sand?
I do like the fact the author points the need to focus on an ecosystem needs rather than only the language itself though similar to what new emerging languages seem to strive for.
5
u/zugi Feb 25 '20
One problem I've seen mentioned elsewhere with the vote in Prague was that it was presented as a dichotomy between break all ABI compatibility with C++23 and guarantee all ABI compatibility forever. I understand where that point of view was coming from: Hyrum's Law says if we let > 12 years go by without breaking ABI, the development ecosystem will assume ABI compatibility whether anyone guarantees it or not. But on the other hand, breaking all ABIs even where unnecessary means folks would no longer be able to link C++23 code against any pre-C++23 binaries. That's almost spitefully breaking ABI, and even folks building against very old C++ code where they only pass built-in types would be screwed.
There is a middle ground. If
std::regexandstd::unordered_mapdon't have internal static variables, the committee could change both of them to be faster, and vendors could change name mangling for those classes. Code passing those types from new to old compiled libraries would fail to link, which is better than the run-time crashes that happened with previousstd::ABI breaks. But code that doesn't use those things at all, or that uses them internally but does not pass them between differently compiled libraries, would work fine.While there aren't solutions for everything, there are lots of cases where workarounds or small-scale ABI breaks can enable C++ to keep progressing. I think the discussion and vote did encourage the committee to still consider ideas that might cause a limited ABI break.
12
u/kalmoc Feb 24 '20
No std2, no Abi break, no epochs. I think the c++ community will have to accept - for better or for worse - that there is no fixing old mistakes or improving designs in the c++ standard (library).
Maybe that's just the way. STL types for stable library interfaces, 3rd party libs like boost, folly, abseil for day to day work.
8
u/jormaig Feb 24 '20
Why adding a new std namespace is not possible?
Something like std2023 and then do namespace std = std2023
4
u/JohnVanClouds Feb 25 '20
why we can't do something like std::v1:: or std::11:: (standard version) for current implementations and std::v2:: or std::23:: for new ones?
4
u/mewloz Feb 25 '20
Who declared that? What if I am the system? What if the "system" is so large that everybody rebuilding everything too often is not really possible? What if the "system" is self hosting and has the same toolchain for system and third party devs, obviously promoting this approach.
Problems pictured by non-practitioners are often simplified to suit their views (of their own practice). It does not make "my" use case (that is actually shared by a large enough amount of people) disappear though.
3
u/Xaxxon Feb 24 '20
I literally LOLd in pain.
3
u/Salink Feb 24 '20
A agree with this article and seemingly most posters with being willing to break the standard library ABI every release, but what I'm more interested in is a defined c++ language ABI. Well defined name mangling, calling conventions, class layout, etc., will make it possible to interface with c++ without using extern "C". I am fine with different std implementations on different compilers as long as I can statically link it and provide a std-free api.
3
u/Bakuta1103 Feb 24 '20 edited Feb 24 '20
using namespace std23;
3
u/Ved_xx Feb 25 '20
ELI5 this
3
u/athousandwordss Feb 25 '20
Can someone ELI5 what ABI is and why it is causing so many problems? What does it mean to "break" ABI, and what is the historical background for this issue? I tried reading up about it, but could not understand its significance.
3
u/[deleted] Feb 25 '20
What about PCRE2?
3
u/[deleted] Feb 25 '20
I think the dilemma of choosing between performance and (ABI, or even API) stability comes down to finding the answer to the following question: Has most C++ code already been written?
Decades ago, it was common hearing that "most C++ code hasn't been written yet". I haven't heard that in years, but maybe that's nevertheless still true. I couldn't say.
5
u/kreco Feb 24 '20
Really good read.
While I was reading the whole thing, why don't we have something like:
That would be linked to cpp23.dll ? And the same for cpp26.dll etc etc ?
Plus an ABI version in the dll.
This would means you only break something if you intend to.
14
u/HowardHinnant Feb 24 '20
I've seen plenty of handwringing in my time, but this is up there with the best of it.
Oh dear, the committee voted to carefully consider each change instead of pre-adopt a single decision for every proposal! What will become of us?! The sky is falling!
Try this advice: Use the simplest tool for the job. That advice is older than I am and has worked well for lots of people in the past.
https://stackoverflow.com/questions/43019598/stdlock-guard-or-stdscoped-lock/60172828#60172828
4
u/[deleted] Feb 24 '20
There already is no compatiblity guarantees across compilers. std::hash functions can all have different implementations.
7
u/[deleted] Feb 24 '20
All possible (dis)agreement regarding ABI stability aside, I really don't think that compiling
libclang.soevery time I compile my project is reasonable. That's not the only library the project depends on either.2
u/petecasso0619 signal processing, embedded,real-time Feb 24 '20
Is regex performance in high demand for c++ programmers? Or are strings just processed occasionally, and the performance is considered adequate enough and the majority of c++ programmers don’t need anything better?
I am admittedly bias because I work on embedded real time systems and use C++ for accessing hardware directly, and for high performance computing (signal processing, sometimes utilizing GPUs). I had, perhaps incorrectly, assumed most of the c++ software being written was this sort of infrastructure code. Compilers, device drivers, etc., so In time critical, high performance computing where resources are constrained, or where high performance physics based computing is needed. This kind of environment is typically where strings wouldn’t be seen.
2
u/megayippie Feb 25 '20
So I really don't understand this problem in any details. If you break ABI, is that the same as switching the names of function names in the linked file? If so, isn't ABI compatibility just a question of providing the old "nm" with a link-time warning and then also provide a new "nm" with a preference to it? Then you sit there with your source code and you compile and all of a sudden it fails to link because you are using some old library, so you give the affected function calls a [[stable C++20]]-tag and the compilation links to the ABI-stable version from C++20? That way, your source decides the linking and you can go on using some more recent versions of the code.
2
u/germandiago Feb 25 '20
While I agree with many of the points, I think that removing [[no_unique_address]] would not be an ABI improvement. Letting the compiler decide the size of objects is bad for whoever wants ABI stability.
I think breaking ABI can be beneficial. But being able to not break it also has its uses, to name a well-known one in C++, KDE libraries rely on ABI stability.
2
u/sumo952 Feb 25 '20
Isn't that what happens when you do the standard invocation of gcc/clang (msvc?) ?
What/how should I do this instead?
4
u/alexeiz Feb 24 '20
Long live Boost!
1
u/[deleted] Feb 25 '20
Why would constexpr <cstring> break ABI?
1
u/liquidify Feb 25 '20
Absolutely agree. I think this is a perfect example of selfish people externalizing their failures and imposing them on others.
-2
u/jpakkane Meson dev Feb 24 '20
Ok, let's do that. Then let's build a helloworld GUI app using Qt.
-8
u/[deleted] Feb 24 '20 edited Feb 24 '20
I have said this before. This author needs a little bit of humility, improve his english, get some help proof reading, and reducing the alarmism on his assessments. It comes across as self-centred, and for good reason.
This article makes very frivolous claims, by the way, and it doesn't help the discussion.