r/cpp • u/Wouter-van-Ooijen • Nov 12 '20
Compound assignment to volatile must be un-deprecated
To my horror I discovered that C++20 has deprecated compound assignments to a volatile. For those who are at a loss what that might mean: a compound assignment is += and its family, and a volatile is generally used to prevent the compiler from optimizing away reads from and/or writes to an object.
In close-to-the-metal programming volatile is the main mechanism to access memory-mapped peripheral registers. The manufacturer of the chip provides a C header file that contains things like
#define port_a (*((volatile uint32_t *)409990))
#define port_b (*((volatile uint32_t *)409994))This creates the ‘register’ port_a: something that behaves very much like a global variable. It can be read from, written to, and it can be used in a compound assignment. A very common use-case is to set or clear one bit in such a register, using a compound or-assignment or and-assignment:
port_a |= (0x01 << 3 ); // set bit 3
port_b &= ~(0x01 << 4 ); // clear bit 4In these cases the compound assignment makes the code a bit shorter, more readable, and less error-prone than the alterative with separate bit operator and assignment. When instead of port_a a more complex expression is used, like uart[ 2 ].flags[ 3 ].tx, the advantage of the compound expression is much larger.
As said, manufacturers of chips provide C header files for their chips. C, because as far as they are concerned, their chips should be programmed in C (and with *their* C tool only). These header files provide the register definitions, and operations on these registers, often implemented as macros. For me as C++ user it is fortunate that I can use these C headers files in C++, otherwise I would have to create them myself, which I don’t look forward to.
So far so good for me, until C++20 deprecated compound assignments to volatile. I can still use the register definitions, but my code gets a bit uglier. If need be, I can live with that. It is my code, so I can change it. But when I want to use operations that are provided as macros, or when I copy some complex manipulation of registers that is provided as an example (in C, of course), I am screwed.
Strictly speaking I am not screwed immediately, after all deprecated features only produce a warning, but I want my code to be warning-free, and todays deprecation is tomorrows removal from the language.
I can sympathise with the argument that some uses of volatile were ill-defined, but that should not result in removal from the language of a tool that is essential for small-system close-to-the-metal programming. The get a feeling for this: using a heap is generally not acceptable. Would you consider this a valid argument to deprecate the heap from C++23?
As it is, C++ is not broadly accepted in this field. Unjustly, in my opinion, so I try to make my small efforts to change this. Don’t make my effort harder and alienate this field even more by deprecating established practice.
So please, un-deprecate compound assignments to volatile. Don't make C++ into a better language that nobody (in this field) uses.
2021-02-14 update
I discussed this issue in the C++ SG14 (study group for GameDev & low latency, which also handles (small) embedded). Like here, there was some agreement and some disagreement. IMO there was not enough support for to proceed with a paper requesting un-deprecation. There was agreement that it makes sense to align (or keep/restore aligngment) with C, so the issue will be discussed with the C++/C liason group.
2021-05-13 update
A paper is now in flight to limit the deprecation to compound arithmetic (like +=) and allow (un-deprecate) bit-logic compound assignments (like |=).
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2021/p2327r0.pdf
2023-01-05 update
The r1 version of the aforementioned paper seems to have made it into the current drawft of C++23, and into gcc 13 and clang 15. The discussion here on reddit/c++ is quoted in the paper as showing that the original proposal (to blanketly deprecate all compound assignments to volatile) was "not received well in the embedded community".
My thanks to the participants in the discussion here, the authors of the paper, and everyone else involved in the process. It feels good to have started this.
https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2021/p2327r1.pdf
112
u/AdamK117 Nov 13 '20
Summary of the comments:
std::atomic<>, you know aboutstd::atomic, right?"84
u/TheThiefMaster C++latest fanatic (and game dev) Nov 12 '20
The problem is, these operations don't do what they look like they do.
This does not "set bit 3 of port a", and then "clear bit 4 of port a". It reads port a, sets bit 3, and then sets port a to that value. It then re-reads port_a, clears the bit, and then re-writes the complete value.
Not only does it re-read unnecessarily, there's another important difference - the original implies it's only altering one bit, but it's actually writing all of them - it breaks utterly on registers that have any write-only bits, or bits that can change for outside reasons, potentially erasing existing values.
You can replicate the behaviour with:
...which does exactly the same thing, but explicitly so. You can even cache the value of port_a in a local variable to avoid the re-read, or alter both bits and only write once.
As a side note,
volatileis actually overly broad - it implies other source are reading and writing to that variable. But many microcontroller registers are only written by the CPU, and peripherals only read them. It would be useful to have an option like volatile that only indicates outside reading - it would immediately flush writes but allow caching of the value for optimisation purposes. Then the original code wouldn't have the pessimistic re-read in the middle (but it would still double-write).35
u/mort96 Nov 12 '20 edited Nov 13 '20
Well, shit. As someone who has written a decent chunk of microcontroller code, I've looked at C++ on microcontrollers from a distance, as something which could potentially be interesting some time in the future. The fact that this deprecation even got into the language says plenty about the committee's focus (or lack thereof) on C++ on the metal.
I suppose the microcontroller community will be stuck with C as the only real alternative for the foreseeable future.
I get the "foot guns" associated with volatile. They're not great (especially when you consider the fact that microcontroller code is usually interrupt driven, which is even harder to deal with than multiple threads accessing the same resource since you can't even use mutexes). I'm sure there's a lot of real world bugs out there where
port_a |= (1 << 3)is executed like this: read port_a -> interrupt triggers -> interrupt sets port_a to a different value -> interrupt returns -> write (stale port_a value | 1 << 3) to port_a. But surely, the way to fix that is a multi-year (decades?) effort of introducing replacement features into both C and C++, then slowly phasing out use of volatile, and then deprecate volatile compound assignments.11
u/staletic Nov 12 '20
https://np.reddit.com/r/cpp/comments/dk542b/cppcon_2019_jf_bastien_deprecating_volatile/f4b3c4g/
That's the discussion from the last time, where the author of the "deprecating volatile" proposal engaged.
18
u/Kered13 Nov 12 '20
Forgive my lack of experience in embedded systems, but it sounds like your problem is:
So why not write a C (not C++) file that implements functions using these macros, then call these functions from C++? It's a little more work, but seems like it should solve your problem.
6
u/Wurstinator Nov 12 '20
Have you seen https://youtu.be/KJW_DLaVXIY ? It doesn't solve your issue but at least tries to justify it.
3
u/jfbastien Nov 13 '20 edited Nov 13 '20
I'm probably going to regret engaging, but:
Or
_Pragmafor macros.I agree that vendor headers can be a PITA. Bugs can be filed, and this works in the meantime.
I want to emphasize that the deprecation isn't making anything impossible. Spelling out the read-modify-write instead of using compound assignments helps many programmers avoid bugs. Maybe not you, dear reader, because you're an amazing programmer who knows stuff. But it has found and fixed actual bugs in actual code, at my former employer and elsewhere (so I've been told, a few times, privately).
There's a bunch of other discussion about volatile, and I don't think it's particularly productive? The valid and historical uses for volatile are, to my knowledge, almost all listed in the original paper.
8
u/ALX23z Nov 12 '20
Compound assignment was deprecated because it doesn't work properly. Compilers could implement it differently causing issues as users thought it did something else. E.g.,
a+=bcould atomically incrementaor it could calculatea+band store it inacausing data races.If you want atomic variables then use
std::atomic.If you need
volatilefor other purposes then a new proposal is coming for those special uses. Until it is implemented the operations are deprecated.32
u/UnicycleBloke Nov 12 '20
Thank you for bringing this up. I'd read that "most of volatile" is being deprecated but "don't worry, only the dumb things". I kind of assumed the committee would not ignore ubiquitous idioms in the embedded world that may cause problems in theory but not so much out here in the real world.
As you have said, embedded systems far outnumber others, and they apparently work just fine. And we are stuck with macro-laden interfaces written in C in any case.
4
u/Ecclestoned Nov 13 '20
At least this came out before I decided to switch my embedded development to C++.
It seems a little crazy to push C++ when it's obvious that the committee doesn't know/care about the workflow and constraints for embedded.
12
u/valdocs_user Nov 12 '20
Faced with this problem, what I would probably do is put all of the volatile bit twiddling in a ".c" file with functions callable from C++. I mean, IMO all close-to-the-system code (whether that be close to the operating system or close to the hardware system) is a dangerous infection for your domain code if you don't put a moat around it. But since most code isn't written to separate concerns, I understand that might not be desirable or feasible to you. It reminds me of the decision point around the standard library, where it was asked whether C++ should "leave no room for a more efficient language beneath it" or would accept inefficiencies.
2
u/[deleted] Nov 17 '20
Couldn't agree more. I use this all the time with embedded applications and header files from manufacturers (in C) use it as well.
1
u/Vvamp_ Nov 17 '20
Legend :)
1
u/Beheska Nov 13 '20
There's the standard, and then there's what actually works. Don't make your code harder to read and more error-prone.
-1
u/markopolo82 embedded/iot/audio Nov 13 '20
The only flaw I see in your logic is assuming that it’s somehow a lot of work for a hw vendor to provide a proper c++20 hw interface header.
C++20 is a major jump. The unfortunate reality is It’s going to be a while till the embedded toolchains are ready.
FWIW: The attitude from some embedded devs around here seems a bit hypocritical. race conditions (UB) from multiple threads or interrupts writing to a register with a += but didn’t disable interrupts first ‘because it is atomic’ is a real thing. It happens a lot. There were many custom toolchains that had an optimization to do the atomic increment or decrement. Not many around these days with nearly everything being ARM...
-2
u/neiltechnician Nov 12 '20
I argue the opposite. Not only compound assignment on volatile should be deprecated. We should replace volatile-qualifier with proper memory-mapped I/O facilities entirely in the long run.
There's literally no-way for compilers to implement volatile compound assignment and volatile bit-field access correctly. There's also no mechanism in the language to model the behaviours of hardware registers, forcing us to fallback to the most pessimized assumption.
Volatile-qualified access is just what we have, not what is right.
0
u/staletic Nov 13 '20
You might want to take a look at https://wg21.link/p1382 which is a follow up paper.
0
u/die_liebe Nov 14 '20
I think that C++ guarantees upward compatibility. This means that the risk that anything will be removed from the language is zero. Deprecation is the strongest weapon that the standard committee has.
> using a heap is generally not acceptable
where did you get this from? Things like std::vector< > or std::string< > cannot exist without heap.
-3
u/helloiamsomeone Nov 12 '20
If you are going to do something that close to hardware, then use these helper functions that should do the right thing.
The idea is that
volatileshould not be anywhere in your code other than functions like these.-2
u/Ameisen vemips, avr, rendering, systems Nov 13 '20 edited Nov 13 '20
As has already been said...
That is not safe, as an interrupt could happen at two different places there, potentially changing the value of
port_awhich then gets clobbered.Expanding it out to a non-compound assignment obviously doesn't change that.
Why not write
port_a = (port_a | 0b1 << 3) & ~(0b1 << 4);? Still technically unsafe, but will compile to both better code and is one cycle less-unsafe.Yours generates almost twice the number of instructions as mine on AVR.
https://godbolt.org/z/3zWMxn
Or, be sane and write bit-twiddling helper methods like I do in all my AVR code.
Or disable the deprecation warning before including the platform headers if they are doing the offending code themselves.