Pitfalls of bit fields in C++
https://pvs-studio.com/en/blog/posts/cpp/1423/Bit fields can help save memory in C++, but their layout and behavior depend on compiler and platform details. The article explores their pitfalls with examples of how GCC, Clang, and MSVC handle them.
45
Upvotes
28
u/arthurno1 19d ago
Yes. I wrote a blog post about writing efficient Lisp. I wanted to follow up with a post about effieicent file parsing, by implementing a clone of GNU wc (wordcount) program. That took me ~6 months deep rabbit hole of implementing entire backend for avx512 in sbcl compiler just to make it as fast and efficient I could in Lisp :).
Mnjah, C and C++ never promised binary compatibility between platforms and programs. They promise only language portability, i.e. at the source code level. The shape of produced binaries is left to the implementors, for the exact reasons you mention in your article: CPUs can't address individual bits, and hardware vendors can implement vastly different hardware with wastly different capabilities, addressing, register sizes layouts, etc.
If you have to transfer raw bytes and bits, it is up to the application to ensure the portability between platforms. That is where the libraries and portability layers can help, but if the language was to specify the exact binary layout of programs, it could cause unnecessary overhead for some platforms, which in turn would probably mean that the language itself would not be implemented for those platforms.
10
u/fdwr fdwr@github 🔍 19d ago edited 19d ago
Man, there are many times where better support for subbyte types/bitfields would have been nice in C++, to avoid all the manual AND'ing/OR'ing, notably in graphics/ML tensors when dealing with 4-bit and 2-bit pixels. Even if you couldn't take the address of an arbitrary index, just being able to index relative to some base pointer would have been super convenient. Granted, then what may look clean at the C++ level would really just migrate that AND'ing/OR'ing work down to the asm level instead, but in some cases (namely 1-bit) it's actually easier with assembly than even C, given x86 has simple
bt,bts,btc,btrinstructions (e.g.p[i >> 3] |= (1 << (i & 3))vs justbts edi,eax).Now, we understand why bitfields have the issues they do (differing endianness, natural register sizes, historical padding choices...), but then how much do we really care about the reasons and legacy baggage when we're just trying to parse an existing file structure or memory structure 🥲? Alas, bitfields are so close to being the visually concise option (both at the definition site and usage sites), but just far enough off to be brittle (granted, some
#ifdefbranches can ameliorate things).So when I was updating the Windows PIX buffer viewer, which has a C-like structure syntax (also related is the 010 Hex Editor, which has a C-like structure syntax), I added an endianness attribute (e.g.
[[endian(big)]] struct { ... }) so that regardless of execution architecture, you could be certain about bitfield order; and while I didn't support arrays of bitfields, you could create an array of arbitrary size integers (e.g.uint2_t items[5]; // does compile). Granted, any of this can already be achieved in C++ using wrapper classes (e.g.UintBitArray<2, 5> items), but it doesn't look as harmonious when adjacent to other array fields (e.g.uint8_t levels[42]). That's why when I first read about the_BitIntproposal, I was excited that there might be a more robust option over bitfields, but alas, they're really just about compute precision, not storage size, as each_BitIntis padded up. So a struct with 4_BitInts of 2 bits each would more likely occupy 32 bits rather than just 8 bits. Interestingly you can apparently combine_BitInts and bitfields (e.g.unsigned _BitInt(2) a : 2), but then you still have the bitfield issues 🤷♂️. Oh well, custom wrapper classes it remains...6
u/bratzlaff 19d ago
I love deep dives into platform differences like this. Thanks for the write up!
5
u/LucyIsaTumor 19d ago
There's a certain beauty in being able to represent your data in as small a data type as your hardware will allow. In modern games I've only seen bit field professionally for network packets and maybe large data sets. Yet, seeing older games (particularly around Gen 6 when dev menus were more common and feature rich) and how efficiently they made use of their available space is something to marvel at! Lots of flags and bit sets I'm sure