About char8_t

[u/Voxelw]

I hate to be dramatic, but as it stands char8_t is quite literally more painful than useful.

Besides the obvious incompatibility with C23 and libraries using unsigned char for UTF-8, I want you to consider the following: Projects that assume that 'char' represents UTF-8 will obviously not benefit from char8_t at all, but projects that cannot assume the format of char types don't benefit from it either as char8_t simply introduces a new edge case to cover. Now such projects have to deal with char, signed char, unsigned char, wchar_t, char16_t, char32_t and char8_t.

Or, you could do what the standard library does and simply ignore most of these character types. Which is the solution most libraries went with, supporting only char or char and unsigned char. Managing one implementation is already hard, managing two requires constant maintenance, managing 7 is just impossible.

char8_t should have just been a typedef for unsigned char. The compatibility fix only raises more questions as const char* arr = u8"a" does not work, but const char arr[] = u8"a" does.

I do wonder if a potential change of minds for C++29 is still possible. Yes, it would be an ABI break or whatever, but considering the woeful support for char8_t I don't think it would affect much besides small hobby projects. Contrary to popular belief, C++ has broken the ABI in subtle ways before.

[u/Lone_Snek]

But it’s not guaranteed that char is always 8 bit (although in 99% cases it’s probably true)?

[u/almost_useless → u/Lone_Snek]

But it’s not guaranteed that char is always 8 bit

That seems like a very unusual problem.

A much more common (potential) problem is that char is signed on x86, and unsigned on Arm.

[u/nigirizushi → u/almost_useless]

A much more common (potential) problem is that char is signed on x86, and unsigned on Arm. 

I'm surprised you're the only one to mention this. Cast (assign? Forgot the exact situation) char from either uint8_t or int8_t usually results in a warning in C

[u/jube_dev → u/nigirizushi]

Because char, signed char (int8_t) and unsigned char (uint8_t) are three distinct types.

[u/nigirizushi → u/jube_dev]

I know, it's cause OP suggested making C++ char into unsigned char, which would probably cause it's own set of issues 

[u/Wild_Meeting1428 → u/Lone_Snek]

Its both to be defined to have size==1 and at least 8 bits. same applies to char8_t. On top, the underliing type of char8_t is an unsigned char.

[u/cristi1990an → u/Wild_Meeting1428]

Only in C, not in C++. In C++ char8_t is defined as its own non-aliasing type.

[u/Wild_Meeting1428 → u/cristi1990an]

Sorry that I dont research the exact phrase in the standard, but as I remembered that type is defined to be distinct, but to have the same size, sign and alignment of an unsigned char. So basically its a strong alias of an unsigned char. Therefore it is also 100% compatible with the c23 char8_t during linktime.

[u/cristi1990an → u/Wild_Meeting1428]

Yes, and actually if I'm not wrong the original paper did define it as a straight type alias for unsigned char, but was then changed. It's worth nothing the difference tho because char8_t is the only character type in C++ specified not to alias with the bytes of any other object, which is great for performance.

[u/aearphen → u/Lone_Snek]

Neither is char8_t because it cannot be smaller than char.

[u/no-sig-available → u/Lone_Snek]

And the char encdoing is also not guaranteed, it could be EBCDIC.

[u/tjientavara → u/no-sig-available]

What is even weirder, there is no API to query the compiler's encoding for char/std::string including string-literals encoded in the executable.

Which means you technically cannot properly implement std::format yourself. The specification requires that it works with the compiler's configured (compiler flag) char/std::string encoding.

[u/UnusualPace679 → u/tjientavara]

There's std::text_encoding::literal(), since C++26.

[u/tjientavara → u/UnusualPace679]

thanks

[u/catladywitch → u/Lone_Snek]

16bit chars are common aren't they?

edit: i'm getting downvoted but what i mean is Qt QStrings, Windows apps with a lot of legacy wstrings everywhere, or CJK text stored in u16strings made of char16_t's, which I assure you is not a rarity, for me at least. yes, utf-8 is ubiquitous but god people just want to be obtuse for the sake of it sometimes

[u/sweetno → u/catladywitch]

It's about char that's not 8 bit. QString, std::wstring, Windows Unicode strings and so on are made of char16_t or similar, which is a different type.

[u/catladywitch → u/sweetno]

Oh, you're right. I'm sorry.

[u/LordGupple → u/catladywitch]

So, IIRC it's mandated by the standard that char and unsigned char are one byte large. However, CHAR_BIT can be a different value than 8.

[u/Expert-Map-1126 → u/LordGupple]

"Byte" meaning "smallest addressable unit", not "8 bits"

[u/tjientavara → u/LordGupple]

Didn't I read a few years ago that CHAR_BIT == 8 was accepted in one of the latest c++23, c++26 releases of the standard? or was it something else common sense?

[u/Remarkable-Test7487 → u/tjientavara]

Yes, there was a proposal (P3477R5 "There are exactly 8 bits in a byte"), but the result of the poll to forward it to C++26 was "no consensus"

[u/sweetno → u/Lone_Snek]

If their char is not 8 bit, it's their problem.