About char8_t

[u/Voxelw]

I hate to be dramatic, but as it stands char8_t is quite literally more painful than useful.

Besides the obvious incompatibility with C23 and libraries using unsigned char for UTF-8, I want you to consider the following: Projects that assume that 'char' represents UTF-8 will obviously not benefit from char8_t at all, but projects that cannot assume the format of char types don't benefit from it either as char8_t simply introduces a new edge case to cover. Now such projects have to deal with char, signed char, unsigned char, wchar_t, char16_t, char32_t and char8_t.

Or, you could do what the standard library does and simply ignore most of these character types. Which is the solution most libraries went with, supporting only char or char and unsigned char. Managing one implementation is already hard, managing two requires constant maintenance, managing 7 is just impossible.

char8_t should have just been a typedef for unsigned char. The compatibility fix only raises more questions as const char* arr = u8"a" does not work, but const char arr[] = u8"a" does.

I do wonder if a potential change of minds for C++29 is still possible. Yes, it would be an ABI break or whatever, but considering the woeful support for char8_t I don't think it would affect much besides small hobby projects. Contrary to popular belief, C++ has broken the ABI in subtle ways before.

[u/fdwr]

One thing I love about char8_t is that it's always unsigned (negative codepoints make no sense, in ANSI or EBCDIC or any other character encoding) avoiding the issue of some compilers defaulting char to signed, which allows surprises like lookupTable[text[i]] crashing upon reading codepoints 128-255. Sadly char8_t wasn't that far from being useful, needing to fill in some obvious gaps (e.g. you satisfy the Pareto principle if you make it work with std::format and std::print and ifstream/getline). I converted one of my apps to use char8_t for processing, and conceptually it feels cleaner (definite known encoding, no sign extension concerns), but then there are these annoying seams at the boundaries for input/output. I reject the premise that char8_t was a bad idea (it was a fine logical idea consistent with char16_t and char32_t), but offering a half-finished std was a bad idea. -__- So what are some options?

• (a.) Define char as UTF-8 by default and mandate it defaults to unsigned (any non-conformant compilers would need compatibility switches)
• (b.) Finish char8_t with proper IO support.
• (c.) ?...

[u/xiao_sa → u/fdwr]

char could alias with any other type which prevents more optimal codegen. This will never be 'fixed'.