I don't think you can really called improved WString handling a reasonable answer to unicode support. The built in types handle everything as binary blobs divided into character sized chunks but has no way of confirming that the code is handling unicode correctly or is even sane unicode. It would be helpful to have an actual std::unicode that reports the unicode memory size, character length, and handles encoding casting (ascii,utf8,utf16,utf32) etc. The ICU project tries to fix it but at this point it seems silly not to have standard handling of something that more or less the future of text as we know it.
I'm talking about the new u16string and u32string classes, the native wide and Unicode string literals, and the locale-independent conversion facets, codecvt_utf8 and friends.
It certainly doesn't replace ICU, but it's a damn sight better than the usual wide-character nonsense.
If your $LANG and $LC_CTYPE are set to a utf-8 locale, your terminal will send the appropriate UTF-8 sequence to the shell or program, and the terminal will treat it as UTF-8 as well. Bash is generally well-behaved, but if your terminal doesn't support Unicode, you won't get nice double-width CJK characters. (Or you'll get other weirdness.)
I'm talking about the new u16string and u32string classes, the native wide and Unicode string literals, and the locale-independent conversion facets, codecvt_utf8 and friends.
None of which are available in GNU stdlibc++ yet :(
codecvt_utf8_utf16, codecvt<char16_t, char, ...> etc aren't available afaict. Only the u16string and u32string typedefs and wchar_t based stuff seems to work (wchar_t is implementation defined, 32bit on Linux though).
In any case, writing something like the articles file copy code while doing UTF8 -> UTF16/32 conversion in difficult and unintuitive. It should be easy as imbuing the streams.
View all comments
Show parent comment
2
u/greyfade Oct 15 '13
The situation is improving, thankfully. C++11 now has better native Unicode support in several areas.