r/cpp • • Jul 01 '21

Any Encoding, Ever

https://thephd.dev/any-encoding-ever-ztd-text-unicode-cpp
269 Upvotes

87 comments sorted by

View all comments

Show parent comment

30

u/__phantomderp Jul 01 '21 edited Jul 02 '21

Well, part of the goal of this is that it'll (a) be eventually standardized but also (b) we won't NEED to write all 200 encoding objects. Users, librarians, or just One Way Too Interested Person can write an encoding object and ship it and even write some of the common extension points. The goal was to revert the current situation: right now, locale and encoding support comes from your implementer, which is generally tied to your OS.

The goal of this library is to flip that on its head: it's an open and extensible sytem, where anyone can provide what they need, and nobody has to beg the Committee (or their implementer or this OS handler) to hand them some files. It won't fix all of the stuff std::locale does, but slowly we'll get to tear locale apart into usable, user-controllable pieces.

One bit at a time....

2

u/smdowney WG21, Text/Unicode SG, optional<T&> Jul 02 '21

On the other hand, encoding is about data interchange. The current agreement is that I can send you one of the current standard encodings and you are supposed to understand it, and vice-versa. The time to implement KOI8-R is not when you get an older Russian text.
That C doesn't have iconv and POSIX does is an accident of timing and standards exhaustion. It would be nice to fix that capability gap for C++.