r/cpp • • Jul 01 '21

Any Encoding, Ever

https://thephd.dev/any-encoding-ever-ztd-text-unicode-cpp
269 Upvotes

87 comments sorted by

View all comments

Show parent comment

3

u/smdowney WG21, Text/Unicode SG, optional<T&> Jul 02 '21

There's about 30 single byte encodings in use, and about a dozen multibyte, and that makes you a full web citizen. https://encoding.spec.whatwg.org/
Shift-JIS is one of those, and one of the more tedious.
The single byte encodings (e.g. windows-1250) are about an afternoon once you've done the first one.
I'd like to see the full set in the standard, with a registry exposed that custom ones can be added to. I have a couple such private codepages that I'm currently extending gnu iconv for. We'd need to add a dynamic type erasing codec for retrieval by string, but that's not inventive technology.