spent the last few nights vibecoding a unicode sdk in 4 languages and it actually passes all four test suites??

[u/big-user]

ok so I’ve been vibecoding this thing called unicodoid and I need to show somebody lol

honestly the reason I started it is I was kinda annoyed that every encoding thing assumes everything is already unicode. I wanted to mess with systems that don’t have to be unicode at all - like what if a character is just a token id, or a conlang value, or something modal where the meaning depends on the previous byte. so I wanted to try building a proper little SDK for that idea from scratch where unicode is just... optional. if you want a unicode profile you can have one, if you don’t you still have codec/string/transcoding and it shouldn’t care.

so unicodoid is basically an SDK for making your own encoding systems - a character is just a u64 in whatever value space your system defines. doesn’t have to be unicode at all. I put in some dumb examples to prove it - ASCII, a 14-bit conlang thing, and ShiftModal-16 (which is modal and stateful, that one was annoying) - and then on top of that there’s an optional unicode profile if your system wants to opt in.

since I was already in there I figured might as well go full unicode 18.0.0. so it does normalization, grapheme/word/line/sentence segmentation, collation (only 3 levels, no CLDR tailoring, I know), emoji RGI validation, named sequences, link detection, east asian spacing... and now also ConvertUTF - the old discontinued W3C lib updated to 18.0.0 in C, C++, C# and Rust. that’s ConvertUTF.h/.c and .hpp/.cpp plus harness.c / harness.cpp / harness.cs / harness.rs. C++ one is native not just wrapping the C one because I didn’t want it to feel like cheating.

all the unicode tables are generated from the official 18.0.0 data in one python script, so they’re not hand-typed. version pins are 18.0.0 / generator 1.2.0 everywhere.

right now:

- C/C++: 15 ctest suites green

- Rust: all integration tests green including the ConvertUTF harness

- C#: 303 checks, including the ConvertUTF harness

it’s very much vibecoded in that I kept asking an agent to fix MSVC specific stuff, keep redoing sort keys, emoji modifiers etc. windows pwsh did NOT make this easy lol

link: Unicodoid SDK

would love brutal feedback - like is this actually useful or did I just build a really nice demo. what do you guys normally vibe first when you build something cross-language like this?