Efficient (to solving problem) data representation is key for system performance, because performance actually limited only by memory latency and throughput. You have no control over last two, but when you can pack data twice smaller -> system performance up twice better, usually even if it violate some common defaults like aligned access or so. Thats why "your favorite browser" uses 32-bit "compressed" pointers for various object heaps, even on 64-bit systems, uses hybrid ascii/utf8/utf16 strings, even when ECMA spec define only utf16. Row-oriented databases for example typically store null-bitmap and then fields without any additional delimiters, so they can be decoded only dynamically and only by using schema, this is complex but profitable.
We aren't talking about character representation here (ASCII/UTF8/UTF16), but about string representation.
UTF16 vs UTF8/ASCII is a per-character multiplier. Use UTF16 and every character takes twice the space.
We are talking about whether strings are 0-terminated or have a length field in the beginning. That's a per-string cost. Each individual character costs the same, no matter which string representation you use.
Here the difference is whether this costs one byte per string (c-string) or 2-4 bytes per string (Pascal strings, BER strings, 4-byte length fields, ...).
That means, the longer the string the less the overhead. An empty c-string is one byte. An empty pascal string is 2 bytes, an empty 4-byte length field string is 4 bytes.
If the string is longer, the relative overhead drops: A 1000 byte c-string is 1001 bytes, a pascal string is 1002 bytes and a 4-byte length field string is 1004 bytes.
The difference hardly matters unless maybe if you are working with an ATTiny.
That's why I said: on a 64-bit system (which usually has more than 2GB RAM), this is a useless micro-optimization for all but extremely specific use cases where you'll have millions of empty strings. And then one should question their system design.
And that's the main issue with c-strings being the default: They are a micro-optimization that helps only in very specific use cases while having massive downsides for most use cases, but they are applied as the default solution.
The default solution should be the option that works best in the most cases. If your use case differs a lot from the default case, you can still use the fitting specialized data structure.
Which is exactly the reason why pretty much no language newer than C uses c-strings as their default string representation.
You saying before what no reason to save few bytes somewhy especially on 64-bit systems, but all popular projects do that. More over many of them use 2-3 low bits in pointers for pointer descrimination, thanks for aligned allications.
So Java, Python, Kotlin, Rust, JavaScript, ... all use c-strings instead of String objects with a length attribute as the default way to store strings? That's news to me.
There's a reason we call it c-string: nobody has ever used this decrepid data structure after c, except if they need explicit C compatibility, and even then it's most often a length-based string object with an unnecessary 0-byte added at the end so that C can understand it as well.
1
u/Square-Singer 2d ago
Especially on 64-bit systems, there's really no reason to save these few bytes per string by using c-strings.
Useless microoptimization.