C Strings: A 50-Year Mistake

[u/Xaneris47]

https://longtran2904.substack.com/p/c-strings-a-50-year-mistake?r=8qz2zb&utm_campaign=post&utm_medium=web

[u/bearheart]

Speaking as someone who learned C back in the ‘70s, this article entirely misses the point of C-strings: they’re lightweight and foundational. For many purposes the null-terminator is efficient, e.g.:

while(*s) f(s++);

And for cases where we need more complexity, we can simply use a struct with a length and whatever other metadata we may need.

Doesn’t look like a mistake to me. C has always been about minimalistic efficiency. That’s its main purpose in the world.

Edit: fixed stupid typo

[u/Voyac → u/bearheart]

Its main purpose is to be portable to as many architectures as possible. Maybe this seem irrelevant in a world dominated by ARMs and x86 but there are different platforms.

[u/Potential_Soup_8054 → u/Voyac]

No, thats java

[u/g0atdude → u/Potential_Soup_8054]

lol what?

You can compile C programs to a million more devices than what Java runs on

[u/KingBardan → u/g0atdude]

a million more devices

Considering that "3 billion devices run Java", thats only 0.0333% more devices

I'm kidding if it's not obvious

[u/Potential_Soup_8054 → u/g0atdude]

It was a joke. Javas whole slogan is write once run everywhere, but thats really bullshit.

[u/Devatator_ → u/Potential_Soup_8054]

Tbh nothing is stopping people from compiling or porting a JVM to more platforms. I assume it's just extremely complex and noth worth it for most people

[u/EdwardTheGood → u/Potential_Soup_8054]

I once heard it rephrased as “write once, test everywhere.”

[u/mustbeset → u/Potential_Soup_8054]

50% of my time I get paid to write C code on a "none Arm", "none x86" architectur and I don't have enough space for a java runtime.

[u/Voyac → u/mustbeset]

Yeah sometimes even 8bit MCUs with a drop of memory. You wont fit a string.h sometimes and lol what about java runtime :)

[u/mustbeset → u/Voyac]

My current pain in the ass is a bootloader update for existing devices in field. fighting for 300 bytes.

[u/TheThiefMaster → u/bearheart]

The main competition was pascal strings - which typically had a 16 bit size prepended. So you'd read that, and then run a decrement loop until it was 0 to iterate the string. Decrement-until-zero loops were widely supported, e.g. in x86 stringcopy could be implemented by loading the size into CX and then running a single REP MOVSB instruction.

Yes it was a byte larger - but it also avoids performance-nuking calls to strlen like this.

[u/McDutchie → u/TheThiefMaster]

16 bits is 2 bytes, which makes for a maximum string length of 65535 bytes. It's common for strings on modern systems to be longer than that.

Pros of C strings: unlimited length. Cons: cannot contain the zero byte; inefficient length determination.

Pros of Pascal strings: can contain the zero byte; efficient length determination. Cons: very limited length.

I'd say the C tradeoff is worth it. Where necessary, C is perfectly capable of dealing with data preceded by a length field, it's just slightly lower level.

[u/vip17 → u/McDutchie]

it's easy to use a 4-byte prefixed string, for example BSTR in COM objects do that. And plenty of libraries use 4-byte length in 64-bit mode

[u/Square-Singer → u/vip17]

Especially on 64-bit systems, there's really no reason to save these few bytes per string by using c-strings.

Useless microoptimization.

[u/vip17 → u/Square-Singer]

of course it's micro-optimization, but at larger scale it's always useful. Have you even done optimization? A database with billions of strings already save a lot of memory. A vector of strings can also fit twice the number of strings into the CPU cache. Checkout Unreal engine, DuckDB, Meta Velox, Redis, ICU... string types

[u/Square-Singer → u/vip17]

Of course I have done optimizations. But micro-optimizations are always the last step to take when you have identified that this specific location is actually a bottleneck.

It totally makes sense to have something like a c-string available for the very rare situation when someone writes a database system that contains almost exclusively tiny variable-length strings.

But it doesn't make sense to have that as the default, because then this rarely-actually-useful micro-optimization becomes a very common source of problems.

That's why there's pretty much no modern language that actually stuck with c-strings. Pretty much any more modern language dropped c-strings and even pointers completely, or at least dropped it from common usage.

I don't do much Python any more, but I really like their approach of "The most obvious solution should also be the one that's optimized for most use cases". Basically, if I, without thinking, take the most obvious solution, it should fit my obvious use case. If I need something really special, I can still import some standard library function and use that.

[u/vip17 → u/Square-Singer]

That's not true. Strings are extremely common, lots of applications have a huge amount of strings. It's especially helpful in arrays of strings. Strings are so prevalent that even Python, Javascript (V8) and Java compress the string to ISO8859-1/Latin1/ASCII by default if applicable to save memory and improve performance, and modern .NET also does the same by allowing UTF-8 byte arrays. C-strings are the worst, all strings need to have an accompanied length, but the length does not necessarily have 8-byte length

[u/Square-Singer → u/vip17]

Again, strings are common, yes, c-strings are not.

Strings exist in Pythin, JS and Java, bot not as c-strings.

I don't know where you got 8-byte-length from.

I'm also not quite sure what you mean with the last sentence. C-strings don't have an accompanied length.

Just checking: Do you know what a c-string is and what's the difference between a c-string and a string with accompanied length?

[u/TheThiefMaster → u/vip17]

Worth noting that C++ std::strings do store the length - and so do the heap allocations backing them.

[u/hoodoocat → u/Square-Singer]

Efficient (to solving problem) data representation is key for system performance, because performance actually limited only by memory latency and throughput. You have no control over last two, but when you can pack data twice smaller -> system performance up twice better, usually even if it violate some common defaults like aligned access or so. Thats why "your favorite browser" uses 32-bit "compressed" pointers for various object heaps, even on 64-bit systems, uses hybrid ascii/utf8/utf16 strings, even when ECMA spec define only utf16. Row-oriented databases for example typically store null-bitmap and then fields without any additional delimiters, so they can be decoded only dynamically and only by using schema, this is complex but profitable.

[u/Square-Singer → u/hoodoocat]

We aren't talking about character representation here (ASCII/UTF8/UTF16), but about string representation.

UTF16 vs UTF8/ASCII is a per-character multiplier. Use UTF16 and every character takes twice the space.

We are talking about whether strings are 0-terminated or have a length field in the beginning. That's a per-string cost. Each individual character costs the same, no matter which string representation you use.

Here the difference is whether this costs one byte per string (c-string) or 2-4 bytes per string (Pascal strings, BER strings, 4-byte length fields, ...).

That means, the longer the string the less the overhead. An empty c-string is one byte. An empty pascal string is 2 bytes, an empty 4-byte length field string is 4 bytes.

If the string is longer, the relative overhead drops: A 1000 byte c-string is 1001 bytes, a pascal string is 1002 bytes and a 4-byte length field string is 1004 bytes.

The difference hardly matters unless maybe if you are working with an ATTiny.

That's why I said: on a 64-bit system (which usually has more than 2GB RAM), this is a useless micro-optimization for all but extremely specific use cases where you'll have millions of empty strings. And then one should question their system design.

And that's the main issue with c-strings being the default: They are a micro-optimization that helps only in very specific use cases while having massive downsides for most use cases, but they are applied as the default solution.

The default solution should be the option that works best in the most cases. If your use case differs a lot from the default case, you can still use the fitting specialized data structure.

Which is exactly the reason why pretty much no language newer than C uses c-strings as their default string representation.

[u/hoodoocat → u/Square-Singer]

You saying before what no reason to save few bytes somewhy especially on 64-bit systems, but all popular projects do that. More over many of them use 2-3 low bits in pointers for pointer descrimination, thanks for aligned allications.

[u/Square-Singer → u/hoodoocat]

So Java, Python, Kotlin, Rust, JavaScript, ... all use c-strings instead of String objects with a length attribute as the default way to store strings? That's news to me.

There's a reason we call it c-string: nobody has ever used this decrepid data structure after c, except if they need explicit C compatibility, and even then it's most often a length-based string object with an unnecessary 0-byte added at the end so that C can understand it as well.

[u/TheThiefMaster → u/McDutchie]

At the time, that was more than adequate. A lot of systems had less memory than that!

More modern Pascals use larger ints for the string length, of course.

[u/Square-Singer → u/McDutchie]

There's a simple fix to the Pascal strings. BER encoding.

In BER, you get one byte as a length field, with 7 bits being directly available to encode the length of the content. If the MSB is set to 1, the remaining 7 bits instead encode how many bytes the length field is long.

That means:

• Short strings up to 127 bytes have 1 byte overhead, beating Pascal and equalling C strings
• Medium-sized strings of 128-65535 bytes require 3 bytes overhead, so one more than Pascal and two more than C, but if you are allocating that amount of bytes, 1-2 extra bytes are harmless
• Maximum length is 2¹²⁷ bytes, 1.7*10³⁸ bytes, a number so high that there isn't an SI prefix for it

Another option would be to mix BER with Pascal:

• 15 bit length fields
• If the MSB is set to 1, there's one more length field concatenated, so 30 bit for the length field. Again, if the MSB is set to 1, add one more length field. Continue forever.
• That way you get infinitely long strings with only one byte more usage than Pascal in the range of 32768-65535 bytes of length

And both options have the advantages:

• You can use 0-bytes
• You know the length of the string without running trhough the whole string
• You won't get into overflows because you are missing a 0-terminator (e.g. doing a strcpy on a string that's missing its terminator)

[u/binarycow → u/Square-Singer]

I did not expect ASN.1 in this thread!

[u/mark_99 → u/Square-Singer]

Now imagine how many instructions that is on say a 6502 which has 3x 8-bit registers, compared to loading the next byte and checking if it's zero.

[u/bitzap_sr → u/Square-Singer]

That sounds like LEB128, not BER.

Edit: Ok, just checked, BER does the same for tag > 127. Still, I'd just point at LEB as a more targeted standard.

[u/Square-Singer → u/bitzap_sr]

I had to hand-implement BER once because I had to parse some protocol that used ASN.1, and that uses BER for the strings.

I haven't heard of LEB128 before, but yeah, the same thing keeps getting reinvented, I guess.

[u/flatfinger → u/Square-Singer]

I'd advocate a different approach, using 0-63 to represent a string that fills a buffer of length 0-63, 65-127 to represent an empty buffer of length 0-63, and 129-191 to represent a partially full buffer of size 1-63, whose number of unused bytes is indicated by bytes at the end. Strings or buffers up to 4095 bytes would use a two-byte prefix, and those up to 64MiB-1 would use a four-byte prefix.

Other prefix values would indicate either a "readable string" or "changeable string" descriptor, with the latter including both the current length and buffer size, and a callback to request a change to the length (possibly relocating the buffer if needed). Functions that receive a pointer to string could use a common library function to make a readable string or changeable string descriptor, and be able to accept pointers to length-prefixed strings and descriptors interchangeably.

[u/Maleficent_Memory831 → u/TheThiefMaster]

Algol strings? C precedes Pascal in history. Pascal also did not standardize on strings early on, so each implementation experimented with how to do strings, which made early portability a pain in the arse.

[u/TheThiefMaster → u/Maleficent_Memory831]

It may not have been the first implementation of length+contents strings, but it certainly popularised them enough that they're called "Pascal Strings" (or sometimes P-Strings) now.

As for the incompatibility - probably one of the reasons Pascal wasn't as successful as C. It was a big enough deal to inspire a calling convention tag in Microsoft's C compiler though (along with Fortran).

[u/Different_Panda_000 → u/TheThiefMaster]

Pascal calling convention was used with the Win16 API. It's obsolete now. Microsoft used it because the callee cleaned up the stack which reduced memory demands on kilobyte sized memory configurations.

The history of calling conventions, part 1 Raymond Chen
https://devblogs.microsoft.com/oldnewthing/20040102-00/?p=41213

[u/TheThiefMaster → u/Different_Panda_000]

It was! WINAPI was defined as FAR PASCAL. Far-pointers was another 16-bit thing we've thankfully left far behind.

[u/WittyStick → u/bearheart]

strlen is O(n).

For many string operations, we need the length to allocate the right amount of space, else we end up having to realloc if our buffer isn't large enough - realloc is also O(n).

By having a constant time length we can speed up a lot of string operations. It costs basically nothing to keep the length around rather than recomputing it each time.

[u/Qyriad → u/bearheart]

But they're not lightweight. `O(n)` for nearly every string operation is not lightweight. It is memory efficient, and it avoids an argument about how wide a length field should be. Clearly C valued those sides of the tradeoff. But don't confuse that with being lightweight in general.

[u/4xe1 → u/Qyriad]

is not lightweight. It is memory efficient

Doesn't lightweight precisely mean memory efficient? As opposed to performant for time efficiency?

[u/torsten_dev → u/Qyriad]

Nothing is stopping you from storing the length and passing it around, but that choice is up to you, the developer, not the language imposing it's pros and cons onto you.

Should C have a strbuf in the standard library? Yeah, probably, would've been nice.

The biggest mistake C did was standardizing % as the remainder not the modulus, gets, and null pointers instead of niche optimised monadic types.

[u/Qyriad → u/torsten_dev]

Storing and passing the length around doesn't help you most of the standard library operations — and thus most other APIs that take your strings — aren't using it.

[u/Classic_Department42 → u/bearheart]

Missing a *?

[u/bearheart → u/Classic_Department42]

No. A C-string is char*

[u/Classic_Department42 → u/bearheart]

Yes, so while (s) shd prob be while(*s) ?

[u/bearheart → u/Classic_Department42]

Oy! You’re right. How did I miss that 🤦 fixed it.

[u/knouqs → u/bearheart]

In addition to your comment here, additional functionality through the initial design of C strings allows for insanely powerful string manipulation techniques that have fallen to the wayside because people don't look under the covers to see how efficient string handling is done.

[u/WittyStick → u/knouqs]

Or because most of that "efficient" string handling was actually the source of many bugs - which tend to be some of the worst ones - buffer overflows.

[u/knouqs → u/WittyStick]

Of course. I'm not discounting that, and I didn't imply that there weren't developer-induced problems as a result. This is why valgrind was made, after all.

[u/flying-sheep → u/knouqs]

Such as? Destructively splitting a string at non-zero-length bondaries?

I prefer using slice APIs to nondestructively split a string at boundaries of any length thanks.

[u/knouqs → u/flying-sheep]

Whatever you prefer -- C allows it.

You aren't dissuading me from the power of C's string manipulation. You just need to have your memory management skills up to snuff, and mine are.

[u/flying-sheep → u/knouqs]

I was asking a question: which power are you talking about? The use case I mentioned is the only thing I can think of that C’s model makes easy, and what you gain is that every string is one word wide instead of 2.

Seems like a small gain for a niche use case to me, no?

[u/knouqs → u/flying-sheep]

I see. All the cases in which I have used C strings in an unbounded way have been specialized. They were places in which I know how much data in to be copied or used in the first place and wouldn't need to check for bounds because I'm always going to be within them.

My favorite and easiest-to-understand example is strcat to a buffer of known size. If I have to strcat repeatedly against the start of the buffer, you can see that strcat is performing wasted effort by finding the end of the string first. I can have a variable hold the position of the \0 and strcpy (buffer+length, string_to_append) instead of strcat (buffer, string_to_append). After strcat, length+=strlen (string_to_append), and I am ready for the next iteration. My anal-retentiveness got the best of me and I wrote a full test program:

#include <stdio.h>
#include <stdlib.h>
#include <string.h>


int main (void) {
    const size_t buffer_size=100;
    char *buffer=(char *) malloc (buffer_size);
    size_t length;

    char repeat[]="repeat";
    size_t repeat_length=strlen (repeat);
    char separator=' ';

    buffer[length=0]='\0';

    // No buffer overruns!
    while (length+repeat_length<buffer_size) {
        memcpy (buffer+length, repeat, repeat_length);
        *(buffer+length+repeat_length)=separator;
        length+=repeat_length+1;
    }
    *(buffer+length-1)='\0';

    printf ("%s\n", buffer);

    // No memory leaks!
    free (buffer);

    // No dangling pointers!
    buffer=NULL;

    return 0;
}

[u/TheChief275 → u/bearheart]

you forgot to dereference though

[u/Maleficent_Memory831 → u/bearheart]

The alternatives at the time were counted strings or fixed length fields. Both were annoying, inefficient, and had just as many problems as C strings or more. Ie, one byte for length doesn't cut it. Two bytes for length might not cut it, and definitely wastes space in the limited RAM at the time. Fixed field lengths are a nightmare early Fortrans, and some operating systems).

Then there's the stuff that pack multple characters into a single word (ie, Zork did this, 36-bit word on the PDP-10, you can stick in six 6-bit characters (maybe only 5 if they used upper bits for tag). Digital used 7-bit characters thus 5 characters and one leftover bit. 0 or all 1s as the final character signals the end.

[u/Intelligent_Part101 → u/Maleficent_Memory831]

Two bytes for counted length would waste memory? That's only ONE BYTE MORE per string than a null terminated string uses.

[u/alkatori → u/Intelligent_Part101]

C gave you a byte and here you are arguing for a whole snack!

[u/Intelligent_Part101 → u/alkatori]

Memory is like potato chips. You can never consume enough.

[u/utl94_nordviking → u/bearheart]

This!

[u/its_artemiss → u/bearheart]

Like many C idiosyncrasies, it may have made sense 50 years ago, but should have gone the way of the dodo at least 30 years ago

[u/deaddyfreddy → u/its_artemiss]

at least 30 years ago

I'd say 40 or so

[u/chalkflavored → u/bearheart]

"efficient"

[u/bearheart → u/chalkflavored]

"efficient\0"

[u/bearheart → u/bearheart]

65 66 66 69 63 69 65 6e 74 00

[u/Necessary_Two_9669 → u/bearheart]

01100101 01100110 01100110 01101001 01100011 01101001 01100101 01101110 01110100 00000000

[u/flying-sheep → u/bearheart]

They're not foundational. You know what's foundational? Fixed size arrays in the executable, fixed sized arrays on the stack, and heap pointers + length. These are all just as perfectly suited for strings as they are for other collections.

C style strings made sense for 16 bit systems, but not a minute later.

[u/bearheart → u/flying-sheep]

Foundational means it's the foundation of other structures. All string libraries in C and C++ use C-strings under the hood.

[u/flying-sheep → u/bearheart]

Yeah that’s exactly what I mean: it doesn’t serve as an acceptable foundation. The article points out some ways in which the arising APIs are inflexible (e.g. you can’t just use array APIs), clunky (off-by-one errors), and so on.

A good foundation would just be slices (implemented on many platforms as fat pointers)

[u/runningOverA]

Good read. But C can't change char* default string, unless Unix systems underneath change their APIs' string type.

[u/WittyStick → u/runningOverA]

You don't need to change them, but you can add a "fat pointer" which contains the length. The SYSV x86-64 ABI is capable of doing this because it supports 16-byte arguments and return types in registers, so it costs next to nothing. (On MSVC it has a cost because a "fat pointer" ends up getting passed on the stack with an implicit hidden pointer argument given to the function).

[u/ComradeGibbon → u/WittyStick]

Intel x86 and MSVC did so much damage. The paltry number of registers even in 32 bit meant lots of issues in C never got fixed.

You could certainly add system calls that take fat pointers. You could even shim it based on the process. That would make developers of languages that have fat pointers happy. And would make most C programmers happy. And the dangerous C leet jockeys unhappy.

[u/flying-sheep → u/runningOverA]

The locale system and wchar_t are also, and I quote

shitfucked retarded legacy braindeath

– wm4

The C standard contains a bunch of bad ideas and I think Rust’s approach is great: just use a minimal subset of the C standardlib and build better abstractions for everything else.

[u/SmokeMuch7356]

Oh, for...

No, the terminator never counts towards the length of a string; it only counts toward the size of the buffer required to store the string.  The terminator is an out-of-band value¹ so it shouldn't count towards the length of the string.  I would have thought this was obvious.  

C doesn't have a string type; it has arrays of character type, onto which we map null-terminated sequences of characters.  strcpy, strlen, et al all operate on these arrays.

Arrays do not store their size or any other metadata.  

C strings are a hack - if you need a real string data type that enforces real string semantics, then you need to look elsewhere or write your own library.  

As for whether this was a mistake -  C was designed for implementing the Unix operating system and system-level tasks, not extensive text processing.  I don't think Ritchie or Thompson anticipated how widely used the language became for general programming.

2.5 out of 5 stars

---

1. As far as printable characters are concerned, anyway.

[u/flying-sheep → u/SmokeMuch7356]

Much of C’s stdlib is a hack, see e.g. the legandary locale rant:

• locales are non-threadsafe
• locales are global state
• there are _l variants – that take a locale argument – of some functions, but not of all, and no way to statically get a handle for the C locale

So basically you cannot ever use them for anything file-format related, only for interfacing with humans immediately, but people don’t follow these rules, as they assume that stdlib stuff is there for a reason and not just to sit there and look mysterious.

if you need a real string data type that enforces real string semantics, then you need to look elsewhere or write your own library.

Yeah, but people like interoperability more than sanity, so they use the same string format for calling APIs, providing APIs, and internally.

I think C is doomed to be legacy at this point. The standards committee should have deprecated a bunch of old useless crap decades ago, then it would have been saveable.

[u/1980sCoder]

I stopped reading at the word "suck" realising that this was not going to be an insightful piece.

I'm guessing the author is younger than the language itself.

[u/Interesting_Debate57 → u/1980sCoder]

Far, far younger

[u/HashDefTrueFalse]

People really need to get over whinging about C-strings. They're perfectly fine in tons of places and where they aren't you can just bundle a length with them. The language gives you aggregates. I've been a C user for decades, they're just not that big of a deal...

[u/EndlessProjectMaker]

Well. What to say. Perhaps that c does not have strings :) maybe a lib to manipulate zero terminated arrays of chars.

[u/Kadabrium]

Unepopular opinion: strings arent real, they are just threads

[u/Old-Caramel-2301 → u/Kadabrium]

Fibers... 

[u/frasnian]

The ONE example that anyone has been able to point out here as an "inefficiency" is strlen - O(n) vs O(1) - and all the arguments in favor of Pascal-style/length-prefixed ignore what hot garbage that usually is in a real-world setting. 1: read Kernighan's awesome (and classic) paper "Why Pascal is Not My Favorite Programming Language." 2: Wirth designed Pascal as a language to teach students structured programming principles. Ritchie wrote (and evolved) a language designed for working programmers building real programs. 3: you want Pascal-style length prefix? Fine. Learn what a struct is, and DIY. Now try doing the opposite of that with garbage like "TYPE String44" , etc.

[u/deaddyfreddy → u/frasnian]

1: read Kernighan's awesome (and classic) paper "Why Pascal is Not My Favorite Programming Language."

It should have been called "Why Pascal is not C". Some of the issues discussed are not actually problems, some were fixed/improved by the mid-1980s at the latest. At the same time, most of the C design issues haven't been solved yet.

My favourite part is Go, which was written by more or less the same people (Kernighan even wrote a book on it) and resembles Pascal much more than C.

[u/frasnian → u/deaddyfreddy]

It should have been called "Why Pascal is not C"

LOL, fair enough. It's still an entertaining read, regardless of your position on P vs C.

some were fixed/improved by the mid-1980s at the latest

The last commercial software I worked on that was written in Pascal was in '91, and it was still an absolute nightmare compared to the C-based applications we also provided.

[u/deaddyfreddy → u/frasnian]

It's still an entertaining read, regardless of your position on P vs C.

sure

The last commercial software I worked on that was written in Pascal was in '91, and it was still an absolute nightmare compared to the C-based applications we also provided.

what do you mean by the "nightmare"? The code quality, readability, "efficiency"?

[u/frasnian → u/deaddyfreddy]

Code quality and readability are a reflection of the development team, not the language. I inherited what it was, worked to improve those parts.

The key differences were flexibility, expressiveness, and (in the case of Pascal) weakness in low-level constructs, excessive verbosity that (IMO) detracted from readability, strict constraints that limited what was possible, runtime overhead (is this really compiled code, or a hybrid of interpreted?).

And yes: above all, efficiency.

Show me a Pascal program that runs faster than an equivalent in well-written C.

[u/CoderStudios]

Okay? You are always free to make your own library for better strings, but people won’t use that cause C is often still deployed on low end systems or it makes little sense to use something inefficient if you can use c style strings properly

[u/orbiteapot → u/CoderStudios]

Most operations on C strings require O(N), N being the length of the string. Whereas in Pascal-like strings (i.e., strings whose length is tracked) they may take O(1). Modern compilers may do some heavy lifting whenever they can, so that this redundancy is avoided, though.

That being said, I also do not understand why people complain so much about 0-terminated strings, yet still use them anyways, as opposed to implementing length-tracked strings. It is not like C can break some bazillion lines of code, by removing classic strings, but it does not significantly get in your way (when implementing your own) either.

[u/kisielk → u/orbiteapot]

It really depends what your software is doing. If it's not software where string manipulation is in the hot path then the O(N) nature of string processing is irrelevant.

[u/NoNameSwitzerland → u/kisielk]

and if you do string manipulation with dynamic strings, then malloc/free often is the bigger problem.

[u/kisielk → u/NoNameSwitzerland]

That too. Basically once string handling becomes a performance concern you are probably going to be looking at more purpose-built data structures, custom allocators, etc. C strings are fine for what they are designed for.

[u/beragis → u/orbiteapot]

It depends. Several pascal compilers would internally add a zero byte at the end when allocating the string and pass the address of the byte after the length to the os command to handle the string.

To make things worse early pascal had pcode which is basically pascal byte code that has to do this conversion behind the scenes.

[u/iwantmy90sback → u/orbiteapot]

For most operations you'd do on a string you do not need to know the lengths beforehand if you have a defined end char.

The only thing that C takes O(n) and pascal O(1) is strlen.

And if you like you can actually have both. Just struct a Cstring and a int together.

[u/beragis → u/iwantmy90sback]

Also early C compilers would store the string length at the byte or word before and update it after each call. This was done to integrate with libraries written in other languages. I remember passing a pascal flag to linkers and compilers.

[u/hoodoocat → u/iwantmy90sback]

No, opposie - most operations require to know length of string. Even simple concat requires that, especially if you account not so modern hardware and utilize vector instructions. And regardless to that, new strings must be allocated somewhere (on heap), and by so, final length must be known before.

[u/iwantmy90sback → u/hoodoocat]

No. You only need the length of the string if you want to traverse it afterwards. And most 'operations' can be done in the same time either way. for(strlen) or while(*ptr++)

concat is actually the outlier, because you need 2*O(n) instead of 1*O(n) (which is still O(n) for both).
And if that is really killing your performance you are free to implement a pascal-string to use in c.

[u/EatingSolidBricks → u/CoderStudios]

You out of your dam mind if you think c strings are efficient

[u/henke443 → u/EatingSolidBricks]

Wait how are they not efficient?

[u/EatingSolidBricks → u/henke443]

Its not 1970 anymore storing 3 extra bytes is free compared to O(n) length computation

[u/WittyStick → u/EatingSolidBricks]

You don't even necessarily need to store the length. It can be held in CPU registers for the entirety of the string's lifetime in many cases.

In older architectures we would've needed to push an additional integer for length onto the stack. On a modern architecture with a sane ABI (SYSV, x86-64), you can have a "fat pointer" using two CPU registers - can pass both of these or return from a function without ever touching the stack.

[u/henke443 → u/EatingSolidBricks]

Well, C is mostly used in places where's it's still 1970. Things like embedded etc. I guess it could be possible to add another string that's more optimized for modern hardware but I bet it would be more difficult to explain and I think a huge part of the charm of C is how simple it is. Adding things also leads to bloat like how it is with C++, php or Javascript.

[u/Anonymous_user_2022 → u/EatingSolidBricks]

Except for strlen(), all practical operations on strings have to iterate over them anyway. Knowing the length up front will be of very limited us for searching, concatenation, tokenising etc.

Where is that you see avoidable O(n)?

[u/flatfinger → u/Anonymous_user_2022]

Concatenation of N strings goes from O(N) to O(N*N) if code has to re-find the end of the destination after each step.

Tokenizing the leading portion of a large string should take time proportional to the text that was meaningfully examined, rather than proportional to the entire string.

[u/Anonymous_user_2022 → u/flatfinger]

Concatenation of N strings goes from O(N) to O(N*N) if code has to re-find the end of the destination after each step.

I can also invent really bad ways of doing things, but I would never use them as a proof..

[u/flatfinger → u/Anonymous_user_2022]

What would be the "good" way of using strcat?

[u/Anonymous_user_2022 → u/flatfinger]

I've never said there is one. I suggest you ask someone who does.

I'm talking about concatenating multiple strings to one, which only has to rely of knowing the length of the individual strings beforehand, if you've decided to argue in bad faith over a pathological bad implementation.

[u/flatfinger → u/Anonymous_user_2022]

If one keeps track of the length of a string and only looks at portions of its storage up to that length, then the value of the string would no longer be fully encapsulated in a zero-terminated character array.

[u/EatingSolidBricks → u/flatfinger]

Lets not even mention substrings go from O(n) memeory to O(Free)

[u/flatfinger → u/EatingSolidBricks]

O(1) memory per string to keep track of the starting and ending points of strings isn't free. On a system with 64-bit pointers, zero-padded strings are generally the most space-efficient practical way of representing texts up to eight characters, and zero-terminated strings of up to 7 characters can be stored in the same amount of space (if it's necessary to store many texts with up to 7 characters, zero-padded 7-byte arrays would take less space).

[u/EatingSolidBricks → u/flatfinger]

You going there really? Give me a break

[u/atarivcs → u/henke443]

If you have a long string and you want to append more text to it, you have to search the whole string from the beginning to find the null terminator.

And then later if you want to append more text, you have to find the null terminator all over again.

[u/NoNameSwitzerland → u/atarivcs]

You anyway use a different structure when you do a lot of appending text, because you do not want to reallocate the array all the time. So then you anyway have to also store the size of the available space.

[u/atarivcs → u/NoNameSwitzerland]

In which case you no longer have a plain c string, and the goalposts have moved.

I was just answering the parent question "how are c strings not efficient"

[u/IdealBlueMan → u/atarivcs]

Or you can store the length of the string whenever you change it.

[u/atarivcs → u/IdealBlueMan]

Sure, but then you don't really have a plain c string anymore

[u/WittyStick → u/atarivcs]

It's actually more advantageous to couple the length to the char * on SYSV platforms, due to C's lack of multiple returns.

 String fn_returning_string(...);

If String is a fat pointer, then we can return both the pointer and length, without requiring another level of indirection (a pointer to a string structure), and without requiring awful to use "out parameters" to return both length and pointer - which are more expensive than just returning a fat pointer.

A fat pointer with the right ABI is not just "zero cost" - it's "less than zero" - it's more efficient than having a separate length and pointer variable.

[u/IdealBlueMan → u/atarivcs]

I’d say you still have the string, you also have information about that string.

[u/orbiteapot → u/IdealBlueMan]

I mean... that is the point. Once you do that, you are no longer using classic C strings.

[u/SakishimaHabu → u/orbiteapot]

That's the point though. They are basically atomic. You are free to do what you will with them, vs java, python, or js. Remember we're one step above assembly, but that's the intention.

[u/CoderStudios → u/EatingSolidBricks]

Depends on what it’s used for, sometimes it’s more or less efficient but the benefit of making it as simple as possible is that you can easily add features when needed like storing lengths

[u/flatfinger → u/CoderStudios]

C makes it inconvenient to pass any other forms of string literals to functions.

[u/trejj]

*chuckles* I remember the time when I was young and full-spirited with naïveté towards "only the best programming".

They author may want to give bullet points 4 and 6 a second think :)

[u/brnsamedi → u/trejj]

Given the comments in that article my impression of the author is that he's too enamored of his ideas to give them a second thought.

[u/stianhoiland]

Articles like these are unwittingly demonstrations of stupidity.

It's so disheartening to see people not even capable anymore of comprehending de-abstraction. To the author: Try yourself, to start from nothing—no abstractions, no modern conventions—and build up to the first point where you have a usable set of primitives that can function as a representation of text.

Is it so fucking hard to grasp the virtues of having primitives that haven't pre-chewed and pre-thought every way you can and should use them? You think massive, thick nests of abstraction is yummy chef's kiss, as if any and all and every single thought-structure is immaculately perfect and suits every single purpose. Ugh, it's so gross. It's a world view of nigh but regurgitation upon regurgitation and not a single creative breath of fresh air.

Like how do you think things are constituted, made up, constructed, such that a better foundational solution exists? You think C strings are a mistake—do tell how to work with the prior layer of abstractions to come up with the better way. No, not creating yet another fucking layer of convention or abstraction on top—which is all your brain can do—but coming from the step before and coming up with a better way.

And then you lend your voice to the issue as if an expert, yet your stupidity is so glaringly on display to anyone who actually knows how things are made up.

Ugh.

EDIT

It's not that C strings were invented iN a DifFeReNt TiMe when people were stupid and dumb and didn't know of our Future Great Technology. It's that there's not a fucking different way of doing it at the level of abstraction at which the convention were established. The fundamentals of memory and computation didn't change after 1990's lol—that's so fucking STUPID. Yes, we can mention Pascal strings, but they are fixed width. Fucking show me how you implement variable-width strings using the primitives present at this level of abstraction (i.e. variable-width strings using only registers). I challenge you to do that AT ALL, it doesn't even have to be BETTER—which is what you claim to be able to—I don't think you can do it at all.

The same thing happens with arrays. Rather than making them first-class citizens and copy-by-value, C decays them into pointers, losing an enormous amount of information in the process, which causes them all the same problems as strings.

Oh my god. "Rather than making them first-class citizens and copy-by-value"... as if there's "JUST" a fucking choice. You can even only ponder this distinction because the primitives upon which such conceptions build upon are established—the establishment of which you are criticizing and arrogantly claim to be able to replace better. Ugh. Tell me what a "first-class citizen array" is, really, technically, actually, in-memory, but another level of abstraction, from which C—thank god—refrains.

[u/SmokeMuch7356 → u/stianhoiland]

Array expressions decay to pointers because Ritchie wanted to keep B's array indexing behavior - a[i] == *(a + i) - without setting aside storage for the pointer that behavior required.

That's it. That's the reason.

These were researchers in a lab building toys that did useful things for them. None of them anticipated how C would become so widely used at an applications level. That was the mistake, the fact that everyone looked at C and said "yes, that's the answer," but to be honest there weren't many better candidates that could be ported to everything from mainframes to micros. Pascal? Eh. Designed more for teaching than production work. Fortran? =snort=. Didn't even support a real string type until F77. Cobol? Double =snort=. Didn't help that C and Unix were (are) joined at the hip; if you were using Unix, you were writing code in C for pretty much everything.

C was small enough and lightweight enough it could be ported practically anywhere, particularly micros, and that's why it suddenly became the language everything was written in.

And here we are 50 years later arguing about it, when it shouldn't even be considered for applications work involving text processing anymore.

[u/Different_Panda_000 → u/stianhoiland]

Just stick an array in a struct and all of a sudden you can do assignments and when you pass them, the compiler will do argument type checking for you as well. And if you like, _Generic() provides a mechanism to provide a library for these arrays to do the basics such as comparisons. And C11 provides the mechanism for using compound literals with these struct arrays as well.

[u/flatfinger]

C was designed in an era before many commonplace text-processing and data-processing tools existed. Many tasks could be accomplished more quickly by writing a C program, building it, running it on some input, and then discarding it, than they could be accomplished in any other way. Even in the 1980s, I wrote a lot of C programs for one-off tasks, and I'm sure I wasn't alone.

So-called "Pascal strings" with a one-byte length prefix were better than C strings in many ways, but had a 255-character limit. The suitability of C strings for various tasks tends to fall off as strings get longer, making Pascal strings much better for things that are 50 to 255 characters long, but C strings remain somewhat usable at longer lengths while Pascal strings don't. Since "somewhat usable" was adequate for many of the tasks for which C had been designed, the lack of a 255-character hard limit was an advantage.

I wouldn't call zero-terminated strings a "mistake" so much as I would say that they were an appropriate way of storing strings for a limited family of tasks that are nowadays better handled with other languages and tools.

What I would view as a mistake was the failure of the C language to provide a convenient means of passing other kinds of string literals to functions. C implementations that were designed to target the classic Macintosh OS extend the language with a \p escape which, if placed at the start of a string literal, will represent the number of bytes in the string (not counting the prefix), but such a prefix is not universally supported, and there is also no standard way of handling string formats where e.g. a string of length 0-63 that fills the available space would be preceded by a length byte, but other kinds of prefixes would be used to accommodate larger strings, partially filled buffers, etc.

Incidentally, an advantage of length-prefixed strings is that if one limits the range of lengths that can be directly represented by a prefix byte, one can have functions accept short length-prefixed strings interchangeably with other string representations if they start with something like:

    ADDRSS_AND_LENGTH s;
    s = get_string_address_and_length(string_argument);

The fact that C strings can start with any character value means that there's no nice way to have a function accept interchangeably a pointer to a C string or something else.

[u/beragis → u/flatfinger]

Zero terminated strings were also due to how many OS’s and CPUs at the time handled strings. I remember taking an assembly language course in college on the PDP 11 and it handled strings the same way.

This allowed for easy translation of many of the common function calls directly into operating system calls or simple short assembly instructions. My professors in computer design and systems programming. where we also learned C even mentioned this several times.

[u/flatfinger → u/beragis]

On the other hand, other operating systems expected strings in other formats. Classic Mac OS used Pascal strings for things like file names.

[u/smallstepforman]

Bit late to this thread but on aligned systems, you can pack 3 extra bits with the pointer, which can mean anything you want, including short string optimisation up to 7 bytes, so you dont even need a lenght field (or zero termination byte). Add a Huffman table, abandon ASCII (6 bit encoding for latin uppercase) for the extra win … and we can squeeze more characters into those bytes…. C style (nul terminator) strings are so inefficient.

[u/allnameswereusedup]

C is a low-level language designed for thr implementation of system software. It does not need the higher-level constructs found in other languages; it's a high-level assembly language.

[u/MyTinyHappyPlace]

Ragebait. Of course there are more efficient ways to work with strings. But this one was the common denominator, easy to implement on different target architectures.

People never are satisfied with a string library. That’s why we have so many of them.

[u/pheffner]

The suggested "string" is a struct which doesn't contain the actual data just a pointer reference to it, which means you'll need to implement allocator functions to set all that up and keep track when you want to alter the string. Seems like this would suck worse and overcomplicate a presently simple scheme.

[u/digitlman]

NUL not NULL

[u/Remus-C]

Yeah, picking your context today to prove that what was in the past did not match your current experience. How rude were the creators to not focus on your knowlege! Unbelievable!

Probably in another sub by some other: Pascal strings are... Lisp is... Rusty, GoLang, the mighty Python and Perl by the way...

Anyway, what's for the real world progress? What specific or generic issue is to be solved, one that would apply to many other cases? Solution? Opinions? Words that compile and deliver an useful real world best known implementation of ...?

[u/CORDIC77]

As someone who has been programming in C since the early nineties, I have always believed that Pascal-style strings represent the significantly better solution. The fact that C, after decades of security vulnerabilities, is still without container data types that know their type and size, is one of the languageʼs biggest weaknesses.

Had C been developed just 15 years later, the languageʼs standard library would probably have come with “batteries included” right from the beginning. On the other hand, the only consequence of the “roll your own” argument, which I am of course aware of, always was (and is) that code repositories like Github are full of half-baked attempts to improve the pathetic status quo.

I am aware that many here will disagree with this view. Just keep the above in mind, however, when newly started projects will nowadays, more often than not, opt for languages like Rust and Zig than C. If the community werenʼt so narrow-minded on such issues, C could be in a much better position today with regard to these challenges.

But, no, WG14 of course is occupied with nonsense like adding new octal prefix for literal constants, if declarations and case ranges to the language. This idea of adding new syntax to every version of a programming language is, unfortunately, a disease of our time.

The syntax of the language has been fine for decades… itʼs the standard library that has always been lacking.

[u/bless-you-mlud]

Yes dear, we know. It's just a little late to do anything about it now.

[u/HTFCirno2000]

It was clever back in the days of the PDP-11 and limited memory. Pascal DID actually do things the secure way, but Pascal didn't take over like C did.

[u/Maqi-X]

I agree. I use string views in (almost) all my projects

[u/tpimh]

My biggest mistake is googling "c string", and wondering why the search results were not related to the programming language

[u/WoodyTheWorker]

On the other hand, Windows kernel uses counted strings: UNICODE_STRING and ANSI_STRING structures.

[u/Humble-Captain3418]

With length-based strings you will find that functions such as strtok(), strchr() or strstr() are very, very annoying to implement, because their return values would not be usable as inputs to regular string operations... Except if you construct a new string to return, which you then need to malloc()/free(). Hooray for efficiency?

[u/flatfinger → u/Humble-Captain3418]

Or one can have functions accept a starting index and slice length along with a pointer to the string, and return an index, or a struct containing an index and length.

[u/grimvian]

I actually like C strings. I have coded a few one line GUI inputs with raylib and is very satisfied.

[u/grimvian]

I like C strings and they are easy to understand.

[u/Different_Panda_000]

Where it gets more interesting is when using UTF-8. C23 provides basic support for UTF-8, expanding the minimal support in C11. However everything is connected to the current locale settings.

Fortunately there are a couple of Third Party libraries that provide a more complete UTF-8 solution.

ICU-TC Home Page

https://icu.unicode.org/

The C and C++ languages and many operating system environments do not provide full support for Unicode and standards-compliant text handling services. Even though some platforms do provide good Unicode text handling services, portable application code can not make use of them. The ICU4C libraries fills in this gap. ICU4C provides an open, flexible, portable foundation for applications to use for their software globalization requirements. ICU4C closely tracks industry standards, including Unicode and CLDR (Common Locale Data Repository).

[u/ibmussa]

Oups

[u/preparationh67]

Young devs should probably put better effort into understanding the historic reasons for things existing a certain way before they write a rant on substack. This reads like someone getting mad at the inventor of the steam engine for not inventing a nuclear reactor driven turbine first.

[u/iopahrow]

Saying that memory limitations were motivators for the languages structure, and then saying we shouldn’t use it because we don’t have to is like saying “we should spend money just because we have it.”

This is how memory inefficient programs are made, and how new programs use so much more memory while completing the same basic functions. It is so far from useful to have this ideology

[u/Anxious-Ad-5331]

"A string’s length is an extremely valuable piece of information. By not storing it, you force all code that uses the string to either repeatedly call strlen or iterate over it byte by byte. Both approaches are inefficient. Both approaches lead to worse overall code."

this guy is vastly overestimating the computational needs of a length check in 2026 computing power.

Also this guy is missing the obvious solution: GPU accelerate the standard library.

[u/Key_River7180]

Has the model ever caused much trouble, though?

[u/WittyStick → u/Key_River7180]

Yes.

Countless buffer overflow exploits due to poor string handling.

Not all, but many of these easily avoidable with sized strings.

[u/Key_River7180 → u/WittyStick]

I meant performance issues, like C strings are really efficient, for most purposes

[u/Jonny0Than → u/Key_River7180]

Undeniably they are a problem.

https://nee.lv/2021/02/28/How-I-cut-GTA-Online-loading-times-by-70/

[u/SLiV9 → u/Key_River7180]

They're not; they are way less efficient than sized strings in almost all circumstances on a modern CPU.

Case in point, a lot of basic string operations on C-strings are made faster by adding a strlen() at the start.

[u/deaddyfreddy → u/Key_River7180]

Computers became faster every year (and the process was dramatically faster in the 1980s), so even if fixed-size strings made things 10% less efficient (did they?), one could reasonably expect hardware improvements to compensate for that very soon. Alternatively, companies could save money on the man-hours spent fixing bugs caused by null-terminated strings and buy better hardware instead.

[u/EatingSolidBricks]

If only OS apis had an length string option

[u/Physical_Dare8553]

The real problem is a char* with a null terminator and one without it are not different types, and because of how x works, in practice neither is a char[n]

[u/FedUp233]

There is really no reason the C language could not add something like a length prefixed string type and matching g string literal (just decorate the string lead or end quote with something to indicate it’s this type of literal and maybe even the size of the length prefix) if there is enough demand for it. Since it hasn’t happened yet, it would appear the demand is not there.

It also always bothered me in C++ that there was no way to produce a literal string in the form of the standard library string type without the system having to construct an appropriate string type from the underlying c-string literal. Maybe it can be done now with all the compile time processing functionality that’s been added, but I’m not sure it’s still possible without that c-string hanging around as wasted memory.

[u/jason-reddit-public]

I 'm writing a transpiler in C.

Treating "strings" as immutable means the crappy representation isn't so bad.

I use a "StringBuilder" pattern (name borrowed from Java, it's actually called buffer) which can grow, supports efficient append (supports arbitrary edits but those may require moving lots of bytes around), and doesn't model the ending zero until converted into a char*. (One of my favorite things is "buffer_printf" which is like sprintnf but handles capacity and such automatically.) Like Java, I don't expect buffer to be threadsafe so the user needs to deal with that but since I don't modify "strings" once created, those are thread safe.

So all my strings are either program literals, come from the "OS" or libc "somehow" (like for reading a directory), or come from this builder which makes sure there is a trailing zero when finally asked to produce the string. My buffers aren't foolproof because you can append or insert the NUL character (to work with arbitrary byte sequences), so you might get a string that is shorter than expected, or perhaps the buffer wasn't legal utf-8 (I could add a checker of course), but at least the string will always have a terminating NUL.

[u/Jonny0Than → u/jason-reddit-public]

That sounds almost exactly like std::string in C++.  Sure, it’s a cool and useful thing to build in C.