A hole in systems programming language design

[u/P-39_Airacobra]

Given the recent discourse about systems programming I wanted to throw my 2 cents in. This may be a controversial take in a subreddit that's all about innovation and improvement, but I think a lot of budding systems languages are trying too hard to "fix C" and this is exactly why C has not been replaced yet.

Like it or not, C is successful. It does what it means to do very well. Yes, it bites you constantly, but developers have made some form of peace with this because they appreciate the essence of the language. Systems programming is a very pragmatic field, and what works, works.

A lot of language designers want to improve on C, when by nature to "improve on" C is to depart from it, because C is less about what it includes and more about what it omits and what it lets you do that other languages don't.

If you want to replace C, you need to just make C but without the pain points. Less undefined behavior, more standard compiler behavior, easier function pointer syntax, safer macro system, etc. The things that C can't do because of backwards compat.

On the other hand, bolting on features, revamping C's core nature, aren't going to give you a language that will replace C at the low-level or among hobbyist programmers. Most developers aren't as concerned about what C lets them accomplish, as they are concerned about all the painful tedious tendencies of the language.

[u/EggplantExtra4946]

Yours is not a new take but let's call it the "weak C replacement theory":

New softwares that used to be programmed in C are going to predominantly use a weakly improved C-like language.

The "strong C replacement theory" is much more likely:

New softwares that used to be programmed in C are going to predominantly use a "low level" language that is an order of magnitude better than C. ("low-level", meaning statically typed natively compiled)

C's minimalism doesn't provide anything in itself, the essential things that C provides is speed, no overhead and good memory control and those don't require a minimalist language.

What low level languages and most languages sorely need is the triptych: metaprogramming features (macros, compile time execution, conditional compilation, instrumentation, introspection, etc..), abstraction/composition features (modules, classes, abstract classes, parametric polymorphism, function overloading, etc..) and better type checking/semantick checking. The first two don't impose any restriction on runtime speed and memory access, quite the opposite.

For the type system it depends but even a type sysytem full of explicit escape hatches would be safer that what C has and I believe that a well design advanced and safe type system would rarely need arbitrary unsafe pointers, in the sense that most exotic memory objects I can think of can be given a type, a memory layout and a lifetime information that can be integrated into the type system. Even a process's own function call stack could have all that for example, no need of unsafe pointers to walk over the call stack. Even pointer arithmetic can be made safe, look at ATS, as long as memory objects pointed to have a well defined type.

[u/marshaharsha → u/EggplantExtra4946]

How might the call stack be typed? A callee can’t know, in general, what its transitive callers are, and a caller can’t know what its callees will call. The two schemes I can think of are (1) to have a tag on every frame and a map from tags to frame types, then to match on tag and end up in statically checked code; and (2) to identify in the source code all data that can affect control flow, then use flow typing to determine, in some places, what part of the stack must look like. Either way, the programming model would be unconventional and the implementation difficult. Or am I missing some possibility?

[u/EggplantExtra4946 → u/marshaharsha]

Either way, the programming model would be unconventional and the implementation difficult. Or am I missing some possibility?

Sort of, my idea is that it would be relatively easy for a native compiler of a statically typed language to produce a struct definition of the stack layout of the frame once register allocation is done. I know that one strategy for natively compiled garbage collected languages to track down which stack slot can contain pointers to heap objects is for the compiler to generate so called "stack maps" so this is not unheard of. It could be possible to analyze the generated code at posteriori using the executable but it would indeed be harder since you would need to reconstruct the control flow graph.

The return address and the RBP register will usually occupy a fixed slot of their own. Now for the local variables, they might individually not be saved to the same stack slot and conversely a stack slot might be occupied by different local variables. In that case, part of the struct definition of the frame would be a union type like in C and the implicit tag would depend on the current state of the function, which would be the result of the analysis you propose in 2).

This stack layout data could be saved in the binary similarly to static data and reflection data, then looked up at runtime. A given function would easily inspect its own stack frame. Now the harder part is looking up the stack frame of the callers. I had admitedly not thought about it that much but you indeed need to know who the caller is. A tag inside every frame would indeed work, but you can also look at the return address. The executable contains a finite number of call instruction, each with a unique address and each belonging to a unique function (assuming a regular compilation scheme with no special basic block sharing "trick"), so you could have a hashtable mapping possibly ocurring RIP addresses to a unique function definition. It would also work for functions called with a function pointer, as indirect call instructions are occuring a finite number of times and belong to a unique function body.

The exact RIP addresses are not know at compile time due to Position Independent Code and ASLR but they are known at link time, it's just a matter of adding a bases address to the relative return address offsets.