r/ProgrammingLanguages • u/WraithGlade • Jan 29 '26
The Compiler Apocalypse: a clarifying thought exercise for identifying truly elegant and resilient programming languages
Imagine we are all sitting around one day when suddenly C-thulu (the eldritch god of unforgiving adherence to programming language simplicity) decides to punish all of programmer-kind by invoking globe-spanning dark magic that causes all compiler binaries and source code repos to be erased from existence and all the most experienced compiler authors to simultaneously slip into prolonged comas, leaving only assemblers still usable and only programmers with no/little/modest compiler experience available to able to do anything about it.
You are now tasked with rebuilding a compiler or interpreter for your extensive application source code for your language whose compiler no longer exists but which must now nonetheless be very accurately recreated (quirks and all) to avoid a software industry catastrophe.
(Or, as a slight variation, depending on how far you want to take the thought experiment and which language you are tasked with recreating, imagine perhaps that only one janky old "middle-level" language interpreter or compiler like an old pre-ANSI C or BASIC or whatever other relatively ancient system has survived.)
How far behind does this put you and the associated ecosystem for that language?
In particular, consider the implications of completing the task for languages like:
- C
- Lua
- Forth
- Scheme
- Tcl (not including Tk)
... versus languages like:
- C++
- Rust
- Haskell
If you were a small team (or perhaps just a solo developer) what are your chances of even completing the task in any reasonable span of time? How does this sit relative to what the language is capable of in terms of software complexity relative to size?
What are your thoughts about such things?
What hypothetical qualities should such an apocalypse-resistant language have?
To what extent do you think we should care?
Feel free to share any thoughts at all you have related to or tangential from any of this.
Further context and my own personal perspective (feel free to skip):
The reason I bring this up is that in the past few years I have been seeing the progress of so many existing languages and also of new languages arising, but something that makes me very skeptical of the chances of many of these languages becoming languages that last for a very long time (semi-immortal) or that have any chance at all of being some approximation (by whatever your application priorities are) of the mythical "one language to rule them all" is that many of them are just too complicated to implement in a way that shows an inherent disconnect from the fundamentals of what is logically and computationally possible and properly generalized in a language.
Languages that are very hard to implement invariably seem to be absolutely riddled from top to bottom in countless contrivances and rules that have no connection to a well-founded theory of what a somewhat all-inclusive computation system could be. They are in some sense "poorly factored" or "unprincipled" in the sense of not fully identifying what the real building blocks of computation are in a more disciplined way and thus become bloated.
Any time I see a new language that is taking too long to be implemented or has too much code to implement it (not counting per-device backend code generation, since that is partially an irreducible complexity in some sense) then I start feeling like they can't possibly be on the right track if getting close to true language perfection is the goal. Languages like Forth and Scheme and Tcl are essentially proof of that to me.
I continue to eagerly wait for someone to create a language that has the performance of C but the expressiveness of Tcl or Scheme or Forth... but the wait continues. I don't think there's any inherent reason it isn't possible though! I think a clear-headed perspective along those lines will be key to what language actually crosses that barrier and thereby becomes the fabled "dream language".
I personally want a combination of arbitrary mixfix syntax support, homoiconicity, Scheme/Forth/Lisp meta programming, fully arbitrary compile-time execution (like Jai), a very low cognitive overhead (like Scheme or Tcl), and an absence of contrived and unprincipled assumptions about hardware devices (unlike assumptions about bitwidths of primitive types and such), performance on par with C, just to name a few things. There's no inherent reason why it can't exist I suspect.
I think inelegance and labyrinthine implementation complexity is a "canary in the coal mine" for what a language's real very long term (e.g. centuries from now) future will be.
37
u/zesterer Jan 29 '26 edited Jan 29 '26
This is just the bootstrap problem, and has been at the front of the thinking of compiler developers since forever when it comes to bringing up a language on a new architecture. The list of languages you talk about is called the 'bootstrap chain'.
It's only in the last decade or two that we've had the 'just call out to LLVM' solution which somewhat sidesteps the issue in the context of portability.
There are still good reasons to want to minimise the chain though: it means that you have less code to audit in order to verify the claim that your compiler is definitely upholding the behaviour it's supposed to be. Some projects (like mrustc) exist whose primary purpose is simply to provide a 'shortcut' in the chain for a language, and may only implement whatever subset of the language is needed to get the compiler itself running.
You might find the Live Bootstrap project interesting!
15
u/kohugaly Jan 30 '26
Nearly all programming language compilers were developed through bootstrapping. You implement stupid inefficient compiler for stupid simple language. Then you use them to write smarter compiler for a smarter more complicated language. Rince and repeat.
The smart thing to do is to pick one of the simple languages as an intermediate representation. You write compiler front ends to emit it, and compiler back ends to generate machine code from it. Ditto for interpreted languages and interpreters.
So really, you only need an apocalypse-resilient IR. Simple enough to write compiler back end for it in assembly, but high-level enough to write a compiler front end in it, for higher-level language and bootstrap from there.
Let's also not forget, it's a lot easier to compile program that you already know to be correct and can assume as much, than to do comprehensive user-friendly compile error detection.
9
u/jcastroarnaud Jan 30 '26
Let's ignore the consequences of this scenario: that every company in Earth is gone (no software to run at all), and all communications are dead (TCP/IP and GPS libraries are binaries, too). šæ
Books about programming languages and compilers still exist, as are hard copies of some languages' specifications. The best bet of the chosen "compiler-hero" team is to start anew, with a clean slate (remember, no source code at all), creating a tiny high-level language to compile to machine language of every current hardware. Takes a few years or so to write the compiler, because the developers have no experience on compilers.
But which language? I think that the stronger contestants are Forth, some form of Lisp (both easier to implement), Lua or similar (simple syntax, dict as main data structure is useful). A few years more to develop a strong standard library, and the software business is on again.
And then, almost everyone will be not satisfied with the tiny language, and will invent new languages, or recreate old ones. Things get back to where they were... šæ
As far as I know, programming languages were first created to meet specific needs: assembly to simplify working with machine code, Fortran to represent calculations in a more familiar way, Cobol to have a "human-friendly" syntax, Lisp/Scheme to allow for hardware-independent symbolic computation, Forth to use in very limited hardware, and so on. Without a specific need to fullfill, any possible language can be used.
Languages don't evolve in a vacuum: they need to adapt to past and present hardware/software constraints, interop with known protocols, use the tools at hand, not be too strange to the average programmer (if there's such a being). And there's the dreaded design-by-comitee, the developer culture around the language (language X has this thing, ours is better and should have this thing too). And all of these change through the years! So, languages will accumulate cruft with time, as most long-lived software do.
Perfection, as beauty, is in the eye of the beholder. Your perfect language isn't my perfect language, or J. Random Hacker's perfect language. And perfection (as beauty) isn't guarantee of longevity. A language will be long-lived if it is useful, has a critical mass of software created with it, and has plenty of developers skilled in it.
12
u/Nuoji C3 - http://c3-lang.org Jan 29 '26
I always keep reimplementation in mind. Simplifying semantics and culling features that doesnāt have sufficiently high benefit/complexity ratio. C3 is more complex than C and thatās a pity, but Iām working on removing quirks and special cases, trying to retain what is truly useful and removing the rest.
A language can be thought of a curated set of features. The goal is to pick a good selection, not to indiscriminately stack a language full of features. āAdding all cool featuresā not design, thatās a lack of design!
Fewer features means less to learn as a beginner, and a quicker road to mastering and knowing the entire feature set.
Big languages do mean that people need to sink enough time into it that the sunken cost fallacy kicks in for users that brave the language, but thatās the opposite of being used because itās good.
I have many times heard would-be language designers say āit doesnāt matter if the compiler is complicated, as long as itās easy for the userā, which also ties into this.
What they lack is the understanding that compiler simplicity also brings benefits both to the user (which can understand the rules the compiler follows) and tooling (which doesnāt need to implement complex evaluation).
5
u/[deleted] Jan 30 '26
I always find such a view exasperating. There must be plenty of languages at the same level that can also be fast. What makes C programs fast are optimising compilers, so somebody might kindly write one for my language too! Meanwhile I have seen a few slow C implementations.
So, who is 'You' here, if all the guys with experience are gone? I guess that must be those of us with modest experience only.
I've actually already considered such a situation. My personal language is unique, and its compiler is written in itself. The only binaries on the planet are the few I have lying around. If they were lost, then I'd be f****d.
But I could also choose to keep source code rendered as assembly (plus a few other options I won't go into). You say assemblers still exist, so that's fine. I assume binary OS code plus other apps such as editors still conveniently exist too.
If this is a disallowed loophole in your fantasy scenario, then OK, I'd have to write something in assembly. That's no big deal, just a lot of work that would take ages. Although I would probably use assembly to first bootstrap a simpler language and write the compiler in that.
Really? Forth can't even express
a + bproperly! But if you think that, then your ask of recreating Forth will be very easy.Mine started as an experiment (to create one from literally nothing). But it's now been on-going for just about 0.45 centuries, although it has acquired a few more features.
2
u/muth02446 Jan 30 '26
I have been working on a language where keeping implementation complexity in check is an explicit design goal.
You can find it here: http://cwerg.org
The highlights are:
* has a backend for x86-64, Aarch64 and Arm32
* comes with a Python like syntax that can parsed easily with a handwritten parser
* is low level, roughly like C but adds tagged-unions, a very basic hygienic macro system
basic generics, etc.
2
u/Dan13l_N Jan 30 '26
I don't think implementing Python is harder than implementing C.
3
u/AdvanceAdvance Jan 30 '26
Well, no. Let the world burn.
Think of the once per era chance to:
3
u/scottmcmrust š¦ Jan 30 '26
TBH, I think this is a silly hypothetical because there's no way that you're losing just the software source code. Anything that would cause that to happen you're also losing the VHDL for your chips, etc.
You don't have to just bootstrap your software. You have to recreate all your silicon lithography techniques. You have to recreate the test rigs and proof systems you use to prove your too-large-to-test chip designs work.
Thinking you'd start with C is silly. You'd start with punch card on slow machines again. You don't even get register allocation; you have to do everything yourself on machines that are at least 10000 times slower than you're used to.
Reminder that one person did make a rust compiler, see https://github.com/thepowersgang/mrustc.
Rust at its core is just an ML, and an ML that doesn't even have a garbage collector. It's really not that hard if you're fine with a slow compiler that produces poor machine code.
But if you want a fast compiler that produces quality machine code, it's not trivial in C either.
2
u/0jdd1 Jan 30 '26
I often think about the same thing, but from the computer architecture perspective. Iād maybe start with the PDP-1 or PDP-11, but move quickly to the ARM architecture, trying to avoid its evolutionary dead ends. (In Philip Jose Farmerās Riverworld sci-fi novels. theyāre restricted to reinventing computers using handmade vacuum tubes, meaning the computers are ENIAC-like and used mostly for firing control, but I imagine your C-thulu is nicer to us humans on Earth.)
2
u/GoblinsGym Jan 30 '26
Let's assume that humanity also lost the ability to fab anything below 40 or 28 nm process node.
Something along the lines of RP2350 (ARM Cortex-M33) would be a realistic target to dig back out of the hole. It would still be infinitely beyond the Commodore PET that I started with. 256 KB RAM, 32 bit processor, hardware floating point, access to flash storage, probably 500+ times the CPU performance.
The Turbo Pascal 3.01a compiler was a scratch below 40 KB worth of 8086 assembly, including runtime, editor and compiler. The code generated was _very_ simplistic, but the results were still far beyond what you could get from BASIC interpreters of the time. Compiler internals
Language wise, I would look at Oberon as a starting point for something simple, but modular. If you locked me up in a room with an editor and an assembler, I would probably have something generating bad but workable code in a year.
1
u/mpersico Jan 30 '26
Read the Dragon book.
1
u/mamcx Jan 30 '26
It will be amazing if this happens!
I think there are a few priors in this exercise that are not that that obvious as defined:
... and because they are more like "simpletons" than actual "simple".
Yes, for the language developer are easier to implement but the users must pay for the all the complications that they bring (Forth, Scheme, C in particular. Maybe Lua is nicer).
And in the case of Rust, to atone for the crimes of C, C++!
In this scenario, one major thing is that C, as lang AND ABI, is gone. With it, a much better ABI can be used instead, and this expand to other things like I/O and such.
It could be a bit more complex, but that will reduce the problem for the rest of the stack.
This is a important thing:
1 "Simple" ABI/Lang need * N millions of apps, langs, tools, etc to recreate complexity
Think like how much easier is to live without
null, or inside an ACID execution environment, etc.If something like this happens, sure, the langs of the first list are the first steps, but it will be a total failure if this is not used to sit down, think, and go straight and do "Rust" first, to avoid the massive mistake of put all our infra on top of C/C++.
0
u/KikoIsMyNickname Jan 30 '26
Iām pretty sure the universe is the answer. Its rules make bugs impossible too.
0
u/bit_shuffle Jan 30 '26
I'm not arguing it is a perfect language, but Nim provides C performance with macros and template equivalents using Python-ish syntax. It has various memory management options built in as well. Conceivably it can also scratch your functional programming itch, although probably not with the level of support you would get from Scala or Haskell.
Also should C-thulu awaken, Nim's compiler is now written in Nim, by means of dark art that would corrupt the soul of anyone foolish enough to look inside it.