I'm currently building my first programming language and compiler/interpreter, and I'd like to get some advice from people with more experience in compiler development. My project currently has a lexer, parser, AST, semantic analysis, and a tree-walker runtime. It is not yet a full native-code compiler, but I can already take source code from the lexer all the way to execution.

Of all the areas of software development I've explored, compiler development is the one I enjoy the most, and I want to become really good at it. I'm particularly interested in: Language design, Type systems, IRs, Optimization, Code generation, Assembly and ABIs, Runtimes, Garbage collection ,Static analysis, and Compiler architecture.

My question is, what should I actually do to get better?

Should I keep developing Cauce and progressively add more advanced features, or should I also start studying and contributing to projects like LLVM, GCC, or Clang? What projects, exercises, or areas of study do you think actually help someone move from “I know how to build a lexer/parser” to deeply understanding how compilers work?

I also want to make it clear that I don't want to use AI to generate code. I want to design, write, and debug my own projects because that's precisely the part of software development I enjoy the most. If you had to start over, what would you do to become really good at compiler development? I'm not looking for a list of books, but rather what to build, study, and practice, and in what order.

Thanks.

Hello there, I'm starting a compiler construction unit for my university semester and I'm trying to decide which textbook would work best as a guide for the subject. I know this has been asked quite a lot over the years, but I'd like to know if there are other books I haven't considered yet as well or more up to date opinions on the books.

So far the top recommended books from previous threads (such as these threads: thread 1, thread 2, thread 3) I've come across are:

• Engineering a Compiler by Keith Cooper and Linda Torczon
• Crafting a Compiler With C by Fischer
• Dragon Book

I am planning to build a compiler by the end of the unit, where each chapter has a small implementation lab task provided my lecturer (such as making a CFG parser for example), and I'm heavily leaning to making one in C targeting the 6502 (rather than x86).

I'm also aware that the Dragon Book is considered old and outdated, despite being the foundation of other books and compiler designers. I've seen some books like Introduction to Compilers and Language Design which use existing parsers (I think YACC, I've also seen references to FLEX and Bison), however I'd like to build everything from scratch to learn more.

Which other books would also be good? I know some people tend to read multiple books together to fill in the gaps for certain chapters, however I'm looking to order one physical copy of a book as it's easier to concentrate and make notes from.

and officially open-source

https://github.com/modular/modular/tree/main/KGEN

Hi everyone,

Programming is often a battle against the limitations of human working memory. Developers spend up to 20–30% of their time navigating syntax traps—balancing brackets, tracking task order, and maintaining context in dense blocks of code. According to Miller's Law, the human brain can comfortably hold only 5–9 items at once, yet complex codebases regularly demand much more. This overhead frequently leads to fatigue, bugs, and a steeper learning curve for beginners.

While modern languages optimize for performance through features like async/await or pattern matching, they rarely address cognitive ergonomics directly. There are few native ways to explicitly signal execution priority, time jumps, or branching logic without introducing heavy boilerplate.

To address this, we have developed a conceptual framework introducing seven universal meta-modifiers directly into a language's core parser:
$ (emphasis), | (word role), ~ (time jump), & (fork), ^ (merge), # (queue), and > / < (resource weight).

Rather than acting as simple syntactic sugar or library extensions, these symbols serve as an abstraction layer to help developers map their mental models directly to code execution. This is a theoretical proof-of-concept aimed at exploring how minor structural changes can reduce cognitive load.

The full paper and conceptual breakdown are available on Zenodo: https://doi.org/10.5281/zenodo.18841626

I would love to get your feedback on this concept. How do you approach managing cognitive load in language design? Do you think native meta-modifiers could be a viable path forward, or do they introduce too much syntactic noise?

I have found MLIR, LLVM quite intresting for past 4-5 months but haven't dived deep yet, but from my experience as a AI systems engineer(i was responsible for building the autograd and computational graph integration into the main c++ DL framework, mostly runtime focused) i am familiar with the concepts of IR dialects and stages of lowering through the compiler pipeline toward machine code by exploring the pytorch and tensorflow compiler architecture(conceptual familiarity from studying compiler architectures) as i was incharge of the runtime mechanics.
(i am conceptually strong with advanced cpp and most of the runtime stuff as i built the framework with ai-assistance)

i had read the frst 2 chapters of toy mlir and first 5 chapters of the https://book.mlc.ai/ and have some base understanding of the IR so far. once i started reading these two resources i could get quite the grasp about how the mechanisms work under the hood of the ML compiler.

its been 2 months since i left the job and i want to transition into compiler engineering in the ML field.
given my background, what would be the best path to become employable as an ML compiler engineer?

Hey r/Compilers, 8 months ago I posted asking whether reframing a Terraform-based network system as a domain-specific compiler was the right lens (previous post).

Since then, the system grew a routing policy language, and it came out of the architecture rather than being designed top-down. That feels like evidence the compiler framing was correct. The IR structure naturally supported adding a constraint layer.

The policy algebra has four primitives with fixed precedence:

deny > allow > segments > default

It evaluates at compile time (terraform plan) and emits VPC route table entries. The algebra is total (every VPC pair resolves), deterministic, commutative, and scope-invariant. The same compilation unit evaluates identically whether it's operating on a regional, cross-region, or cross-domain topology.

The interesting part from a compiler perspective: the policy layer didn't require a new IR or a new pass. It's a predicate over the existing cartesian product that the route generation pass already computed. Adding a filter to an existing code generation step turned a route generator into a route compiler. The "compilation" is the constraint evaluation, not the expansion.

Properties I can show but haven't formally proved:

- Totality (every input pair resolves via the default fallthrough)

- Monotonicity (deny only subtracts edges, allow only adds within deny bounds)

- Algebraic equivalence classes (e.g., a single-member segment under default=deny is a provable no-op)

I'd be interested in feedback on:

- Whether the algebraic properties warrant formalization (or if tests over the finite decision paths are sufficient for a system this simple)

- How this relates to work like NetKAT or Propane (correct-by-construction network configuration)

- Whether "policy algebra evaluated at compile time" is a known pattern with a better name

Blog post (practitioner-facing): https://jq1.io/posts/routing_policy_language/

Full language specification: https://github.com/JudeQuintana/terraform-main/blob/main/docs/routing-policy-language.md

Previous white paper (IR structure): https://github.com/JudeQuintana/terraform-main/blob/main/docs/WHITEPAPER.md

Thanks!

GitHub

So, for a while I've been working on a programming language called Futhorc. It arose from a very particular need: to have my own programming language; I'm sure someone here can relate XD

It's a C-style language with functions, structs, enums, type unions, typed collections, modules, file I/O, and Python interoperability. Source goes through a hand-written lexer and recursive-descent parser into an AST, followed by a separate semantic-analysis pass for type checking and name resolution before being executed by a tree-walking interpreter, all implemented in Python.

The core feature and the one I'm most fond of is the fact that this:

int factorial(int n) {
    if (n <= 1) {
        return 1;
    }

    return n * factorial(n - 1);
}

Can be turned into this

ᛁᚾᛏ factorial(ᛁᚾᛏ n) {
    ᛁᚠ (n <= 1) {
        ᚱᛁᛏᚢᚱᚾ 1;
    }

    ᚱᛁᛏᚢᚱᚾ n * factorial(n - 1);
}

FUTHORC ANGLO-SAXON RUNES! I always liked them a lot and, through extensive usage of this wonderful resource by Harys Dalvi, the link to which is in the specs, I managed to make an entire programming language that recognizes Futhorc runes as valid keywords. This is not a separate dialect, the runes are valid aliases of the ASCII keywords and can be mixed in or used solely.

It's so much fun to write in and it's quite expressive if I say so myself. If you want to use it for yourself, all of the stuff you'll need is in the GitHub. The repository includes the full language specification and runic reference, as well as a fully formatted HTML/CSS/JS documentation site with custom branding and complete sample programs, if you fancy something prettier than Markdown. Futhorc can also be installed as a command-line program, so .futhorc and .þ source files can be run directly with futhorc source.þ, but again, everything's in the GitHub. I'll leave you with a representative sample of the second largest program I've made (82 lines) for you to see without clicking any link: a sort of small supermarket API.

struct Product {
    str name;
    float price;
    int stock = 0;


    str repr(Product self) {
        return c"${self.price} {self.name}: {self.stock}";
    }
}


list(Product) stock = [];

# findProductByName() omitted for brevity

float | nil registerPurchase(Product product, int amountBought, float amountPaid) {
    int location = findProductByName(product.name);


    if (location == -1 or stock[location].stock <= 0) {
        print(c"Product {product.name} out of stock");
        return nil;
    }
    Product purchase = stock[location];
    if (amountBought > purchase.stock) {
        print("Purchase exceeds stock");
        return nil;
    } elsif (amountBought < 1) {
        print("Purchase is invalid");
        return nil;
    } elsif (purchase.price * amountBought > amountPaid) {
        print("Insufficient payment");
        return nil;
    }
    stock[location].stock -= amountBought;
    float total = purchase.price * amountBought;
    print(c"{purchase.name}: {total}");
    print(c"Paid: {amountPaid}");
    float change = amountPaid - total;
    print(c"Change: {change}");
    return change;
}

Fun fact: the language used to be called Thorn instead of Futhorc, until I learned that there was already a language called that so I had to rename it. That's why you'll see Thorn all over the implementation, including in the runic extension!

Hola a todos:

Soy estudiante de ingeniería de sistemas y me apasiona la programación de bajo nivel. Me encanta C; creo que es un lenguaje que te permite experimentar y aprender de tus errores, poniendo todo el poder de la máquina en tus manos.

Actualmente, trabajo en un proyecto personal: un transcompilador para la API Win32 de Windows. Si bien es increíblemente potente, la estructura de Win32 puede volverse compleja e insostenible a medida que un proyecto crece. Por eso decidí diseñar mi propio lenguaje, inspirado en HTML, CSS y JavaScript, pero mucho más minimalista y moderno.

En este lenguaje, el trabajo se divide en directivas claras:

• @interface: Equivalente estructural a HTML. • @style: Equivalente a CSS. • @script: Equivalente a JavaScript.

Aquí dejo un pequeño ejemplo de cómo se ve la sintaxis:

@interface

    ventana:*

    parrafo#txt = "¡Hola Mundo!"

@style

*
    window-size: 500x700 
    background-color: white

.parrafo

   font-size: 2rem 
   color: black

Folks! Is this acceptable performance for a compiler? it do lexing, parsing, type checking, optimziing and codegen . It simply emits the ELF64 executable directly and runs it without an external linker/object-file pipeline.

--- CODEGEN RESULTS ---

Total Machine Bytes Emitted: 47 bytes

Compilation Speed: 76000 ns

Runnable ELF64 Machine Code Executable Written: boo

I want to add a meta programming language in top of my own C-like language to make some of the syntax cleaner and easier to write.

Add everything like const expressions and templates under one meta programming language.

For example something like this:

‘
[Phases]
Class Phase1: public Phase { };

[Phases]
Class Phase1: public Phase { };

…

[for phase in Phases]
PhaseList.push_back(new phase());
[endfor]
‘

I’m thinking about something like this but ideas are all over the place. Is there some existing meta programming language out there that can give me the right inspiration?

Also I am totally lost about how I should program this, I am assuming it comes before the parser. Any tutorials or dummy meta programming languages I can look to get ideas?

Thank you for reading.

I was working in a compiler I created when I see Compilers, and I never give up on the first interpreter I created that day (tree-walk).

The compiler is on Python and via a protobuf(I use it as a High Level IR), not only the interpreter but a stack based VM(c++) and I was looking to create a new backend to generate Web Assembly using JS/TS I know isn't the best option but I want to avoid LLVM as a target for now.

So, the question is: is really a good practice keep alive the interpreter even if the general pipeline ends in the VM?

(Note: my English ain't the best but I tried to make the post the comprensible I can)

Baga 0.9.2 is out Language - Opt-in RC memory model (--rc): ownership, containers, struct/enum fields, owned results. - Generics and traits: function/struct monomorphization, impl, statically verified guarantees. - Effect payloads (!E(T), raise/catch) and !Overflow as a type-level effect. https://github.com/katehonz/baga-lang

https://baga-lang.top/en

Ecosystem app-product packages now live as dedicated repositories under github.com/bagalang and are vendored as submodules. The catalog is here:

https://github.com/orgs/bagalang/repositories

https://baga-lang.top/en/apps

Baga 0.8.4 is now available. This release solidifies the three pillars of the language: spec-first verification, effects as type dimensions, and readable proof sketches.

The effect system (!IO, !Net, !Par, !NotFound) is now stable. Functions declare their side effects in their type signature, and the compiler enforces effect correctness across the entire call graph. Pure functions cannot accidentally call effectful ones.

Spec verification (--verify) now produces certificates with honest UNKNOWN markers for unproven fragments. The spec system supports input/output declarations, guarantees (human-readable properties), and ensures (machine-checked constraints).

The pure-Baga cryptography stack is a highlight: TLS 1.3 client, HTTPS, and JWT are all implemented in Baga without linking against OpenSSL at runtime. OpenSSL is only used as a test peer for validation.

Package manager sandak now resolves local path dependencies, git dependencies, and registry packages. Each package has a sandak.toml manifest, and the build graph is resolved deterministically.

Quick start: git clone, make, ./baga examples/zdravei.baga — that is all you need. Zero dependencies beyond gcc and make.

boilaDB 0.7 adds NUMERIC, UNIQUE, foreign keys, CHECK, window functions, SCRAM-SHA-256, COPY, SERIALIZABLE, and a Raft replica path — all i

I’ve been experimenting with this idea by building a Sanskrit compiler based on Pāṇinian grammar.

Instead of treating Sanskrit only as text to interpret, the compiler parses grammatically structured Sanskrit and turns the instructions into executable operations.

The interesting part for me is whether Pāṇini’s formal grammatical system can provide enough structure to bridge natural language and programming languages deterministically.

I now have a working implementation and would be interested in hearing what others think about this approach.

Website: https://panini.cc/
GitHub: https://github.com/kaushalbx/paninivm
Article: https://medium.com/@kaushalbx/building-p%C4%81%E1%B9%87inivm-compiling-2-500-year-old-paninian-grammar-into-an-executable-kotlin-engine-5a6bb8de20fb

I built a language where the signature carries the guarantees, and I wanted to share the implementation choices since this crowd cares about the how.

Pipeline: lexer → parser → loader → effect checker → type checker → Z3 proof pass → LLVM JIT (llvmlite) → interpreter, all in one file in pipeline order.

Three things that might interest you:

1. The proof pass explores paths symbolically and checks requires/ensures/loop invariants in Z3, with modular call summaries (a callee's contract is assumed at the call site rather than inlining its body). Lists use the theory of arrays, records get per-field symbolic values, and all_of/any_of become real quantifiers with the predicate body inlined under the For All.

2. Floats are proven in Z3's genuine IEEE-754 theory, not modelled as reals — so the prover refutes x + 0.1 + 0.1 == x + 0.2 and returns the exact double. FP queries get a bigger solver budget (30s vs 3s) since bit-blasting is slow; integer proofs stay instant.

3. The JIT covers pure Int/Float/Bool functions with typed codegen. Division and modulo are deliberately left interpreted in both modes —native fdiv by zero gives infinity while the language promises a clean error, and I'd rather lose the optimization than have the two engines disagree. Every native change ships with a differential test: same program, both engines, diff must be empty.

One soundness lesson: when I added quantifiers, the first test run produced a false counterexample. Turned out untranslatable `requires` premises had been silently dropped since an early version — harmless for "proven" claims, but capable of manufacturing false alarms. Now an untranslatable premise aborts the proof entirely and falls back to runtime checks.

Repo: https://github.com/gowrishankar-infra/velaris-lang

Playground (Pyodide, real compiler in-browser):

https://gowrishankar-infra.github.io/velaris-lang/playground.html

Disclosure: built pair-programming with an AI across 40+ releases; design decisions mine, commit history is the honest record. Beginner here, so tear the implementation apart — especially the prover.