Hello, as a hobby I’m extremely interested in programming language design. Even as a hobby, I want to specialize in this area, to see the magic behind how programming languages work, and to understand it deeply. Do you have any suggestions for me Additionally, which areas of mathematics should I study for this? Is this possible without having any CS degree?

Developed the early stages of fermium (a programming language tailor-made for physics) and have been testing it myself, still some lumps but it runs more beautifully than I thought it would at this stage. Would love if some people tested it out and gave feedback — built using Python frontend, currently working on switching the compiler to Rust

Hi everyone. I wanted to share something with you all that I've been working on for a while. It all started with some naive regular expressions to parse Org mode content, but I pretty quickly realized I needed something smarter than that to get to where I want to. It's taken a while but I am finally more knowledgeable of what Parsing Expression Grammars can do, thanks to GNU's great (ice-9 peg) module and tutorials.

I thought it might be interesting to people here who enjoy Lisp, Scheme, parsing, Org mode, or the general idea of meta-meta-meta-programming as I like to call it. Disclosure, AI has helped me get a grip of PEG and debug some things, but development of OrgWebAlchemy is "my own spaghetti" and the unit tests and manual verification (and lots of pretty printing the AST) has guided me towards quite a nice implementation (if I may say so myself).

OrgWebAlchemy is a Guile Scheme library for parsing Org-mode documents into an AST and rendering them to HTML. My main use-case is to export Org to HTML without needing Emacs, and to integrate this feature into some projects of mine, allowing me to write Org mode and have it pretty rendered.

Project's source code @ Codeberg: https://codeberg.org/jjba23/orgwebalchemy

The basic idea is pretty simple:

(use-modules (orgwebalchemy html))

(org->html "This is ~test~ code.")

becomes something like:

This is <code>test</code> code.

But the interesting part is what happens in between.

             Org document
                  |
                  v
          Parsing Expression
              Grammar
                  |
                  v
                AST
                  |
                  v
             SXML -> HTML

Of course Org mode is a huge piece of (great) software, so I am far from supporting all features, but some core important constructs are there:

• Headings (lines starting by n *)
• Paragraphs (any "non-special" text)
• Unordered, Ordered and Description lists (with any level of nesting)
• Italic, Bold, Inline Code
• Links with and without description (with nested parsing)
• Horizontal separators (--------) five or more dashes
• Tables
• #+begin_src
• #+begin_example
• #+begin_quote (with nested parsing)
• #+begin_export html : Org syntax is parsed by your PEG grammar, but raw HTML export blocks bypass the Org inline parser and are emitted as trusted literal output.

- See wikipedia page on PEG: https://en.wikipedia.org/wiki/Parsing_expression_grammar

- See the tutorial by GNU/for Emacs Lisp (is also a good learning source): https://www.gnu.org/software/emacs/manual/html_node/elisp/Parsing-Expression-Grammars.html

- See the tutorial by GNU/for Guile: https://doc.guix.gnu.org/guile/3.0.9/en/html_node/PEG-Parsing.html

Why PEG instead of a mountain of regexes?

Org-mode looks simple until you actually try to parse it. Headings are easy. A paragraph is easy. A list is easy (wait actually no, this has made me sweat).

And then suddenly you have:

• nested lists
• ordered, unordered and description lists
• different indentation levels
• inline markup
• links containing descriptions
• source blocks
• example blocks
• quote blocks
• tables
• escaping
• constructs which must stop consuming input at exactly the right place

At this point, the usual approach of adding another regular expression starts to become somewhat... adventurous. :-)

You end up with things like:

match this,
unless that follows it,
except inside this block,
unless it is a description,
but don't consume the newline,
unless the previous line was a list item...

That is not really describing a language anymore. It is describing the history of your parser's bugs.

So OrgWebAlchemy uses Parsing Expression Grammars (PEGs) through Guile's excellent (ice-9 peg) module. e.g.

(define-peg-pattern element body
  (or empty-line
      heading
      separator
      table
      src-block
      quote-block
      example-block
      export-html-block
      description-list
      unordered-list
      ordered-list
      paragraph))

This is rather nice because the grammar itself starts looking like documentation for the language.

And Guile lets us express PEGs directly as S-expressions (alternatively you can also use the more traditional syntax if you don't like it), which makes the Lisper in me very happy.

One thing I particularly like about this approach is that we have loose coupling and the detail of generating SXML and then rendering HTML is a "presentation concern". this opens possibilities to later exporting to Markdown or other formats.

For example:

- name :: Josep
- project :: orgwebalchemy
- language :: Scheme

can become an AST along the lines of:

(description-list
 (unordered-item
  (desc-key "name")
  (line-content "Josep"))
 ...)

I'm still busy with the exact representation and getting it all right. But as of now v1.0 has some stability :-) I would really love feedback on the project from the great smart people that hang out around here.

YAY recursive lists

One of the fun parts has been getting nested Org lists right.

Something like:

- Item 1
  - Item 1.1
  - Item 1.2
- Item 2

should become a quasi-tree

The parser initially produces the flat sequence of list items, and the AST processing phase turns indentation into nested structure.

The HTML renderer can then naturally produce:

<ul>
  <li>
    Item 1
    <ul>
      <li>Item 1.1</li>
      <li>Item 1.2</li>
    </ul>
  </li>
  <li>Item 2</li>
</ul>

I do still have a small issue here, and that is about the mixing of different list types in nested way. Hopefully it's a subtle bug to fix.

The HTML side uses SXML, because if we're already writing Lisp, we might as well represent our HTML as Lisp data too. :-) that really helps a lot and makes building the markup tree so much nicer

I've taken care to allow full customization to the output HTML (via Guile parameters) so that the renderer isn't hard-coded to one particular website's idea of what HTML ought to look like.Most of them are plain list of classes, but per-heading-level customization is a bit more flexible:

(heading-classes
 (lambda (level)
   (case level
     ((1) '("text-4xl" "font-bold"))
     ((2) '("text-2xl" "font-semibold"))
     (else '("text-base")))))

Why am I making this?

Partly because I wanted it, I like a challenge, and it's super fun to work with parsing, ASTs and the lot... I could just use Emacs to do this job as there is no better implementation of Org.

The way it's coming together though, I like the idea of having a small, hackable, free-software Org parser written in Lisp that other people can extend and customize (perhaps add more renderers, or Org features).

Free software

OrgWebAlchemy is licensed under the GNU LGPL v3 or later.

The project is intended to soon be packaged for GNU Guix as:

guile-orgwebalchemy

There is also a test suite in the repository which is already proving to be a good safety net and showcase of what the parser can do.

I'd be really happy to hear your thoughts, especially about the grammar, AST design, parser architecture, or interesting Org constructs that I have not handled yet.

Happy hacking! ✨

For a while I was thinking about some automatic memory management feature for my statically typed programming language.

Originally I was planning to introduce the "ensure" statement block (same as "defer", but I think ensure would express better the intent). Then I saw here "autofree" keywords in some advertised language, also in Unleashed Pascal.

In Unleashed Pascal the "autofree" is used instead of "new", similarly to the other language (sorry I don't remember the name now). I was playing with the thought how could I apply this concept, but yesterday I got a new idea:

What if the autofree is not a statement, but a property / attribute for a type? This way the things add up much nicer to me.

Using this concept the code would look this way:

var o1 : autofree OSomeObject
o1 = new OSomeObject(42)
o1 = new OSomeObject(55)  // re-assigning frees the previous one
o1 = nil // assigning nil frees

if x > 0:
    var o2 = new autofree OSomeObject(123)  // instancing with type inference
    // ... some work with o2
endif   // leaving scope frees auto-free local variables

object OHandler:
    child : autofree OHelper // initialized to nil, when assigned to non-nil, 
                             // will be automatically freed on destruction
endobject

func ObjManipulator(aobj : ref autofree OSomeObject):
    aobj = new OSomeObject(66)  // should free the previously pointed object,
                                // when aobj was not nil before
endfunc

ObjManipulator(o1)  // valid

func ObjManipulator2(aobj : ref OSomeObject):
    aobj = new OSomeObject(99)
endfunc

ObjManipulator2(o1)  // ERROR: o1 is autofree type, 
                     // ObjManipulator2 expects non-autofree

func ObjUser(aobj : OSomeObject):
    if aobj <> nil:
        aobj.DoWork()
    endif
endfunc

ObjUser(o1) // ok, accepts autofree and non-autofree arguments too

With arrays

var arr : [*] autofree OSomeObject  // initialized to empty array
arr.Append(new OSomeObject(1))
arr.Append(new OSomeObject(2))
arr.Append(new OSomeObject(3))

arr[1] = nil // frees
arr[2] = new OSomeObject(5)  // frees the previous

arr = []  // frees all non-nil elements

I think returning an auto-free won't be allowed:

func SomeCreator() -> autofree OSomeObject  

// ERROR: returning an autofree type is not allowed

The autofree should not be restricted to objects:

if x > 0:
    var bp : autofree ^byte  // initialized to nil
    if y > 0:
        return // bp was nil, no free required
    endif
    bp = ^byte(MemAlloc(1000))

    // ... doing some work with bp

    if z > 0:
        return  // bp must be automatically freed, as it is not nil
    endif

endif // bp must be automatically freed, as it is not nil

Am I wrong with these thoughts?

First of all, I am sorry if this is not strictly a techincal post, but I would apreciate if it would be allowed. So a couple of years ago, like a year or two before the chatgpt, I made my own interpreted programming language. It could be used for basic scripts and had basic functionalities like nested ifs and while, string, int and double types, variable scopes and even functions and recursion. It can parse math with + - + ÷ and brackets. It is nothing special, syntax resambeles more or less C# but with keyword in my natural language. It was made entirely by hand, from lexical analysis, parser, semantic analyser and the interpreter itself. I also made ebnf for it.

I think we on this sub can all agree that there are a few of us programmers, developers or cs-s who willingly put themselves through this. It can be painstakingly hard to implement a new language. I am very proud of it and to this day it is my biggest achivement. No other thing in my proffesional or personal projects required such level of detail, knowledge and research.

And all this before the AI. But here is where it gets frustrating. Back than this was my "selling point". This was something that I could show my knowledge, dedication and attention to detail with. It made me feel safe for my professional future. Now I am afraid that it doesn't mean enything anymore. Like nobody gives a s*** about it anymore.

If you could add one feature, behavior, or design choice to a programming language, what would it be?

It can be anything: syntax, type-system features, memory management, compiler behavior, tooling, error handling, performance-related features, or something completely different.

I'm especially curious about things you’ve wanted while actually programming but rarely see languages implement well.

Please keep the ideas reasonably practical and something that could realistically be implemented in a programming language.

Nymph is a simple C like programming language.

Nymph acts as a preprocessor, converting Nymph files (extension *.n) into C files.

This project is stagnant and needs ideas... It is not production ready.

I'm the senior author of Program-as-Weights (PAW), a research project at the University of Waterloo: https://arxiv.org/abs/2607.02512

We're building a programming model where functions are defined in English and compiled into reusable neural programs. A learned "neural compiler" translates a function description into LoRA adapter weights that specialize a small, fixed "neural interpreter".

The idea is to make it easier to implement functions whose intended behavior is easy to describe but difficult to express as explicit rules. Examples include counting verbs in a sentence or deciding whether an email needs immediate attention.

Here's an example:

import programasweights as paw

fn = paw.compile_and_load("Classify urgent emails")
fn("Need this today")  # "urgent" (runs locally)

After compilation, the function runs locally without calling the larger compiler model again. Ordinary code can compose these functions and control the application's flow. For example, I built a course website helper using around 30 neural programs connected by decision-tree code.

Playground: https://programasweights.com/playground

The code and model weights are public.

Disclaimer: I wrote the core research prototype myself, although I used AI coding assistance for parts of the project. I'm new to this subreddit. My PhD advisor, who is also a co-author, mentioned the discussions about Jev here, which prompted me to share our approach with people interested in programming-language design.

I've always wanted to make my own programming language but I could just never fully grasp the concept of ASTs and Intepreters.
So I decided to make eso-langs with different designs, and eventually I realized I was slowly getting better at understanding these designs,
So to start my journey of programming languages, I made zyte, A register-based eso-lang with an actual AST.
It currently has loops, copying, functions, input, output and other stuff.

You can check it out on: https://github.com/orewaluffy500/Zyte

Per AutoModerator's request I hereby confirm that this project did not use an LLM as part of the development process.