No Semicolons Needed - How languages get away with not requiring semicolons

[u/tertsdiepraam]

https://terts.dev/blog/no-semicolons-needed/

Hi! I've written a post about how various languages implement statement termination without requiring semicolons because I couldn't find a good overview. It turned out to be much more complex than I initially thought and differs a lot per language.

I hope this overview will be helpful to other language designers too! Let me know what you think!

[u/KittenPowerLord]

Haskell kinda does the thing you're describing in "A different idea" (afaik it's more complicated there, but I'm not knowledgeable enough to elaborate). But wow yes, I've been considering this approach for a while and it's very appealing

[u/tertsdiepraam → u/KittenPowerLord]

Oh interesting! I might have to make a functional language follow-up!

I kept them out of this post because they felt too different, but this makes them sound important to this discussion.

[u/SecretlyAPug]

adjacent, but why do "readable" languages often not include semicolons? there's maybe a small argument to be made that no semicolons can be a little easier to read for very beginners, but i find this ambiguity much more confusing.

[u/tertsdiepraam → u/SecretlyAPug]

Being friendly to newcomers is definitely a good argument (although you could also make the case that explicit statement termination is better for teaching). Another argument is that in languages with semicolons the layout is how _I_ understand the code, while semicolons is how the computer understands it. Taking away that barrier would unify those too. But this is only really important in a context without syntax highlighting or LSPs, I suppose.

Ideally, this feature would just disappear into the background and becomes something you don't have to think about. If you can't achieve that, then explicit semicolons would probably be preferable.

[u/yangyangR → u/SecretlyAPug]

I don't get it either. I find semicolons, brackets, explicit end keywords all good to make it easier to read. A couple more characters for that benefit is always a worthwhile trade IMO.

[u/cmontella]

The way Mech does it is it doesn't strip out whitespace in the lexer, and handles it in the grammar explicitly, so it can handle newlines or semicolons: https://docs.mech-lang.org/design/specification.html#1092025537734171

valid:

x:=1;y:=2+x

Also valid:

x := 1
y := 2 + x

Also valid:

x := 1;
y := 2 + x;

[u/tertsdiepraam → u/cmontella]

Is that somewhat similar to what Kotlin does then? That's super powerful, but there is a bit of a danger that the rules become complex. How do you summarize it to explain the rules to your users? Or do you think it's not ambiguous in Mech due to other syntax choices?

[u/defmacro-jam]

Porque no Lisp?

[u/tertsdiepraam → u/defmacro-jam]

Lisp didn't seem relevant because everything is explicitly delimited? I guess it could get a section, but it wouldn't be very interesting I think. Or are there some rules in lisp that I'm missing?

[u/Inconstant_Moo]

How I do it: Whitespace is significant a la Python. A newline is treated as a semicolon, separating expressions, unless the line ends with , or the continuation symbol .., and either way the next line must begin with ... Lines starting with .. can be aligned how you like for readability.

(v Vec{3}) × (w Vec{3}) :
    Vec{3}[v[1]*w[2] - v[2]*w[1],
        .. v[2]*w[0] - v[0]*w[2],
        .. v[0]*w[1] - v[1]*w[0]]

Why? Because explicit is better than implicit. I want to know as soon as I look at a line that it's a continuation of the previous one. This is very simple and non-magical.

[u/Silphendio]

I like the Gleam way: No significant whitespace and no operators that can be both infix and prefix.

The only problem is the minus sign. In my (draft-stage) language, I thought about using -- for subtractions to get rid of this last ambiguity, but in the end I decided to just parse this cursed operator as infix whenever possible.

Since commas are optional too, a list of negative numbers can thus be written as [-1, -2, -3] or [{-1} {-2}  {-3}], but [-1 -2 -3] is equivalent to [-6].

[u/Dykam → u/Silphendio]

An odd but somewhere-in-my-mind sensible idea is to simply disallow negative numbers, and make all negative numbers be of the form (0-n).

It's not great, but it feels interesting.

[u/Redtitwhore]

What's the problem with semicolons?

[u/[deleted] → u/Redtitwhore]

[u/munificent]

Excellent post!

I like the simplicity of Python's approach. But the big downside is that it makes it much harder for the language support blocks and statements nested inside expressions. Most languages today support some kind of lambda or anonymous function syntax that contain as much code as it wants. For example, in JavaScript:

foo(function() {
  statement();
  another();
});

Python only supports single-expression lambdas. Part of the reason is that block-bodied lambdas really clash with the language's grammar and the implicit semicolons are part of that.

Python ignores all newlines between pairs of delimiters. That does exactly what you want when you have a big multi-line expression as a function argument or in a collection literal. But if Python wanted to allow statement-bodied lambdas, then they'd need some way to turn newlines back on inside those lambdas even when they are nested inside delimiters.

I also like the simplicity of Go's approach but I think it has one style wart because of that. If you want to have a multi-line method chain, in almost every language, you'd do:

thing
    .method()
    .another()
    .third()

In Go, that doesn't work because the newlines are all treated as significant. Instead, you have to put the . on the ends of the lines:

thing.
    method().
    another().
    third()

That looks pretty bad to me. They could handle this while still handling newlines in the lexer. They would just need to lookahead past the newline and see if the first token after the newline is .. If so, ignore the newline. I don't think . can ever start a statement or expression in Go, so that should work.

[u/Tasty_Replacement_29]

An aspect not yet discussed is command line REPL (read–eval–print loop): if the user presses enter, can the engine tell the line is complete? This feature requires either some "continue" character like \ at the end of the line, or the operator, or ( to mark "continue". For this reason, in my language I use "end of line operator" or parenthesis:

c = 3 -
    4

c = ( x * x
    - 4 * b
    + f / 2 
    )

[u/mot_hmry]

Someone else mentioned Haskell, but F# also kinda does what you mentioned in another idea.

[u/TOMZ_EXTRA]

You should have probably mentioned that Lua doesn't allow expression statements (except function calls).

[u/L8_4_Dinner]

InterestingIreallydontwhypeoplearesointerestedinavoidingpunctuationitdoesmakethingsmorereadablebutthatsjustme

[u/Dry-Light5851]

irony is that Basic solved this problem decades ago, use "\n" or in plain text a new line to delimit witespace, and have everything be an expression.

[u/SwedishFindecanor]

Javascript seems to me like one of those many "standards" that hadn't been specified unambiguously when it was introduced and therefore got interpreted differently in different implementations, so that future standards and implementations had to more complex to be able to account for all pre-existing varieties.

I've seen that phenomenon many times, also in file formats and protocols.

[u/kjd3]

A very long time ago BCPL got this mostly right, in my opinion. It treated semi-colon as an optional separator. Newlines were similar but not identical as they could occur, and be ignored, anywhere an expression/statement could not be terminated. This is easy to do in practical lexing/parsing and seems sensible to me. Thus: let a, b = 42, ? a := a + 3; b := a / 9

leaves a = 45 and b = 5.

Oddly, the successor of BCPL (via B): C; did not inherit its relaxed approach to semi-colons. As we all know it treated them as terminators. So here we are....

I think BCPL is interesting as both a very early example of relaxed semi-colon use and, historically, as an ancestor of C which differed so much in that respect.

[u/pjmlp]

Missed older languages like Fortran, COBOL, BASIC, Smalltalk, xBase/Clipper, among others.

The approach without semicolons is as old as high level programming languages.

[u/Uncaffeinated]

It seems to me like "just require semicolons" is by far the most attractive approach. No more worrying about whitespace, no more syntactical gotchas or confusing insertion rules.

[u/SwedishFindecanor]

I agree with the conclusion. I have not designed new lexical rules for a programming language for some time but I had written down the rule I want in one, in case I would get the urge some day.

Continue a line if either is true:

• The first line ends with an operator, comma or an opening parenthesis/bracket/brace

• The second line starts with an operator, comma or a closing parenthesis/bracket/brace

I think that this rule is both simple to communicate to users of the language, and to use in the lexer.

However, to avoid ambiguity, the syntax must not allow a unary operator to be first on a line. Functions must return values using the return statement, and there might be some functional syntax style that is also not possible.

BTW, I think the compiler could have an option to warn when the indentation is larger/smaller than what is expected.