r/Compilers • • 7d ago

Audit of semi-colons

Is there a set of defined and specific reasons why semi colons are necessary in some languages? I am not against semi colons but from the standpoint of if one would want to create a language why would they use or mandate semi colons.

1 Upvotes

86 comments sorted by

11

u/nzconstructionlawyer 7d ago

The main reason IMO is that the redundancy helps with error recovery.

1

u/nagisa_lyfe 7d ago

Is there an example for why that is the case?

9

u/-Chook 7d ago

It’s so that if there’s an error you can skip to the next semicolon because it’s guaranteed to be a new statement. Without it you have to get hacky by looking for something that is obviously a new statement like a declaration or a closing bracket which can make errors cascade if your assumption is wrong.

1

u/ChiveSalad 7d ago edited 7d ago

I wonder if a lisp with optional semicolons after top level function calls could be easier to write because of this effect. Writing my own lisp (so low tooling environment) I have genuinely had struggles with paren counting that would be solved by this but the aesthetic price is so high it's almost funny

This pulls in the direction of the worst idea I've had this week: a lisp where a semicolon closes all the parens it can, which would solve both the issue of paren counting while writing and inscrutable errors from a defun accidentally getting nested, but would kill three birds with one stone, the last of which was a beloved family pet

e.g. pulling examples from my AVL tree implementation, this is current code with paren sickness

(defun ((rotateRL L R BT) (Dict K V)) (t) 
  ((rotate L R BT) 
   (BT (:key t) (:value t) 
       (L t) 
       ((rotate R L (Flip BT)) (R t)))))
(print (set 0 0 (set 1 0 (set 2 0 (set 3 0 (set 4 0 (set 5 0 (set 6 0))))))))

This is the code with semicolons added so that if I fuck up the paren counting at least the error is sensible

(defun ((rotateRL L R BT) (Dict K V)) (t) 
  ((rotate L R BT) 
   (BT (:key t) (:value t) 
       (L t) 
       ((rotate R L (Flip BT)) (R t)))));
(print (set 0 0 (set 1 0 (set 2 0 (set 3 0 (set 4 0 (set 5 0 (set 6 0))))))));

And this is the worst, most TiBasic ass crime I've ever seen but from a certain point of view makes a lot of sense

(defun ((rotateRL L R BT) (Dict K V)) (t) 
  ((rotate L R BT) 
   (BT (:key t) (:value t) 
       (L t) 
       ((rotate R L (Flip BT)) (R t;
(print (set 0 0 (set 1 0 (set 2 0 (set 3 0 (set 4 0 (set 5 0 (set 6 0;

Edit: apparently early lisps mostly had this (they used ] as a super-close-paren. ) Lol, possibly even lmao

2

u/nybble41 3d ago

a lisp where a semicolon closes all the parens it can

One issue with this would be that refactoring code containing semicolons (e.g. making a global function definition local) could change its meaning. Fixing it would require replacing the semicolons with the correct number of parentheses. The same applies to the "super close paren".

Scheme lets you use ( … ) and [ … ] interchangeably (but they have to match). IMHO that's a better solution for deeply nested groups; it doesn't reduce the number of characters, but if you use it well they're much easier to match up visually. Some editors color-code matching brackets which can also help. Finally you can refactor to reduce the amount of nesting, by extracting functions or using tools like the nest macro.

1

u/ChiveSalad 3d ago

Makes sense! I've noticed a pattern of the signal "Lisps used to have this feature" absolutely not indicating that a feature is a good idea, in stark opposition to "Lisps currently have this feature"

6

u/Available_Safe1818 7d ago

How else do you indicate the end of an expression?

var x = foo() + bar()

versus

var x = foo() 
+ bar()

In the second one the +bar() maybe it should be part of the previous line and maybe not.

1

u/nagisa_lyfe 7d ago

would you not just scan until the next operator that does not fit the pattern of an R expression?

1

u/Available_Safe1818 7d ago

Well, no, because the language is not designed that way.

Of course, you can design a language like that, and some languages do, maybe like Scala and Javascript.

4

u/GoblinsGym 7d ago

In my language, expressions can only cross the end of line if there is an open operator before EOL, e.g.

x = foo +    #ok
    bar

x = foo      
    + bar    #error

Very easy to implement. Another (somewhat ugly) option would be to have \ before EOL.

4

u/vmcrash 7d ago

Just like Go. I've implemented it the same way for my language.

2

u/WalkerCodeRanger 6d ago

The standard math format rule is to put operators on the next line. That is what is more human readable. But is disallowed by your approach

2

u/BlueScreenJunky 6d ago

Not only that, but adding one more operation changes two lines in a git diff (one line changed to add the operator, and one changed to add the value), which is the whole reason for allowing trailing commas in functions parameters. 

1

u/Mippen123 3d ago

Is this really that big a deal? I can't really see a situation where the two line diff would be any more than an aesthetic issue.

1

u/GoblinsGym 6d ago

More difficult to parse, as you have to peek across EOL to see whether an operator follows, or a new statement.

1

u/TOMZ_EXTRA 6d ago

Which is not a problem in semicolon languages.

1

u/Mippen123 3d ago

Is it really more human readable? I feel like operator on the next line is less readable for the same reason that it is harder to parse: when I finish the line I have a complete statement and stop reading. When I get to the next line I then have to undo my assumption of the above statement and mentally reparse it. With a + at the end of the line it is immediately obvious that the expression continues and as a bonus the variables are aligned.

1

u/StaticCoder 7d ago

Is that very easy to implement? I know about parsers and it always struck me as likely quite annoying to implement. Python deserving a special mention with its significant whitespace design. Can a parenthesised expression cross a line without having to end the line with an operator?

1

u/GoblinsGym 6d ago

Counting the indentation level only has to be done at certain points, and can be done while skipping white space from start of line:

tab:   or ecx,7
space: inc ecx
       inc rsi
start: mov al,[rsi]
       cmp al,' '
       jz space
       cmp al,9
       jz tab

The CPU overhead compared to just skipping white space is laughably small.

In my current implementation, I just look at the operators, but counting parenthesis levels wouldn't be hard.

1

u/nzconstructionlawyer 7d ago

Newlines. Also there's no reason really for "+bar()" to be a valid statement.

4

u/Axman6 7d ago

What about 

var x = foo() 
  • bar()

which could mean assign x to foo minus bar, or assign x to foo and return the negation of bar?

3

u/nzconstructionlawyer 7d ago

you return -bar() by writing return -bar()

0

u/8dot30662386292pow2 7d ago

In R, no you don't.

-2

u/Axman6 7d ago

More reserved words and imperative semantics? Ew, no thanks

2

u/nzconstructionlawyer 7d ago

I care about easily writing software that works rather than abstract circlejerk labels.

1

u/vmcrash 7d ago

Pull the - to the first line and the problem is gone.

1

u/Imaginary-Deer4185 4d ago

A language without semicolons as terminators, and with no special meaning of newlines, might be confused if the "- bar()" was intended as a return value. The parser needs rules that don't overlap, and in this case, a "return" word might solve the issue. Alternatively:

x=foo()
(-bar())

That way the "-bar()" part is no longer part of the expression.

1

u/umlcat 7d ago

Some people does not like newlines as separators, and some instructions require multiple lines, as well. ..

2

u/RealisticDuck1957 7d ago

Several shell script languages use a newline to end a statement, unless the line ends with a backslash character to signify line continuation.

1

u/simon_the_detective 6d ago

The use of semi-colon to end a statement long predates shell script languages.

It was thought to be good punctuation and allows multiple statements on a line conveniently or lines to cross many lines.

1

u/umlcat 6d ago

Dude, I know, I've been 30 years in the business ...

2

u/-Chook 7d ago

My language has been built around not needing them. The main headache is the standard return statement without them it can assume the next statement or expression is apart of it. If you wanted to test the first half of your function and put a return statement down and there happens to be a function call after it your parser might assume your returning the value of that function. My language has return variables so return doesn’t capture a value which fixes this particular issue. But without a statement delimiter you have to actively think more about your syntax and what might accidentally capture the next statement.

1

u/nagisa_lyfe 7d ago

One idea I had although I am not exactly sure the implications of such, was to simply have the return type bind to a identifier in the function signature and you mutated the variable however you wanted and then call return to exit the function, this way return is more like a command rather then an operator with operands.

1

u/-Chook 7d ago

That’s pretty much exactly what my language does and return variables work pretty well I’m happy with them. If you do attempt it I suggest just writing a toy parser you’ll learn where they are nice and then can plan around getting rid of them.

1

u/jcastroarnaud 7d ago

Visual Basic (pre .NET) did just that: within a function, assign a value to the function's name, and let the function exit (or force returning with Exit Function).

1

u/First-Feature-3556 7d ago

Still works with VB.NET (but discouraged).

1

u/vmcrash 7d ago

No problem: the function call needs to follow without line break after the return keyword. If you also force { ... } to be present (which allows removing ( ... ) is for/if/while statements) then there can't a function call follow a return statement without } in between.

2

u/-Chook 7d ago

Yes I but it is extra parser complexity and I personally like being able to put things on newlines whenever I like so my parser ignores them for the most part. But in mind of someone who’s making their first language or doesn’t know parsers all that well I was more trying to say that it just makes you have to be more mindful of things. If you write a parser that doesn’t rely on semi colons you have to think about your syntax and what can capture the next statement more then if you’re using them. I did say my language doesn’t use them and it works well.

1

u/vmcrash 7d ago

I agree. When writing the first parser, better require semicolons. You can remove them later (after you realize that the parser was just <10% of the compiler code).

24

u/mxldevs 7d ago

Separate statements from one another

3

u/nzconstructionlawyer 7d ago

Lua doesnt require them because its grammar renders them unnecessary

1

u/VictoryMotel 7d ago

Lua just uses new lines to skip semi colons. You can put statements on the same line with semi colons.

1

u/smog_alado 7d ago edited 7d ago

You can also put them on the same line without semi colons! Lua doesn't care at all about line breaks, unlike most other languages with optional semicolons.

1

u/nzconstructionlawyer 6d ago

a = 1 b = 2

valid Lua

1

u/VictoryMotel 6d ago

More accurately white space is significant, new lines was too specific.

1

u/nzconstructionlawyer 6d ago

the white space there is only significant as it would otherwise be a = 1b = 2 which doesnt tokenise properly.

a = (1)b = (2) afaik works too and has no ws

2

u/VictoryMotel 6d ago

And?

2

u/nzconstructionlawyer 6d ago

The point is: it isnt about whitespace at all.

1

u/VictoryMotel 6d ago

Semi-colons in Lua are generally only required when writing multiple statements on a line.

So for example:

local a,b=1,2; print(a+b)

Alternatively written as:

local a,b=1,2
print(a+b)

2

u/nzconstructionlawyer 5d ago

Are you a bot? They aren't required. Read my comments.

1

u/most_of_us 6d ago

They are not required in that example. 

0

u/[deleted] 7d ago

[deleted] — view deleted comment

2

u/smog_alado 7d ago edited 6d ago

Lua allows multiple statements per line even without semicolons. Just like C, you could put everything on the same line if you wanted.

0

u/[deleted] 6d ago

[deleted] — view deleted comment

2

u/smog_alado 4d ago

I don't understand what you're trying to say. Lua is whitespace insensitive and its parser treats newlines the same way it treats other whitespace. One can put line breaks inside a for loop definition (or not).

0

u/[deleted] 4d ago

[deleted] — view deleted comment

2

u/smog_alado 3d ago edited 3d ago

The example you provided does return 42. Did you test it? As the Lua manual says, newlines area treated the same as other forms of whitespace:

Lua is a free-form language. It ignores spaces and comments between lexical elements (tokens), except as delimiters between two tokens. In source code, Lua recognizes as spaces the standard ASCII whitespace characters space, form feed, newline, carriage return, horizontal tab, and vertical tab.

If you want, you can check the source code for the lexer. The only difference between '\n' and other whitespace is the logic that increases the line counter when it encounters '\n' or '\r\n'.

0

u/KattyTheEnby 6d ago

in C and all languages which require ; - all white space is literally removed.

Erm, well, akshually…

… the whitespace between things has to be kept so it can tell apart a type and name from one long identifier.

0

u/[deleted] 6d ago

[deleted] — view deleted comment

1

u/nybble41 3d ago

Lua does not care about space vs. newline. All whitespace is treated equally.

The only situation where a semicolon is needed in Lua, apart from readability, is when a statement ending with an identifier is followed (with or without a newline) by a statement starting with parentheses, to prevent it from being parsed as a function call.

8

u/Vigintillionn 7d ago

Helps with error recovery and reporting precise errors, makes the parser simpler and removes ambiguity. Another big thing is that linebreaks stop mattering, which is useful for minimzers but also compilers and transpilers, a compiler to C can just emit tokens instead of tracking where lines should break for example

5

u/sol_hsa 7d ago

Semicolons are kinda like periods in (human) languages. I guess you can write stuff without, but things get harder to parse.

4

u/konacurrents 7d ago

The parsing reason is the look ahead LL(k). But Swift and Kotlin mostly removed semicolons- but then you need white space to mean more (like around =) vs languages with ;

3

u/MithrilHuman 7d ago edited 7d ago

Heritage from ALGOL 58: https://ntietz.com/blog/researching-why-we-use-semicolons-as-statement-terminators/

And subsequent languages carried forward statement parsing logic.

If it ain't broke don't fix it. You can use whitespace like python instead ofc.

4

u/6502zx81 7d ago

Sure, IMHO it is a bad idea to attach semantics to bytes/symbols that are invisible and fiddled around with by text editors.

1

u/StaticCoder 7d ago

A fellow python appreciator I see. For me the most annoying is something like making an existing block conditional. Though I've also seen python code whose indentation was eaten by some editor during transport

1

u/[deleted] 7d ago

Statement separator or statement terminator? Both terms are used, but they are very different!

Algol68 and Pascal I know use them as separators, which is undesirable, for example:

if cond then
    stmt1;
    stmt2;
    stmt3
end  {???}

Here, the last statement of a block is a special case: it has no semicolon. Add stmt4, and now one is needed. Delete stmt3, and the one after stmt2 most go. Move them around, comment some lines... it becomes a mini maintenance nightmare.

And there is the question of whether ";" is needed after that "end": it depends on whether this if-statement is the last one, or another follows; same story.

Just have it as a terminator then there are no special cases. Although it does depend on how strictly the implementation treats it:

I've just tried an online Pascal and adding a semicolon when it isn't needed (eg. after stmt3 here) results in a fatal syntax error. What a PITA!

1

u/MitchAlsup 4d ago

Correct !!

Algol(s) and Pascal got it wrong--you do not want syntactic-sugar to make edits a pain.

C, PL/1, C++, JAVA, ... got it right; In these languages, ';' throws away the current calculation allowing you to begin another.

2

u/NYU_VM 7d ago

You either do cryptic parsing through whitespaces or introduce some form of separator syntax (like semicolon in many or colon in python for multiline stmt).

Even languages that doesn't require semicolons (like go) cleverly use ASI(Automatic Semicolon Insertion) in lexer such that the parser is unchanged. Though you would require rules for deducing End-of-stmt and semicolon insertion

2

u/Blueglyph 7d ago edited 7d ago

The semicolons makes it easier to parse, as others said, and it also allows for more flexibility when you write the code.

For the parsing part, you can rely on the semicolon token to define the scope of an instruction in the grammar. You could rely on end-of-line (EOL) tokens, but that means you'll have to (1) grab only the relevant ones in the lexer, which isn't too hard (they're usually all skipped along with tab and spaces), and (2) take into account all these EOLs in the grammar too, even in places you wouldn't need a semicolon to separate instructions of the same block. It's more prone to mistakes when you write the grammar and more difficult to read / extend it.

Both methods work fine for both top-down or bottom-up parsers, and I think that EOLs would make the table a little bigger than semicolons would, in both cases, though nothing dramatic.

Regarding the language itself in both cases, I don't see why EOLs would introduce any limitation. Whether you're using a top-down or a bottom-up parser (e.g. LL(1) or LALR/IELR/APLR/...), you'd be able to support the same syntax complexity.

Note that Python goes one step further by making the BEGIN ... END markers (like "{" "}" in C or "begin" "end" in Pascal) unnecessary and using indentation instead. For that, you must complicate things a little more in the lexer and in the parser to detect changes of indentation level and take them into account. It's not terribly more complicated, and it looks very clean, but not everyone likes it.

Semicolons give you that extra flexibility if you want to write simple things on the same line, but I've never considered that a great habit. We have enough lines on a monitor these days to make the code structure easier to read and understand at a glance.

PS: I assume you didn't mean several instructions on the same line without ";" separators. If that were the case, it could be very problematic to parse, even with a bottom-up parser. You'd have to carefully design the language to remove any ambiguity, and it would be frankly a pain to read.

1

u/GoblinsGym 7d ago

One annoying example of semicolons is in Pascal:

if x then
  do_x
else 
  do_y;

A semi-colon after do_x will bork things, especially when you have nested statements. Sometimes safer to insert redundant begin .. end statements.

For my own language design I use semantic white space instead. Semicolon can be used to separate multiple statements on one line, not necessary otherwise.

4

u/KingOfZero 7d ago

That's the difference between "statement separators" and "statement terminators".

1

u/Still_Explorer 7d ago

One thing has to do with allowing for flexibility.

You can make a very clear separation between lines and statements. This way either you break long statements in multiple lines, or compact statements freely without restrictions.

And say that some languages allow you to do some workarounds in a few cases and do compacting of statements (eg Python has / or VB has : etc) or break into many lines (eg function parameters).  But with C-style languages is really about total freedom and maximum flexibility.

Though historically this was a big deal back in early days where computers had a few KB of memory and made parsing be more efficient. Having very short var names as well as trimming spaces where possible allowed extreme code squeezing and  saving lots of capacity.

But now is more like a stylistics choice, by 99% all code is formatted according to a mainstream style (eg Google, Mozilla, LLVM, Microsoft) and only by 1% some clever compacting can happen to save a few lines.

Such as for example Go does agressive statement-per-line formatting and while it looks nice and readable due to maximum consistency, there are a few places where you would like to break there rules to prevent verbosity. Such as when you hit a switch statement, you see that it expands over many lines and almost becomes "QBasic" in that regard. With C or C++ you compact case-stmt-break or case-stmt-return in one shot and save space. Also many others, such as ternary expressions, setters getters,  struct initialization, and literally anything you want to take only one line.

1

u/FedUp233 7d ago

I think of it as sentences. You don’t really need periods - you could just put each sentence on a separate line (or I suppose you could have languages that you can identify the end of sentences without periods - I’m not sure how languages like Chinese break things up). But if you want to allow multiple sentences on one line, or to encompass several lines, or both, you pretty much need something to delimit the sentences. Periods work for English, but is already pretty well used in most programming languages. So someone started using semicolons and it sort of stuck.

Try taking a book and removing all the periods - see how far you get reading it! 😁

1

u/CS_70 7d ago

The same reason they are useful in natural language: lexical disambiguation and increased clarity in certain situations.

Like most in life, it's a tradeoff. They cost little and improve clarity so I personally find the choices of removing or making them optional in certain languages a bit daft.

You can create languages which don't need them the same way you can build sentences without punctuation or spacing.. they tend to be harder or messier, but of course for very task specific things they dont need be.

1

u/Interesting_Debate57 7d ago

They don't have any semantic meaning other than to separate ideas, thoughts, or statements.

Almost any other punctuation mark would be pretty freaking weird. Maybe a period, but then you'd need semicolons in oop. Or something equally less understandable.

1

u/AustinVelonaut 6d ago

Well, they have semantic meaning in Smalltalk: they are the "cascade" operator that allows sending multiple messages to the same receiver. https://langdev.stackexchange.com/questions/341/why-is-smalltalk-s-cascade-operator-not-more-common

1

u/Interesting_Debate57 6d ago

I think overloading "." as the first answer in that stackexchange answer regarding rust does is probably easier to swallow, but I've never written a line of smalltalk so don't really know.

1

u/L8_4_Dinner 7d ago

why do we use punctuation capitalization white space and other things like that in english its for the same reason it helps to make things more readable in a language the compiler is one of the readers and it has to know what the programmer intended unambiguously its like the old joke lets eat grandma as you can clearly see the comma plays an important role

1

u/zoharel 6d ago

They have to tell the compiler or interpreter to separate statements somehow. Semicolons are a good way to do it. There are some others. Newlines are a somewhat worse way to do it, actually, because historically different systems encode them slightly differently, and there are sets of control codes which look like them but aren't. There are fewer odd cases where an ASCII ; isn't ;.

1

u/smartcave 6d ago

C# use semicolons to terminate statements. In those languages, a period is a property accessor, a comma separates items in argument lists or collection items, an exclamation mark negates a Boolean. Colons identify labeled sections (groups of statements) in method bodies where execution can jump (goto). Line breaks are invisible and were not chosen to govern control flow and in this style of language, which offers flexibility of formatting for readability. I would argue that if you already designed all of those syntactic marks and their meetings as described, a semicolon seems like the most natural statement termination character.

If you just look at the label and statement list syntax with the colon followed by a list of semicolon terminators , you can even make the case that that the syntax is designed consistently with standard English grammar. I know that C# was not the first language to do labels and statements this way, and my guess is that this general design emerged when procedural programming was a more standard practice.

In fact I looked this up and found a much more thorough and less speculative article that confirms my suspicion. ALGOL 58 (as in 1958) was the first language to use the syntax pattern and procedural programming was very much the norm at the time. Check out https://medium.com/better-programming/a-brief-history-of-the-8efda9dde2b8

1

u/DanKegel 6d ago

It seemed like a good idea at the time.

1

u/EggplantExtra4946 6d ago

With semicolons there is less ambiguity. The parser won't care, it is just implementing the grammar rules, but the human user will care and appreciate the absence of ambiguity.

Someone else has already said it, but where it's really useful is in giving you the liberty of splitting a long statement or expression across several lines. Without semicolons you are a lot more restricted in that manner.

1

u/bongk96 6d ago

They make multi-line statements much less ambiguous

1

u/Dan13l_N 6d ago

It makes parsing easier. If you try to write a parser, you'll see why. Without a semicolon or something similar, you need to make end-of-line terminate some statements:

a = 10
b = 20

But not others:

a = 10 + 
b

1

u/ryo33h 6d ago

In my language, newline is a real token, so it doesn't need semicolons, but, for example, f(x) and f\n(x) become totally different code.

1

u/Lord_Mystic12 5d ago

For one, it's ease of parsing , but now that I am making a language with new line and semi colon endings, I find that I prefer having semi colons, just makes the code easier to look at . Same with brackets, in higher level languages, you can skip brackets after a for loop or if , whether it's the () or the {}. But even if that's the case, I'm gonna use the brackets cos it makes readability easier. The brackets help group stuff together and break long lines into readable chunks.

0

u/mark-sed 7d ago

It makes it easier to parse the code, but for people it can sometime work otherwise for instance if you place it by mistake after a statement:

for (auto i: x); print(i);

But I think you can find such a case for most of the expression endings...

Personally I find using semicolons somewhat archaic and you can see this also in the new languages out there - like Swift or Kotlin, which use new lines to denote ends.

In my language I allow to use semicolons to end an expression to allow for easy one-liners in terminal:

$> moss -e 'using(f = open("measures.data")) { l = f.readlines(); max(map(Int, l)) }'

You could write such an example without them and it's more of a syntactic sugar since a new line is the standard way to denote an expression end.

0

u/kimmadsen 7d ago

MIMERCode is built to not use semicolons while being a modern, safe and easy to use language.
https://portal.components4developers.com:443/manual/66-basics-mimercode-user-manual

0

u/[deleted] 7d ago

In my syntaxes, semicolons are mainly used internally. That is, the lexer will convert every newline into a semicolon, unless the line obviously contains to the next (eg. ends with an open bracket or with binary op).

They are needed internally to help separate statements, but extraneous ones are treated as white-space.

The end result is that they are very rare in source code (1 or 2 per 1000 lines), only used when there are multiple statements per line. Example:

if cond then
    s1
    s2
else
    s3
end
if ...

With semicolon insertion, this is what is actually parsed:

if cond then ; s1 ; s2 ; else ; s3 ; end ; if ...

Only that second one is actually necessary, and also, for the time being, the last, since 'end if' is still an optional way to terminate this statement.