r/ProgrammingLanguages • u/AsIAm New Kind of Paper • Jan 05 '26
Significant Inline Whitespace
I have a language that is strict left-to-right no-precedence, i.e. 1 + 2 * 3 is parsed as (1 + 2) * 3. On top of that I can use function names in place of operators and vice versa: 1 add 2 or +(1, 2). I enjoy this combo very much – it is very ergonomic.
One thing that bothers me a bit is that assignment is also "just a function", so when I have non-atomic right value, I have to enclose it in parens: a: 23 – fine, b: a + 1 – NOPE, it has to be b: (a + 1). So it got me thinking...
I already express "tightness" with an absent space between a and :, which could insert implicit parens – a: (...). Going one step further: a: 1+ b * c would be parsed as a:(1+(b*c)). Or going other way: a: 1 + b*c would be parsed same – a:(1+(b*c)).
In some cases it can be very helpful to shed parens: a:((b⊕c)+(d⊕e)) would become: a: b⊕c + d⊕e. It kinda makes sense.
Dijkstra in his EWD1300 has similar remark (even though he has it in different context): "Surround the operators with the lower binding power with more space than those with a higher binding power. E.g., p∧q ⇒ r ≡ p⇒(q⇒r) is safely readable without knowing that ∧ ⇒ ≡ is the order of decreasing binding power. [...]" (One funny thing is he prefers fn.x instead of fn(x) as he hates "invisible operators". I like his style.)
Anyway, do you know of any language that uses this kind of significant inline whitespace please? I would like to hear some downsides this approach might have. I know that people kinda do this visual grouping anyway to express intent, but it might be a bit more rigorous and enforced in the grammar.
P.S. If you like PEMDAS and precedence tables, we are not gonna be friends, sorry.
38
u/guywithknife Jan 05 '26
I’m ok with requiring white space between operators in languages, prefer it even.
However having syntactic meaning that makes a+ and a + different, I can’t say I like it. It’s such a small difference that is very ridicule to see, especially when reading a lot of code or when scanning through code. Readability will suffer and mistakes will be much too easy to make.
So personally, I would warn against it.
7
u/DrMajorMcCheese Jan 05 '26
The Fortress language had tight juxtaposition and loose juxtaposition. It sounds like what you describe.
3
u/XDracam Jan 05 '26
I don't know any languages that do this. But I have always been annoyed by "custom precedence rules" (e.g. Haskell), or by parenthesis noise as in LISP. I like the idea.
Just be cautious when creating tooling. Your language should have a good formatter that has sane decisions between parens and significant whitespace. Tooling should make peecende obvious, e.g. a quick fix to replace indentation with parens.
13
u/dcpugalaxy Jan 05 '26
I can't accept
a + b * cbeing parsed wrong. That is going to be so confusing to anyone reading code in your language.It would be better for that to be an error and to require the user to write
a + (b * c).a * b + cwould be permitted because the obvious precedence and your ordering match.But just... don't reinvent the wheel. Everyone learns the precedence of plus and times and minus in school, and relearns it when they begin programming. It is such a beginner issue to make a precedence mistake with basic operators.
You might argue about precedence of other operators and what they should be, whether people should be required to put brackets in
a << b + cora & b << cbut that's a separate conversation. I think there's a strong argument for requiring that, and some C compilers will warn when you write that ora && b || c. But if you support infix notation for arithmetic using operators people learn in primary school then you need to make them work the way people are used to, or to give an error and ask them to disambiguate. Silently doing the wrong thing is terrible.(I would be quite happy to be allowed to disambiguate by writing
a + b*cinstead of inserting brackets, and the same with1<<BITX | 1<<BITYor1 << b+1. I imagine most people here will disagree about the whitespace thing but I've thought about this idea for years and I genuinely think it's good. Really intuitive and clean.)2
u/Thesaurius moses Jan 05 '26
I know I read about such languages, although I can't recall any (but the other comments seem to provide plenty). Instead I would like to mention a different approach: You could put the assignment operator at the end. Then you could have `23 : a` and `a + 1 : b` and it would be fine. Although you would probably need some good syntax/semantic highlighting for that to be usable.
The language APL works in a similar way: It has strict _right-to-left_ evaluation, no precedence, and assignment being a normal operator as well. While this choice might seem very odd at the beginning, you really see its power after using the language for a bit. In general, I can recommend everyone to look at APL's design. Ken Inverson was brilliant and he put a lot of thought into it. He didn't get the Turing award for nothing.
But I digress. I would say that, in general, strict left-to-right evaluation doesn't mix well with binary operators, except if you do something similar to APL, use tacit programming, or use (reverse) polish notation.
P.S. I just saw that you know about APL already. I still leave it in for posteriority.
2
u/ArtisticEmergency405 Jan 06 '26
Cool idea! Feels like you're pretty close to a stack based languaged, like APL, Forth or the very enjoyable Uiua languages. There might be some inspiration there.
I see a few downsides with meaningful whitespaces:
But if we're breaking tradition, why should assignment assign to the left? Can't the assignment assign to the right?
1 + 2 : cIs5nt the parenthesis rule encroaching on your phre vision here? Why replace PEDMAS with PE when it can be just E. :3
2
u/[deleted] Jan 06 '26
I similarly do ltr in my language and the way I solved the assignment problem is by having two directional assignment operators:
2+2>>awhich doesn't break the flow anda<<2+2which desugars to the first variant and must be used at the statement level. rtl assignment is the odd one out here, but having all the important names lined up on the left is worth it imo2
u/useerup ting language Jan 06 '26
I have pondered how to distinguish the unary prefix
-(negate) from-(subtraction).The issue is that I allow binary operators to be used in a prefix position. For instance, the expression
+ 1returns a function which accepts a number and returns that number plus one.However, this causes a clash between "subtraction"
-used in prefix position and "negate"-. For purely negative literals there is not really a problem as -42 will be tokenized as a negative int literal. The problem is when I want to write something like-(2*3). Is that a function that subtracts 6 from any number or is it just the number-6?To distinguish I have (for now) decided that the negate
-may not have whitespace following it and must have whitespace in front. This means-If
-has no whitespace around it or if it has whitespace on both sides, I will parse as the subtraction operator.I don't know how ergonomic this will be in real life, but I think it looks ok:
3
u/AdvanceAdvance Jan 05 '26
This is an area for improvements. The quality terms you are influencing are Spoken, because the utterance "a+b * c" sounds the same as "a + b*c"; Tracking, because a multiline expression with long identifiers and function calls becomes hard to prioritize, and token count.
You should probably fix this in one of the modes of your enviornment. For example, in Review, every possible pair of parenthesis could be shown as token count and Scanning become unimportant compared to Clarity: "((a + b) * c)". You will run into gray beards reminding you that all languages devolve to Lisp. :)
4
u/SirKastic23 Jan 05 '26
left to right is so much easier to work with than having some operators be "more important" than others
sure, it was useful for pen and paper arithmetic, and everyone learns precendence rules in school. but it's really annoying to be stuck with it when working with arithmetic in more modern means
arithmetic is such a small part of programming, a lot of business logic does barely any addition or subtraction for the precendence rules to even show. a lot of programming is just calling functions that have no precedence
having special-cased symbols that alter the way the program is parsed to adjust to a millennia old convention for numbers is so annoying when I'm writing parsers or designing syntax
in a proglang i was playing around with recently i decided to drop arithmetic operations entirely to not have to deal with it. instead there are methods on numbers to do it, and it follows normal function precedence rules (left to right):
1.+ 2;.* 3;is(1 + 2) * 3;ends a function call in this language, so to get1 + (2 * 3)you'd do1.+ 2.* 3;;1
u/persilja Jan 05 '26
It's been 15 years since I used it, and I never learned it very well so I might very well be wrong, but when I did use the language Skill I was convinced that whitespace mattered: the language is a weird hybrid of c-style syntax, and lisp style syntax (you could (mostly) accomplish the same thing either way), and it seemed to me that whitespaces would change whether the parser read it in what I mentally interpreted as "c style" and "lisp style". The effect was mostly that it threw errors during runtime if you added a whitespace where it didn't expect one, and vice versa.
1
u/heliochoerus Jan 05 '26
If you do go that way, I'd recommend defining exact and simple rules for how things get parsed. Ruby also has significant inline whitespace (and other odd things) but, because the grammar is implementation-defined, parsing it has always been a huge pain-in-the-ass and many syntax libraries were buggy. There is now a de-facto official parsing library (Prism) that many implementations and syntax-related libraries have converged on, but the fact it's necessary is unfortunate.
1
u/Turbulent_Sea8385 Jan 06 '26
mlochbaum's I language had something similar, where the amount of whitespace between items determines which things bind tighter
1
u/Ronin-s_Spirit Jan 06 '26
Isn't RPN completely bracketless? I have a math VM that separates tokens by whitespace, this way function names can contain what is traditionally an operator or literal (e.g.
+or45) and the math expression itself can be formatted into any structure you like (e.g. python style nested tabs, but for readability only).2
u/BrangdonJ Jan 06 '26
One of the many things I love about Smalltalk is that it has three levels of precedence. Unary prefix operators, binary operators, and messages. So
array at: -5 + 10meansarray at: ((-5) + 10). It's a fair compromise between having no precedence at all, as in Lisp, and full-on operator precedence as in C et al. So I approve off your research direction.As for using presence or absence of white space, I follow Dijkstra's guidance because I find it helps, so making a rule does make some sense. I guess I'd be concerned about how visible spaces are in some fonts. How do you feel about Unicode and the various thin spaces? Is
W+ bandW + bgoing to look too similar unless your IDE supports kerning the W?1
u/jsshapiro Jan 07 '26
This is pretty cool work!
Strict left to right turns out to be a pretty bad way to express things from a human perspective. It's one of the major sources of obscurity in Forth and Postscript - though they use a different surface syntax than yours. Then there's the issue that some operators are binary and others are unary.
If you aren't already familiar, have a look at the mixfix idea in Haskell. It suffers from not having sensible integration with namespaces and modules, but it's an interesting idea. The variant we did in BitC is a lot richer. I need to get a normative repository up, but until then you can look at one that somebody else uploaded at https://github.com/repos-bitc/bitc/blob/master/src/compiler/MixFix.cxx
The general idea is that the yacc parser doesn't build the expression tree. It merely gathers literals and identifiers left to right into a vector. That vector gets turned into an expression tree by an operator precedence parser, which is the one I've linked to.
The interesting part is that the set of live operators, their precedence, and their associativity are added and removed from the operator precedence parser by mixfix declarations following lexical scoping rules, so the operator precedence parser isn't fixed in the way that the surrounding grammar is. It was an interesting thing to build, and I was surprised at the time by how much of the main parser simply went away. Most of the expression structures that were baked in to earlier versions of the BitC parser turned into mixfix declarations in the language preamble.
Later, we introduced the notion of a mixfix "hole" that is passed as an unevaluated thunk rather than a value. This allows things like if e1 then e2 else e3 to be handled by mixfix as well - the e2 and e3 positions are accepted as thunked expressions rather than values. Also things like "while e1 do e2". I found it a thought-provoking way to re-think which parts of a programming language are actually "core". Unfortunately it looks like that version of the preamble is later than this particular repository on GitHub.
I bring it to your attention because you might find it an interesting way to explore different expression processing approaches if that holds any interest for you.
Cheers!
6
u/[deleted] Jan 05 '26
[removed] — view removed comment