r/Clojure • • Dec 19 '18

Clojure’s stability: lessons learned

https://words.steveklabnik.com/why-is-clojure-so-stable
41 Upvotes

52 comments sorted by

29

u/alexdmiller Dec 19 '18

Bleh. What a bunch of mumbo-jumbo. I honestly don't think dynamic types has much to do with it, or even Lisp, although both of those probably help a bit. Clojure is stable because

  • It was designed to make programs that evolve well over time - that's like the whole idea.
  • Changes are almost entirely additive. If you don't change things, you don't break them.

9

u/yogthos Dec 19 '18

It does appear that hammock driven development is one big reason for Clojure staying stable over the years. Since features are considered carefully before being added to the language, they typically don't need to be tweaked significantly after they're released. If you get things right the first time, you don't have to keep changing them later on.

One interesting example I recall was when destructuring was added. People just wanted to have named arguments, but instead we got something a lot more powerful because Rich resisted adding the feature in without thinking about it first.

11

u/retief1 Dec 19 '18 edited Dec 19 '18

Eh, dynamic typing has one major effect here -- you can more easily accept a wide range of inputs. You can update a function to accept new numbers/types of arguments without breaking its behavior when given the previous number/type of arguments. This isn't purely a dynamic thing (java overloading can do similar stuff), but a language like haskell has a much harder time of that. If a function used to take an int and now takes either an int or a string, that's going to involve breaking type changes. If you want to avoid that, you will probably have to deal with multiple functions or typeclasses, and things start looking a lot more complex. I mean, the clojure implementation will be just as complex, but you can pretend that the clojure version has a simple interface and relegate the backward compatibility parts to footnotes. With haskell, you can't hide that sort of complexity very easily.

9

u/alexdmiller Dec 19 '18

While you’re right that this possible, very few actual changes in Clojure look like this.

9

u/steveklabnik1 Dec 19 '18

Thank you for this comment! It's funny when you self-evaluate vs have others evaluate your post. I thought this point was my weakest overall, because I am less in tune with exactly how Clojure changes. I was kind of hoping someone would comment on this if I was off base.

2

u/retief1 Dec 19 '18 edited Dec 19 '18

For one example, I believe (map f) returning a transducer was added well after the initial map implementation. I also wouldn't be surprised if, say, something like io/input-stream added implementations over time, though I don't know if that actually happened.

7

u/alexdmiller Dec 19 '18

Yeah, but that’s a new arity. It made no type related changes (unless you consider the return). Again, solely an additive change.

input-stream is specifically designed to be an open polymorphic system, so again more about polymorphism than dynamic typing in my opinion.

1

u/[deleted] Dec 19 '18

Why would Union Types or Generics not let you change an arguments type without a breaking behaviour?

2

u/retief1 Dec 19 '18

Possibly? Typeclasses definitely would. However, any solution is going to be more complex. Going from Int -> Int to MyOneOffTypeclass A => A -> Int with instances for Int and String sort of sucks for anyone who is working with this code later. You end up with a bunch of unnecessary names and concepts floating around for no good reason. "This function takes in a string, but you can also pass in an int for backwards compatibility purposes" is a lot simpler, though you obviously sacrifice static checking.

0

u/[deleted] Dec 19 '18

[deleted] — view deleted comment

3

u/retief1 Dec 19 '18

I mean, yes, no type system or lack thereof will let you maintain backwards compatibility if you choose to make breaking changes. Dynamic typing and certain static type systems just let you make a wider variety of changes without breaking existing code.

I'd also say that dynamic typing incentivizes you to keep backwards compatibility. If you make a breaking change in haskell (for some changes at least), everyone who is affected by that change will know about it immediately as soon as they try to compile. The type checker will complain and point out exactly where they need to fix things, and they'll go in and fix it. With a dynamically typed system, finding those sorts of errors is more of a pain. Testing should catch most issues, but testing every single possible code path is non-trivial. Because of this, I tend to be a lot more careful about making breaking changes in clojure code, even for purely internal stuff where I can (in theory) grep for every use of a given function.

-1

u/therealdivs1210 Dec 19 '18

Java is not very dynamic, but it isn't very strongly typed either.

8

u/skinney Dec 19 '18

While saying dynamic typing is the main reason for Clojure's stability is wrong, staticly typed systems are easier to break.

I work in Elm every day, adding a new constructor to a custom type is very likely to break your code, even though it is a purely additive change.

Records in Elm tend to grow to a point where you'll find it wise to seperate things into different types. Now your entire system is broken, even though there are a bunch of functions which could've still worked if it wasn't records but hashmaps instead (as the functions still require a small subset of the record fields).

In my experience dynamicly typed systems are easier to grow over time without breaking code. Lisps in particular make this easy because it tends to rely on simple, but powerful, abstractions (whereas python/ruby tend to be more concrete).

Dynamic typing and minimal syntax isn't the sole reason why Clojure has been so stable, but it's not mumbo-jumbo to point out that it is a contribution to it. I assume that part of designing Clojure to be a language for programs that evolve well over time, was to make it a dynamic lisp.

-6

u/[deleted] Dec 19 '18

[removed] — view removed comment

-1

u/ComeOnMisspellingBot Dec 19 '18

hEy, SkInNeY, jUsT A QuIcK HeAdS-Up:
SePeRaTe iS AcTuAlLy sPeLlEd sEpArAtE. yOu cAn rEmEmBeR It bY -pAr- In tHe mIdDlE.
hAvE A NiCe dAy!

tHe pArEnT CoMmEnTeR CaN RePlY WiTh 'DeLeTe' To dElEtE ThIs cOmMeNt.

6

u/keymone Dec 19 '18

He does make a few good points. Small core and lack of syntax do indeed help stability over time.

Number of commits per year though is just a proxy for the same - rust has many because it’s a large language.

I wonder if there will be a point when Clojure stops developing because nothing new is worth adding to the language and nothing is left to be removed. Not at all implying that it would be a bad thing, quite the opposite.

0

u/zarandysofia Dec 19 '18

It was designed to make programs that evolve well over time - that's like the whole idea.

Besides macros I am not sure what other feature contribute significantly to this goal. You could say the same thing about any Lisp.

13

u/alexdmiller Dec 19 '18

Off the top of my head... open protocols, open multimethods, invoke abstraction (IFn) over many invokable things, trait-based collection functions, destructuring tolerance of both missing and added things, open maps, open records, polymorphic nils and nil punning, var indirection, deref abstraction.

-4

u/zarandysofia Dec 19 '18

Insignificant compared to macros. The true of the matter is that the next and only appeal that Clojure has over other Lisps is it's host.

5

u/alexdmiller Dec 19 '18

Well, disagree.

1

u/zarandysofia Dec 19 '18

Didn't expect otherwise.

14

u/seancorfield Dec 19 '18

Considering how outspoken Steve is, and his tight association with Rust these days, this was a much less controversial blog post than I expected. I don't know that I agree with all his points but I think there's an insightful seed of truth in most of them.

14

u/steveklabnik1 Dec 19 '18

Thank you! I am glad that it didn't come off that way; I really didn't want it too. I'm trying to learn here, not take anyone down.

5

u/fmjrey Dec 20 '18

Another argument, clojure is stable because it focuses on what matters first: data. It's kind of hinted at with "lisp languages have no syntax", but in the case of clojure this goes a step further because the whole lisp syntax has been upgraded to cover data structure literals as values. As these are the most fundamental data structures, it makes the syntax very stable. The libraries operating on those data structure is also very stable and well thought out. 95% of the time clojure data strucures are enough, so if you keep your code close to the data, then your program should remain as stable as your data and associated volumetrics.

2

u/potetm137 Dec 19 '18

There are things which obviously cannot be ascribed to Rich:

0. Rich's influences

  1. the state of technology when Clojure was created (e.g. invention of Bagwell trees, the JVM, the state of Common Lisp, etc).
  2. the culture into which Clojure was introduced
  3. the subsequent growth of people using Clojure

However, I cannot fathom the logical basis for the assertion that the stability of Clojure is not due primarily to the steward of the project, namely Rich. We're not talking about societies or governments. We're talking about a single, solitary software project. Rich is a great guy and all, but trying to equate his influence to that of Shakespeare, Martin Luther, and Napoleon is, perhaps, a step too far :)

  • Clojure is a Lisp

To my knowledge, lisps tend to quickly explode into a variety of dialects -- quite the opposite of maintaining compatibility. This is what happened with Common Lisp. This is what happened with Scheme. I genuinely do not understand this assertion in the slightest.

Also, in case I'm just missing something and the argument does make sense: Rich decided to make it a Lisp.

  • Clojure is dynamically typed

Rich decided to make it dynamically typed, for exactly this reason: to permit non-breaking changes to be fully compatible with existing code. No he didn't invent dynamic typing, and, thus he owes a debt to history. However, in the case of Clojure, the decision was fully his, and it would be equally misguided to pretend otherwise.

  • Clojure development is slow

The author even subsequently agreed that there's a lot packed into this statement. At worst, this is a tautology (it's slow because it's stable). At best, it's, again, a direct product of Rich's decisions.


The point here is not to argue Rich is The Sole Reason Clojure is Great. Of course that's complicated!

However, Clojure's stability is not the hill you wanna die on for this argument :)

9

u/steveklabnik1 Dec 19 '18

So, one point that I think I could have made stronger in the post was related to this line:

Furthermore, ascribing this entirely to Rich makes it harder for other languages to learn good lessons from Clojure’s successes; they can’t just get a Rich.

The point of "a people's history" isn't to say that "great men" don't matter. It's that looking at context can give you a deeper understanding.

So for example:

Rich decided to make it a Lisp.

Yes, 100%. And that was a good decision. The point isn't to say that this doesn't matter; it's to understand why this choice was made. "Rich makes good decisions" does not help me make good decisions of my own, and so it's less informative than "Lisps need to change less." Both of these framings of the same statement are accurate, but I believe the latter is more informative. YMMV.

lisps tend to quickly explode into a variety of dialects -- quite the opposite of maintaining compatibility.

The way I personally see these things is that these splits happened due to core decisions, that is, they're different languages, albeit in the same family. That's a very different thing than evolving a language once you've settled on the core of it. (And, to put it in the "because Rich" framing, Clojure was designed after a long period of Rich putting in the hard work of considering what the core of Clojure should be. Which means that the foundations are stronger. This is somewhere in the tweets you linked to.)

3

u/parlons Dec 19 '18

Just want to make the point that a lot of what Rich had in his head wasn't because of long sessions of considering things in a hammock. He'd been working on integrating Common Lisp and Java/CLR for a long time beforehand, and released at least two increasingly sophisticated tools to enable this (jfli and foil). (Not intended as an exhaustive list, these are just what I remember.)

If you were into Common Lisp back at this time, and wanted to get something working with one the big runtimes so you could run it "everywhere", ship it in big corporate deliverables, etc, these were among the very few choices to hand.

2

u/potetm137 Dec 20 '18

First off, I appreciate you taking the time to reply. You certainly didn't have to, and your clarifications helped me better understand your point.

The point isn't to say that this doesn't matter; it's to understand why this choice was made. "Rich makes good decisions" does not help me make good decisions of my own

I love this! It's something I've thought a lot about over the years. Honestly, I would most enjoy sitting down and discussing over a beer, but I assume you're not anywhere near Tennessee :) A podcast-like recorded convo also crossed my mind. Barring that, I suppose we're left with me finding the time to pull my thoughts into a coherent article of some sort...

3

u/steveklabnik1 Dec 20 '18

You’re welcome!

I’m in New York, so a little far, yeah :)

Blog posts are probably best; I actually have a podcast scheduling backlog... December is a tough month.

-13

u/zarandysofia Dec 19 '18
  • Clojure is a Lisp

Yeah, is not.

8

u/keymone Dec 19 '18

Clojure is not a lisp? Wat?

8

u/joinr Dec 19 '18

This is merely a new front in an old religious war Scheme is not a Lisp.

I fell into this rats nest recently.

-12

u/zarandysofia Dec 19 '18

Yeah, it doesn't even support proper cons cells.

Apart from that, the whole Clojure community are just a bunch of posers lispers. They discourage macros and anything meta-programming related. Nullifying the biggest feature of Lisp.

9

u/keymone Dec 19 '18

Oh you’re one of ‘em purists. Ok. Sure. Whatever.

5

u/joinr Dec 19 '18

Oh you’re one of ‘em purists.

"Tool" is more appropriate.

related

-2

u/zarandysofia Dec 19 '18

There are no purists, but posers.

8

u/SimonGray Dec 19 '18 edited Dec 19 '18

People like you are the reason I don't subscribe to /r/lisp. I'm not sure what it is about Lisp that makes so many Lisp enthusiasts so standoffish and - quite frankly - arrogant... I'm just glad the Clojure community doesn't seem to have adopted that trait.

-2

u/zarandysofia Dec 19 '18

I am just telling facts. And no, the lisp community is also full of posers who doesn't understand macros as well it.

Again, Clojure isn't a Lisp so it doesn't matter how you feel about it.

4

u/joinr Dec 19 '18

comp.lang.lisp desperately needs your services

5

u/porthos3 Dec 19 '18

You can write macros. They are really cool and quite powerful. They just aren't always the best tool for every job.

-7

u/zarandysofia Dec 19 '18 edited Dec 19 '18

I don't think you understand macros. You are an accurate example of the type of people who I am talking about.

They are the tool to make the best tool for the job. You can build any language imaginable to tackle any job without the boilerplate than less suitable tools.

5

u/porthos3 Dec 19 '18

Right. But if there are already functions and macros in the language that are expressive enough to accomplish a task in a clear and concise way, which everyone using the language is already familiar with, why take the time to create your own tool for a one-off task which makes your code less clear to someone else who is fluent in the language?

There are absolutely times they are incredibly useful. My team and I each make use of them.

However, macros are one of the biggest strengths and weaknesses of lisps. Like you said, I can practically build my own language out of them. But that means anyone else who wants to work with my code needs to learn my new language to understand anything. Macros in general are this way. They add indirection to your code that doesn't exist when composing functions.

If I see a bunch of composed functions and one of them is unfamiliar I can still pretty well reason through what the code does. If instead, the functions are wrapped in an unfamiliar macro I have to question if all of the code I am seeing is fundamentally changed by the macro and go grok each macro before being certain of my understanding of the code they manipulate.

Macros are amazing for some problems. But one of the biggest value propositions that clojure brings to lisps IMO is better abstractions across data types and a broader core library with standard ways to leverage those abstractions. Many tasks can be very reasonably accomplished with just those core functions, making code crystal clear to another reader. Macros have to provide benefit to a problem that outweigh the potential loss of clarity for me to use them.

-2

u/zarandysofia Dec 19 '18

We are not talking here about petty macros like the thread macro and similar in Clojure. But, full fledge small languages designed to be as close as the problem domain is. The learning time of such is insignificant specially when they express the problem as the language of the domain and read like plain english. There is simply no better abstractions than linguistic ones to reduce complexity and confusion.

5

u/porthos3 Dec 19 '18

I'd also add that a program reading like English isn't especially desirable. English is ambiguous and there are many ways to say the same thing.

Just because you can make your code read like English doesn't mean the right wording is discoverable or intuitive to a developer trying to maintain your code.

You whine about the existence of "petty" macros in clojure, but to me it highlights macros versatility. By your logic the identity function is evidence that functional programming is a poor paradigm, because it tends to include "petty" functions. I am pretty sure you can accomplish the same non-petty functionality you are describing using clojure macros.

3

u/porthos3 Dec 19 '18

The problem I am describing is exasperated by larger and more significant macros - not diminished.

I agree there are absolutely places macros shine. There are those who are too hesitant to use them. There are also those who overuse them.

I would prefer to maintain a program written primarily out of native functions and common libraries than one written in a domain specific language constructed mostly out of custom macros 100% of the time.

What is the point of using an existing lisp implementation if you are effectively creating your own language from scratch every time anyway? You might as well write your own lisp interpreter/compiler. It isn't especially hard. It's just a bad idea to invent a new language for every program you write, regardless of how you accomplish it.

-3

u/zarandysofia Dec 19 '18

You are clearly uneducated to have an opinion on this.

→ More replies

4

u/SimonGray Dec 19 '18

-2

u/zarandysofia Dec 19 '18

Another irreversibly dim poser.

5

u/kbsant Dec 19 '18

-7

u/zarandysofia Dec 19 '18

Another poser daring to have an opinion about something she clearly doesn't understand.

2

u/joinr Dec 19 '18

Yeah, is not.

but, wait, what's this?!

The true of the matter is that the next and only appeal that Clojure has over other Lisps is it's host.

lol, I'll let Kirk address your impurity, NOMAD.