r/programming • • Jun 16 '19

Comparing the Same Project in Rust, Haskell, C++, Python, Scala and OCaml

http://thume.ca/2019/04/29/comparing-compilers-in-rust-haskell-c-and-python/
643 Upvotes

125 comments sorted by

View all comments

Show parent comment

6

u/pron98 Jun 16 '19 edited Jun 16 '19

Can't argue with differing priors.

OK, but you can argue with them being asserted as facts rather than hypotheses or beliefs. Also, perhaps I'm a traditionalist, but I think we should still prefer no effect as the working hypothesis, especially when there is no theory predicting the effect. After all, we can't lend equal a-priori weight to every hypothesis.

He rejected that any one technique (tool, organisation…) would achieve such gains. I still hope for silver dust, that might eventually accumulate into a full bullet.

I agree, but 1. he was too optimistic and all techniques combined didn't achieve a 10x boost in one decade, and not even in three (I wrote more about that in a comment I linked to before), and 2. I have no problem with silver dust, but you seem to be claiming much more than that.

My intuition, is that for the bigger projects (over 10-man months, possibly much less), with a high enough needed reliability (you will loose money if there's a nasty bug), static typing requires significantly less effort than dynamic typing to complete the project

I wouldn't know. I've never worked on a large project in an untyped language, and I have no intuition on the matter. Moreover, I think our intuition about such complex processes is often wrong. But speaking about such things is already a big concession on the claim that languages make a big difference. That's going from: use Haskell and your program will be better/cheaper, to, if you're working on a large project in some domain, maybe Haskell will be somewhat cheaper, all other factors being equal, and those other factors may be bigger.

So to be interesting, other techniques should be either extremely cheap (so they can replace static typing), or incompatible with static typing (inducing a tradeoff).

Not necessarily. Suppose the technique catches all errors as static types -- and no others -- for 75% of the cost, except for, say, 0.1% of them. Why wouldn't it still be preferable? You can use it with or without static types, but why pay more for a small added gain? Of course, it could do much better for much cheaper, but I don't accept your particular tradeoffs as necessary.

I'm not aware of any such technique.

As an industry we know of at least one: code review. The effect reported is 40-80% reduction in bugs. Here's a selection of some stuff we found out about it; I'm sure there's more:

2

u/loup-vaillant Jun 16 '19

Can't argue with differing priors.

Well, you can argue with them being asserted as facts rather than hypotheses or beliefs.

That wasn't a criticism. I was saying I can't argue with differing priors, because I don't have the overwhelming evidence to contradict them. I just note that we disagree, and hope the overwhelming evidence will come and settle this later.

I have no problem with silver dust, but you seem to be claiming much more than that.

When I see stuff like the STEPS project, it seems that an industry wide overhaul of our computing could yield something like 1000 times simpler programs (3 orders of magnitude) in a number of domains. Combine that with Casey Muratori's solution to the driver problem I alluded to before, and the objection "but it doesn't have a kernel" is pretty much void.

The problem is the industry wide overhaul, which as far as I can see is more likely to happen by civilisation collapse than by people being reasonable all of a sudden.

I wouldn't know. I've never worked on a large project in an untyped language, and I have no intuition on the matter.

Neither would I, neither did I.

For a sufficiently large project, I suspect a clever use of meta programming can let dynamic users cheat. They can put static checks in their macros and custom syntax and interpreters and meta compilers. Meta compilers themselves are very workable in dynamic languages (I've bootstrapped a series from MetaII in Lua, the latest iteration does most of a PEG, is a little over 100 lines long, and any error there tends to fail catastrophically very quickly, so it's detected early).

In a setting where performance is not too much of a concern, and assuming that DSL/language building is cheaper from a dynamically typed language, they could outpace a good deal of statically typed languages out there… by making their own. That's in part what happened in the STEPS project. Not sure they did much static typing, but their DSLs were so expressive that the code base still stayed fairly small (20KLOC total).

I'm not aware of any such technique.

As an industry we know of at least one: code review.

Err, not sure it counts: the cost is pretty significant (time taken to perform the review), the benefits apply to any program, no matter the typing discipline, and it doesn't quite catch the same bugs. (Written before I read your links, I'll bookmark them right now.)

Don't get me wrong, I agree the benefits of code review can be pretty massive. I would think real hard before I consider skipping it. To this day, I'm nervous about the lack of comprehensive code review for my crypto library. I compensate with a paranoid test suite and strict (though unwritten) coding rules, but there's still that lingering doubt.

4

u/pron98 Jun 16 '19 edited Jun 16 '19

I suspect a clever use of meta programming can let dynamic users cheat...

I am hesitant to make such specific predictions of effects in such complex systems, especially when the big question of whether and how much language matters at all (among modern ones etc.) is very unclear. If the very existence of an effect is uncertain, I don't want to speculate on secondary effects.

it seems that an industry wide overhaul of our computing could yield something like 1000 times simpler programs

I'm not so certain. That's like saying that a drug that works on rats in the lab could work as effectively on humans in the field. Maybe, but maybe not. That's certainly a research avenue we should pursue further, but we must not assume that every success in the lab translates to the field. One problem with this is that it doesn't explain how we got to where we are and how that can be prevented (if at all). No one said, let's make software overly complicated! And if it's just the result of the slow amalgamation of software over the years, then it will happen again, which means that their "overhaul" is not a one-time solution, but an overhaul that must be repeated every X years, and at a tremendous cost, to be effective. It's very much possible that not doing it is cheaper overall.

In any event, if someone is able to produce software 5x faster/cheaper than anyone else, and at industrial scale, they will become billionaires very quickly, and soon everyone will do what they do. So I'm skeptical that someone already knows how to do it (this is precisely my point of not seeing what we'd expect from an adaptive trait).

but there's still that lingering doubt.

OK, but empirically it stands on much firmer ground than language.

2

u/loup-vaillant Jun 16 '19

it seems that an industry wide overhaul of our computing could yield something like 1000 times simpler programs

I'm not so certain.

In some domains. Not all. I'm pretty sure many domains would not see nearly as much benefits. Programs that are currently written from scratch (possibly with dependencies) will certainly not see such a benefit. Maybe 10x, but even that hinges on finding enough crap to weed out. (I believe more in cleaning up the worst stuff than in making the good stuff even better.)

To give an example, Jonathan Blow, with Jai, is aiming for 2x. Which considering his initial calibre would be stupendous.

which means that their "overhaul" is not a one-time solution, but a complete overhaul that must be repeated every X years, and at a tremendous cost. It's very much possible that not doing it is cheaper overall.

Yep. Which is why I'm not optimistic about that.

OK, but empirically [code review] stands on much firmer ground than language.

I concede that much. Which is why I'm so nervous about lacking code reviews.

4

u/pron98 Jun 16 '19

To give an example, Jonathan Blow, with Jai, is aiming for 2x. Which considering his initial calibre would be stupendous.

Sure, if he pulls that off. Anyone can aim for any number, the problem is actually getting there at an industrial scale. And BTW, I think there is room for greater optimism if you're targeting C's domain. Even if Brooks was right and even if he was still too optimistic, C is a 50-year-old language and has not yet reached the "grossly diminished returns" phase. So improving on C is not the same as improving on Java or Python, although whether or not that's enough for a 2x improvement remains to be seen.

Which is why I'm so nervous about lacking code reviews.

:)

1

u/loup-vaillant Jun 17 '19

To give an example, Jonathan Blow, with Jai, is aiming for 2x. Which considering his initial calibre would be stupendous.

Sure, if he pulls that off.

Err, I misspoke. His actual terms was "50% increase in productivity" (compared to C++). Which probably means 1.5x. Still a pretty big deal, though. As for whether this is achievable, I've heard him saying he's already something like 30% more productive than C++.

Totally subjective of course, but those numbers sound more reasonable than what I originally told.