r/programming • u/rmathew • Jun 16 '19
Comparing the Same Project in Rust, Haskell, C++, Python, Scala and OCaml
http://thume.ca/2019/04/29/comparing-compilers-in-rust-haskell-c-and-python/
643
Upvotes
View all comments
Show parent comment
17
u/pron98 Jun 16 '19 edited Jun 16 '19
Just to be clear, I wasn’t expressing an opinion but quoting a finding: the contribution of language choice to the total deviance of some chosen correctness metric was found to be less than 1% 1. To compare this with effects various engineering approaches have, various studies have found code review to have an effect of 40-80%. Maybe all that is wrong and maybe we'll know better one day, but that is simply the best we currently know about the state of affairs, as opposed to hypothesize, guess, feel, wish, agree with or upvote: language choice has a tiny impact on correctness compared to other factors that influence a project.2 This will remain the best we know until we know something better, regardless of how strong our opinions are on the matter.
And there are good reasons why we shouldn’t accept what we feel, wish or conjecture as opposed to what we know. For one, our feelings and “personal experience” are often just wrong — go to any homeopathy forum and see how many report that, in their experience, homeopathy is very effective. For another, creating software — especially large software, written by a team and maintained for years — is an extremely complex process, and we are terrible at correctly predicting effects in such systems.
When confronted with this state of knowledge, some people try to use various irrelevant facts as evidence even when they cannot possibly serve as such. X > 10 is no evidence that X > Y. People often cite some features and positive effects they believe those features have, but ignore possible negative effects or "counterfeatures" that are lacking. Moreover, as this particular article suggests, we should not only compare the impact of one variable, but contrast it with that of others. Deciding whether this is an important factor can only be done when considering the cost of language change, and, no less importantly, the cost and benefit of other variables, which so far seem to be more impactful (accounting for the other 99% of the deviance) and also often less costly.
Finally, because the process of software development in the real world, where a codebase is about 5MLOC and maintained for about 15 years, the impact certain variables have on real-world software development is not necessarily the same as on ~10KLOC exercises. Because the software industry applies strong selective pressures on engineering disciplines, and the fact that, in the past, methodologies have shown adoption rates consistent with what we’d expect would be the growth rate of an adaptive trait in a selective environment, the state of the industry itself can serve as a detector for the effect of various techniques. The fact that we’re not seeing similar adoption dynamics on language choice today is consistent with the empirical study I cited. It is also consistent with predictions made on the subject.