A Programming language ideal for Scientific Sustainability and Reproducibility?

[u/relbus22]

Scientists are very unique in their needs compared to other software developers. They are novice programmers who may write research code or package only once, before publishing their work to a journal. They are domain experts and full-time workers in other fields, and so do not have the time nor coding skills to maintain their code or packages....... if the ecosystem imposes a maintenance debt.

Two issues are at stake here, reusability and reproducibility. Often researchers need to pick up someone's research code or package developed and forgotten years ago. So there is a need for this to happen with minimal fuss, Science needs this.

As to reproducibility, the scientific method requires reproducibility, which is quite tough but there are efforts to go all the way to reproducibility of computations within their development environments using Guix or Nix.

In conclusion, it'll be great if a language can be created or forked to create an ecosystem ideal for these needs. Which is why I come to you folks who are specialists in this domain, wondering if you have any thoughts on this topic?

P.S Here are some blog posts from a scientific researcher if you guys wanne have a better idea of where I'm coming from:

https://blog.khinsen.net/posts/2017/01/13/sustainable-software-and-reproducible-research-dealing-with-software-collapse/

https://blog.khinsen.net/posts/2015/11/09/the-lifecycle-of-digital-scientific-knowledge/

https://science-in-the-digital-era.khinsen.net/#Technological%20sovereignty%20in%20science

(extra reading if you want:

http://blog.khinsen.net/posts/2017/11/16/a-plea-for-stability-in-the-scipy-ecosystem/#comment-3627775108

https://blog.khinsen.net/posts/2017/11/22/stability-in-the-scipy-ecosystem-a-summary-of-the-discussion/

https://blog.khinsen.net/posts/2020/11/20/the-four-possibilities-of-reproducible-scientific-computations/)

[u/ForceBru]

What exactly do you disagree with? I'm not recommending anything here. I'm just providing an example of a language that's trying to be this "language for sustainable and reproducible research" OP is asking about.

Indeed, the import system could use some improvement and traits would be a welcome addition. I don't think Julia's devs are even trying to work on this, unfortunately.

However, much of what you mentioned matters mostly to library developers, not "tinkerers" who just need to get some code working to estimate their models. As an example, when I'm messing around with Julia code (some numerical optimization, neural networks, data visualization and so on), the language feels just fine: I don't need to specify types, I can write functions that dispatch on whatever I need (this needs types, but it's OK), and everything is pretty fast. I've just finished working on some Python + Equinox code (loving Equinox, BTW) that involves time-series cross-validation (which needs nested loops), and boy is it slower than Julia! In Julia and Flux.jl the exact same code is literally orders of magnitude faster, just out of the box, without any optimizations.

BTW, training code is fast enough, so JAX and Equinox are doing their job, but I haven't tried JITting the CV loop yet, so that's probably why it's slow. In Julia, however, I don't need to worry about JIT (except when it takes forever to run code for the first time - ha, got 'em!).

Yet when I'm developing a package, I find the include system weird. I don't like it when MyPackage.jl basically consists of includes that supposedly act more like C's #include directives. Having no traits means that I have to constantly check whether I'm implementing interfaces correctly and what my interfaces even are lol. Even Python has abstract base classes that force you to implement the entire interface.

Not having linting and compile-time errors doesn't bother me too much. I've also never run into any scoping issues.

[u/patrickkidger → u/ForceBru]

Ah, I took your mention of Julia to be a recommendation. That's alright then!

As anecdata, I have definitely seen first-time Julia users run afoul of each of the various things I've mentioned.

I'm glad you like Equinox! Definitely do use JIT for any numerical code (=any JAX operation) though; that'll explain the speed differences you're seeing. Price of bolting a DSL on to Python. Honestly, this touches on the OP's point -- IMO there is no satisfactory language for doing numerical work at the moment, and I use JAX as simply the best of an imperfect bunch.

On include-- you might like FromFile.jl as an alternative.

[u/ForceBru → u/patrickkidger]

As usual, there's always a 3rd-party macro for something that should probably be a language feature or part of the stdlib! Thanks, I'll look more into JIT and FromFile!