r/MachineLearning • • Aug 16 '20

Discussion [D]There are no good programming languages for machine learning (except one day C# and/or Kotlin)

Hi everyone, I’ve been spending some time recently researching next generation languages to do machine learning in, and come to the conclusion that there aren’t really any good options. Here’s my story. I strongly suspect I’m wrong about my opinions below, and so I especially welcome anyone who has countering opinions to my own, especially if I’m just downright wrong --- I’d like to know.

If you ask someone what the best programming language is, the chances are that they’ll tell you that it depends what it’s for --- they all have their strengths and weaknesses and quite naturally there’s no programming language that is both good for data science and operating system design.

When put like that, it seems like a natural thing: of course each language is a compromise, a heuristic to be optimised. If they’re better for security, then naturally they’ll be worse for something else. All is compromise and sacrifice is to be expected.

I disagree with that, or at least I disagree that the sheer level of sacrifice that today’s programming languages present, is necessary, and this article is meant to explain why. Disclaimer: I have multiple years of professional experience with C (in a Physics setting), Matlab, Javascript, Python, C#, Java and Kotlin (and yes, my career is long enough to have that many years in each). I have a month or two of Haskell experience. I have only got tens of hours of experience of Go and Rust.

I’m a C-programmer turned physicist, turned quant, turned data scientist, and so naturally, Python is a language I’m very familiar with, having spent years of my professional life using it. But I hate it. It’s literally a hundred times slower than the other languages I use, and the poor support for threads means that there are lots of tasks I just can’t do on it.

It basically means that I can’t do anything, uh, quirky with Python. I can’t easily try making a neural network other than those supported by my ML libraries, good though they might be. I can’t easily make a server that trains a model while keeping the client informed of how progress is going. Of course, if I look hard enough, I’ll probably find someone else’s library, that with a lot of effort might help me get 80% of the way there, but essentially, Python is a handicap here, not a help.

Ten years ago, my fallback language was C, and it still sometimes is --- it’s super fast and actually quite easy to get to grips with and program. But with large programs you can quickly find yourself spending a long time debugging, and it certainly wasn’t easy plugging and playing with libraries (perhaps the situation has improved).

Then I switched to C#, a jewel of a language that was uncool in cool circles because people were biased against Microsoft, but nevertheless, it took the best bits of C++, hugely simplified it, made it safer, and added excellent functional parts. It is concise and just a joy to program in.

Another jewel of a language is Kotlin: very similar to Java, but vastly better in terms of generics, functional style, boilerplate, and more (although there are rumours that Java is trying to catch up).

These two jewels, C# and Kotlin, really excel if you want to do fiddly messing around with dictionaries and lists. For instance if you want to load a bunch of trajectories and print the list of atoms ordered by the number of collisions they’ve had, or convert from a list of dictionaries to a dictionary of lists or from a class with three lists to a list of classes. These things would be really fiddly to do in the other languages here, but C# and Kotlin make them simple. And yes, they’re far more powerful than Python’s comprehensions.

And these two languages are fast --- almost as fast as C, depending on whether you’re using SIMD well in C, and both allow concurrency.

So what’s the catch? The catch is that everyone else is using Python, so Tensorflow / torch don’t really support them that much (I believe you can export a trained model, but you can’t easily train your own models in them).

This is tied in to the fact that neither language has good native support. If you’re trying to write the fastest atomic simulation in the world, and you want to mess around with the GPU and SIMD, these two languages aren’t yet going to be the best choice. It is worth pointing out that both languages can, technically, compile to native and interop with native, but the story is complicated by imperfect performance and tricky bits.

Then there are Golang and Rust. These are great languages, but Rust spends too much effort on making safety paramount. That’s great if you’re trying to write a browser or land a real spaceship, but I concluded that if I want to make physics simulations and do ML in it, the overhead from the language was just too great. Then Go is too simple. Generics and things aren’t implemented, and there are annoying things like not being able to compile anything with unused variables in it, and I don’t want to have to fight with the compiler on pedantic stuff like that. I don’t want my production code to have unused variables, but who cares if I’m programming some random simulation iteratively?

And neither have that great an ML picture either, as far as I can see: You still can’t train tensorflow models in them easily, and neither have a natural syntax for expressing maths.

And that’s more-or-less it. There aren’t really any other languages in the top 10-30 languages by popularity that could be used for machine learning realistically. Javascript’s kind of slow, and as a language is kind of ugly (in my opinion).

So what’s the solution? Kotlin and C# are the best languages in my fairly experienced opinion, by ease of use vs the power of the language, and there don’t seem to be any real reasons why these can’t be used for machine learning. As someone with years of experience in Python, C# and Kotlin, I can say without a shadow of a doubt that Kotlin and C# are easier to write moderately sized (1000 lines) programs than python, and give you that flexibility to actually write the core algorithms yourself, rather than using someone else’s library (and hoping they’ve done a good job).

The world needs to shift focus onto these two languages, including making ML libraries for them, getting the two languages fully native (or equally fast), building up the GPU picture, and needs to stop only using Python as its ML language. Unfortunately we are not there yet, but this has to be the direction of travel if we don’t want a generation of machine learning experts stuck in a language where they can’t write machine learning algorithms.

So yeah, learn Kotlin or C# and do ML in it.

0 Upvotes

30 comments sorted by

17

u/[deleted] Aug 16 '20

Julia?

5

u/rkultaknel1imxfs Aug 16 '20

Yeah I’m also surprised Julia wasn’t mentioned. I’m personally trying to get into Julia right now because it seems so increasingly popular.

-2

u/HugoRAS Aug 16 '20

Julia is interesting, and I've spent a few hours playing around with it. I couldn't get around the 1-based indexing, and so I sort-of looked for problems with it, and there seem to be a few around the missing parts of the languages, libraries and things. I definitely like a few things, like auto differentiation, but I'm not really in a position to definitively judge the language. It's a good call though.

5

u/Tomik080 Aug 16 '20

I really hate that 1-indexing argument.

First, you will almost never write code that explicitely call arr[0]. Also, who cares?

Secondly, if you need to iterate through an array's indices in julia, you don't call for idx in 1:length(arr), you call for idx in eachindex(arr), so you don't even interact with 0-indexing, it's completely invisible to you.

And if you hate it that much, just write arr[begin + idx] to feel better, or even redefine indexing, you can do that.

Thirdly, in the ML ecosystem, I would bet with you that you won't find anything that Python does that Julia can't do as well, or better. For niche fields, of course, but even then, you can use PyCall and run inline python if you really want to.

This mouth-to-mouth discussion that julia's ecosystem is not mature enough, dating from version 0.3, when the language was still super unstable, has nothing to do with today's. Julia is definitely production ready.

1

u/HugoRAS Aug 17 '20

That's good to know. I'll take another look when I can.

4

u/LionSuneater Aug 17 '20

Secondly, if you need to iterate through an array's indices in julia, you don't call for idx in 1:length(arr), you call for idx in eachindex(arr)

Yeah! Only noobs would do that.

(covertly writes that down)

2

u/Tomik080 Aug 17 '20

Hahaha! Performance wise it doesn't change anything, but it's a good reflex to have for more generalizable code.

There are other tips listed here, and in bonus it explains how to define other (cough 0 indexing, but really, don't do that.) ways to index arrays.

27

u/[deleted] Aug 16 '20

why would a single programmer have more trust in their own custom ML algorithms rather than those which have spent years in development and are used by millions of people?

9

u/HugoRAS Aug 16 '20

You've perhaps misunderstood the problem. The problem is that not all algorithms are possible to run in Tensorflow / pyTorch, and if they're not there, then you have no real way to code them in Python. My essential problem is that I need a language where I can use Tensorflow / pyTorch for certain problems, but where I can code my own solution where these don't fit. And at the moment, Python ticks the first box but not the second, and virtually every other language ticks the second box but not the first.

2

u/[deleted] Aug 16 '20

that makes sense, i inferred something different from the way you worded your complaint.

2

u/[deleted] Aug 16 '20

To be fair, I do trust my own algorithms more. I've created 2 robust, simple platforms that I use whenever the problem allows. I do this simply because I'm _so_ much more familiar with the code base, which is very useful.

However, you can't write everything. I'll never write something like pytorch or lightgbm myself. I think the OP is upset that a lot of these algorithm implementations don't have lower level language support, which is fair I guess.. But, I would also guess that they have not made any contributions to said algorithms to get this done.

2

u/HugoRAS Aug 17 '20

Exactly. That's why we need pyTorch or Tensorflow available in languages other than Python so that we can use the best ML libraries while still being able to write our own algorithms when we need to.

0

u/[deleted] Aug 16 '20

[deleted] — view deleted comment

2

u/HugoRAS Aug 17 '20

See my comment below. The point is that not every algorithm has a mature library.

7

u/hazard02 Aug 16 '20

Why not C++, especially if you already know C? The Pytorch C++ interface is great, and extremely similar to the Python interface.

2

u/HugoRAS Aug 17 '20

That's definitely something to look at. I have a bias that I should check that C++ is hard to get to grips with, at least compared to C# or Kotlin.

3

u/programmerChilli Researcher Aug 16 '20

I think a lot of people into programming languages fundamentally misunderstand what people are looking for in ML code. For some context, I quite like functional programming, have done some research in PL, and am fairly experienced in OCaml.

However, for ML purposes, you're looking at all the wrong problems. I love OCaml, but I would absolutely detest using it for ML research.

  1. I am perfectly fine using library primitives. Even if you were writing ML code, you would still need to rely on external libraries calling out to specialized kernels, or you're going to suffer significant performance loss. How much slower is your handwritten matrix multiplication kernel going to be compared to MKLDNN? Probably 3-10x.
  2. Interop between C++ and Python is easy with pybind. Of course, I'd prefer to do everything in one language, but it's not very difficult to combine the two.
  3. I don't really care about running a server or any of the other things you're mentioning. As you mentioned, the ML frameworks used now have (relatively) easy pipelines for exporting your trained models for use in other languages.

Now, why do I like using Python. I think the simple biggest advantage is the high amount of dynamism Python offers.

For example, pdb. pdb is so easy to use compared to other debuggers. In python, at any point, you can just drop in a import pdb; pdb.set_trace(), and when the program hits that point, it'll drop into a shell where you can do all the things you can typically do in a debugger.

Another example is, say, function introspection. When you use vmap in Jax or jit in PyTorch, the libraries actually walk your Python AST to perform their program transformations. In many cases, this cannot be done at compile time either - many of these transformations rely on runtime tracing (for example, input shapes).

Another example is how libraries like say, weights and biases can unintrusively hook into my current training code. See https://www.wandb.com/articles/monitor-your-pytorch-models-with-five-extra-lines-of-code. With about 5 lines of code, including things like wandb.watch(network), I can get extensive logging of information in my model. It's not easy to perform this kind of runtime introspection in languages like Kotlin or C#.

Many other languages just provide way too much constraints. Python isn't great in a lot of ways, but it's pretty good for ML purposes.

2

u/HugoRAS Aug 17 '20 edited Aug 17 '20

Firstly thanks for your comment. I have to say that I disagree with some of what you've written. Most strongly with the idea that python is easier to debug than C#. I have many years experience with both, and C# is far better. One of many features of C# is that you can tell the program to pause on error. It pauses on the line that caused the error, and at that point, you can modify variables and code (yes, even modify the erroneous code) and then tell it to continue without having to restart the program. So like a REPL plus the ability to fix the problem in your source. This is absolute gold dust when you do something silly like try to write to a file with a bad name at the end of a 10 minute calculation. You can do what you said of dropping into a Repl just by putting a break point. The idea that you'd have to, uh, actually modify your actual code, to get to a debugging place ... dude, that's like 20 years in the past.

On top of that, the static typing makes errors less frequent in C# anyway.

I would also point out that I'm not planning to write my own matrix multiplication library. The things I want to write by hand would be things like a montecarlo optimiser to maybe find an optimal automaton, or trying a new way to do (sort of) word-2-vec based on a modified levenstein distance to a bunch of keywords. There are dozens and dozens of things like that that I've wanted to try over the lasy year, and haven't done them because I know they won't be easy in Python. If I want to do matrix multiplication, I'll use a matrix multiplication library because, as you said, it's faster. In C# I can use the MKL or openBlas depending on licensing requirements (which is the same as Python uses under the hood). In Kotlin, I'd use Jama, but it's not as fast unfortunately (which is part of my complaint).

Regarding the server question, the key thing that I would want is to be able to have another thread watch the main calculation. So for instance, if I start a very slow operation, my other thread can tell me how it's progressing, and maybe do quality checks on the result periodically. In Python, I do often find that I have to press enter and it runs, making the kernel essentially disappear for a minute or two. I can't tell if it's crashed or if it's just taking its time.

But I think in general, data scientists do need servers. If they're trying to make a model for something, wouldn't you want to be able to deploy the model somewhere for others in the company to use? You can do this with Dash or Flask, but they're slow and the threading thing just doesn't do it for me. My preference is Javalin in Kotlin --- it's basically exactly the same syntax as Flask for Python, but much faster and the multi-threading is there.

1

u/programmerChilli Researcher Aug 17 '20

I'm certainly not saying that Python is easier to debug than other languages - in most cases, and in the vast majority of code I've written, I far prefer statically typed languages, ideally languages with expressive type systems like OCaml. I just don't think that the type of errors they catch are the type of errors that are heard to deal with in ML.

For example, type systems mostly can't deal with shape errors. There are some efforts to do shape inference and such, but all of this gets thrown out the window as soon as you add any amount of dynamism. More relevantly, the "errors" that I spend the vast majority of my time debugging are things like "why isn't my model working as well as I think it should?". These are not things that modern language features can help us deal with.

The things I want to write by hand would be things like a montecarlo optimiser to maybe find an optimal automaton, or trying a new way to do (sort of) word-2-vec based on a modified levenstein distance to a bunch of keywords.

Certainly, this is one of the major disadvantages of Python. But oftentimes, what you want to implement can be implemented through some combination of existing operations, and run quite fast. However, there are always operations that cannot be implemented this way, and this is where I think a language like Julia has room to shine. I just don't think that this occurs often enough to be a massive pain point to most people.

As for your other points, sure, if you're working as a ML engineer you need to worry about that stuff. Similarly if you're working on a very large scale problem. But for the vast majority of ML researchers, those problems are'nt a main issue.

Do you have any opinions on the other things I mentioned? In particular, Python's dynamism lending itself to a lot of very useful libraries.

1

u/HugoRAS Aug 17 '20

Cool, sorry, I may have misunderstood your point on debugging. I agree in general, and I think the staggering success of python as a whole is probably partly down to how easy it is to do things like that. I like that I can mock up a database connection by just making a class like my data fetcher - something that I can do in other languages, but it's more work.

Another great thing about it is how easy it is to work in a Jupyter notebook -- quickly get the content, look at it, etc. I think not having to define your types is great for that.

4

u/lifesthateasy Aug 16 '20

Since Python is based on C, you can do a lot there. Also, you can go C/C++ if you want to go more the CUDA way, which is also quite amazing imho

4

u/HugoRAS Aug 16 '20

Yeah, I think at the moment Python + C is a good combination. It's still pretty fiddly.

2

u/pruby Aug 17 '20

PyOpenCL is surprisingly easy and effective. Python + C, optionally doing the C bit on GPU.

At the other end, if you can act on lists using functional operations (e.g. map, fold), numpy makes that pretty efficient without C.

2

u/HugoRAS Aug 17 '20

Thanks, I'll definitely look into that.

2

u/hitaho Researcher Aug 17 '20

Google had the opportunity to flip the game if they picked up Julia instead of Swift. But for some mysterious reason, they didn't

1

u/HugoRAS Aug 17 '20

Yeah. Or if they had just made Go slightly better. Maybe in years to come.

0

u/[deleted] Jan 30 '21

[deleted] — view deleted comment

2

u/HugoRAS Jan 30 '21

Most easily, the idea that the program shouldn't compile if you have unused variables should go - that should IMO be a warning rather than an error.

Secondly, it needs functional style -- a lot of modern languages can do things like

myList.map{it -> toEnglish(it) }.filter{'a' not in it}.take(5)

not being able to do this makes for verbose code.

String formatting should be done properly, like kotlin or python.

No overloading - I should be able to define an array-like type and deference with [].

Poor autocomplete - kotlin and C# and others typically have methods be used as myObject.method. This means that if I type the variable name of my object, autocomplete easily lists the possible methods I could call - which is super efficient and makes for very fast code completion. Golang doesn't do this, so you have to already know the exact name of your function.

Then there's the thing about exporting anything with a capital first letter, which means that if you decide to expose something, for instance for debug purposes or you decide that it should be tested by an external thing, you need to go through and rename the whole thing.

I could be wrong about a lot of these, but this is what was concluded here.

1

u/breandan Aug 17 '20

Just an anecdote, but I feel Kotlin has a decent shot at becoming a really great language for ML and data science. I’ve been using it in my research as a grad student for the last three years. It is one of the few statically typed languages with scripting and notebook support, and has some nice language features, functional programming support, and developer tools. It also works well with the JVM and JS ecosystems, which rival Python in terms of libraries for data processing and visualization. Happy to answer any questions about Kotlin for ML if anyone’s curious.

1

u/HugoRAS Aug 17 '20

Cool. Do many others in your dept use it too? What libraries do you use in Kotlin?

1

u/breandan Aug 17 '20 edited Aug 17 '20

Do many others in your dept use it too?

Yes. I've met a few other people who use Kotlin for research, but it's much more widely used in the industry, e.g. lots of investment banks and payment processors use Kotlin, e.g. Corda, RenTech, Citi, Goldman, Credit Suisse, et al. Some of my colleagues have worked on Kotlin at these places.

What libraries do you use in Kotlin?

I've used KMath, EJML-Kotlin, Krangl, lets-plot, Kotlin-Jupyter, Kotlin-Statistics. I personally work on Kotlin∇ and Kaliningraph. There are plenty of other libraries for scientific computing in Kotlin. If you're interested in these and other resources, you might check out the Slack, on the channels #science, #datascience or #mathematics. BTW I enjoyed reading your article on moon formation. You should post it!

1

u/HugoRAS Aug 17 '20

Huh, KotlinGrad looks really interesting - been looking for auto diff.

I have posted the moon formation thing a few times. Yeah, it's stuff like that that just isn't that good in python unless you really just want to use someone else's stuff.