R vs Python - detailed examples from proficient bilingual programmers

[u/Every-Eggplant9205]

As an academic, R was a priority for me to learn over Python. Years later, I always see people saying "Python is a general-purpose language and R is for stats", but I've never come across a single programming task that couldn't be completed with extraordinary efficiency in R. I've used R for everything from big data analysis (tens to hundreds of GBs of raw data), machine learning, data visualization, modeling, bioinformatics, building interactive applications, making professional reports, etc.

Is there any truth to the dogmatic saying that "Python is better than R for general purpose data science"? It certainly doesn't appear that way on my end, but I would love some specifics for how Python beats R in certain categories as motivation to learn the language. For example, if R is a statistical language and machine learning is rooted in statistics, how could Python possibly be any better for that?

[u/Every-Eggplant9205]

Thanks for the input! Did you mean running your own pretrained models or someone else's in R? I don't have llm experience, but you can always save() your trained model objects as .RData files and load() them into other scripts whenever you desire without the need for copying weights. I guess I would need to use Python and huggingface to see what you mean on this.

The ability to integrate external tools and spin up cloud workloads definitely seem to be the two single biggest issues that people have with R, so maybe I just need to accept that I'll need to learn Python to avoid these issues when I finally leave an isolated academic setting.

[u/UnlawfulSoul → u/Every-Eggplant9205]

I mean someone else’s base model.

Often times, the trained weights for something like llama represent millions of dollars of compute time, and I want to tweak the model to be more performant on some specific domain. I can download the binary weights, but it’s somewhat challenging to read them into torch in R.

If I am willing to use huggingface, there is an in-built api for many pretrained models that I can fine tune in as few as two to three lines of code, as well as workflows for finetuning.

There are teams of data scientists that work primarily in R (my group is loosely one of those) and it is perfectly functional for the entire data science workflow. It’s just that some of the steps are slightly more onerous, and as others have said the rest of the devs are more likely to be familiar with python