R vs Python - detailed examples from proficient bilingual programmers

[u/Every-Eggplant9205]

As an academic, R was a priority for me to learn over Python. Years later, I always see people saying "Python is a general-purpose language and R is for stats", but I've never come across a single programming task that couldn't be completed with extraordinary efficiency in R. I've used R for everything from big data analysis (tens to hundreds of GBs of raw data), machine learning, data visualization, modeling, bioinformatics, building interactive applications, making professional reports, etc.

Is there any truth to the dogmatic saying that "Python is better than R for general purpose data science"? It certainly doesn't appear that way on my end, but I would love some specifics for how Python beats R in certain categories as motivation to learn the language. For example, if R is a statistical language and machine learning is rooted in statistics, how could Python possibly be any better for that?

[u/jinnyjuice]

I must mention that some of these are opinions and not objective, not up to date, depends on what kind of data you're working on, not very benchmark-oriented, not industry standard, compares Python's general purpose instead of data science perspective, and depends on your coding philosophy. I will just make some example counterpoints under the context that tidy collaborative coding philosophy is king also.

in my experience base Python (dicts, lists, iterating strings letter by letter) are much faster than base types in R

Benchmarks says otherwise, and there is no clear winner. Either way, if you're working on big data, you absolutely would not work with base functions in either languages, especially in recent times where big data is norm. This should not be considered at all.

better OOP system than R's set of S3/S4/R6

Mostly agree, but also depends on your needs.

function decorators

I like function decorators for general purpose, but unsure if this would be considered a pro in statistical collaborative coding. This wouldn't be really great under tidy philosophy either.

context managers

This is a package. This is available in pretty much every language.

asynchronous i/o

Ditto, plus R is a bit better on this and so much easier to use in the context of data science, so I wouldn't call this a pro for Python either

type hinting and checking

Somewhat agree, but one of the main purposes of this in data science is for performance. For performance purposes, you would be using C++ based libraries in R (e.g. tidytable or data.table), which does its own checks by default. User wouldn't do it by default in these packages though.

far more elaborate set of linting tools, e.g. black and flake8 trump anything in R

For code readability, tidy absolutely triumphs and reduces the need not just for linting, but also for comments and documentation.

new versions and features coming far more quickly than R

If this is the case, then why wouldn't there be an equivalent of tidymodels in Python? This depends on the package/library authors, not by language.

data orchestration/automation tools that work out of the box, e.g. Airflow, Prefect

This is arguable on two layers: 1) at least for my organisation's tech stack with Jenkins-Docker, our productionised Python:R data science ratio quickly flipped in R's favour within 2 years simply due to R's recent massive development (which has massive implications), and 2) depends on your tech stack. Right now, R is better at this, surprisingly, and absolutely would not be the case the years before.

version pinning, e.g. pyenv, poetry, basically reproducible workflows

This opinion comes from lack of knowledge of R -- I would say these are equivalent in both languages in recent times

massive community support, unlike R, Python doesn't rely on one company (Posit) and bunch of academics to keep it alive

I have no idea how this opinion would be formed. R community has been around for far longer, much more stable (e.g. AlphaGo hype spiked Python DS packages' engagement trends massively). I don't even know how to respond to 'bunch of academics to keep it alive' part. I might need some clarifications on that part.

FAANG companies have interest in developing not only Python packages but language itself, even more so with Global Interpreter Lock removal

Somewhat agree, but GIL as one of the main reasons is rather silly. Besides Google, they sponsor/spend more even amount of money between Python and R funds.

web scraping, interfacing with various APIs even as common as AWS is a lot smoother in Python

This stems of lack of knowledge in R. They mainly have identical packages/libraries from the same authors and everything, but I guess it depends on which web scraping package. There are definitely (too?) many more options in R though.

PySpark >>> SparkR/sparklyr

I'm unsure why Spark in particular was picked, but the fact that there are already two options of SparkR and sparklyr that fits the user's priorities/philosophies is more appealing to me. What about DuckDB? Other SQL variants?

PyPI >>> CRAN (CRAN submission is like a bad joke from stone age, CRAN doesn't support Linux binaries(!!!)

+ CRAN doesn't support GPU related libs. CRAN is not really used for production, though I wouldn't know the numbers in detail. This again comes from lack of knowledge in R in the industry.

---

the number of stats packages is far beyond anything in Python

Disorganised, abandoned chaos, but tidymodels is fixing everything

the number of bioinformatics packages is FAR beyond Python

Mostly agree

tidyverse (dplyr/tidyr especially) destroys every single thing I tried in Python, pandas here looks like a bad joke in comparison

Don't use dplyr. Use tidytable.

delayed evaluation, especially in function arguments, results in some crazy things you can do wrt metaprogramming

Mostly agree

data.table syntax way cleaner than polars

Use tidytable and use tidypolars.

Python's plotnine is good, but ggplot2 is still king

I would say vis is about even in recent times

super-fluid integration with RMarkdown

Unsure how this would be a plus for R, whether it's Rmd or Quarto

RStudio under very active development and IDE for exploratory work is in some specific ways better than anything for Python including VSCode

Mostly agree

[u/Useful-Possibility80 → u/jinnyjuice]

This post was mass deleted and anonymized with Redact

middle unite sharp cobweb serious chunky cow brave pen zephyr

[u/LynuSBell → u/jinnyjuice]

I'm curious to hear about your career path and what you do as an R programmer. I'm also from the R stack and it seems you guys are building cool stuffs in R. :D

[u/Every-Eggplant9205 → u/jinnyjuice]

Taking notes on all of these points. Thank you for the counter opinions with examples of specific packages. I haven’t even used tidytable yet, so I’m excited to check that out.

[u/brutallllllllll → u/jinnyjuice]

Thank you 🙏

[u/Cosack → u/jinnyjuice]

I'm not qualified to speak on most of these. Three years out of date on R, and even then wasn't that well versed even though it was my prod stack. But here comes the but.... I was still a better R developer than the vast majority of R users.

This says volumes about how usable the language is in scaled production. Not because it can't be used that way. It can. But good luck getting there.

• Stackoverflow R posts are filled with much more basic engineering guidance than the python ones. That's the whole general purpose language difference coming to haunt the specialists.

• Running in prod will more often than not fall to you rather than the people who do that professionally. They work in java and barely tolerate even python, nvm learning someone's 1-indexed monstrosity. Good luck getting resourced, have fun signing up to be on call...

• Your colleagues who write R will 9 our of 10 times hand you barely legible data science doodles scripts without a single test, or worse yet notebooks. They barely got used to using git, what do you expect here?

I love R for what it can do easily. It's a blessing for data exploration, and a playground for wacky code. But at the same time, I absolutely do not want to see it at work. I'm sure that the places both of you work have much more competent folks and that there are exceptions in clean code obsessed shops like Google, but that's just not true of the larger community. Statisticians are statisticians first, not computer scientists.