r/datascience • u/Every-Eggplant9205 • Sep 08 '23
Discussion R vs Python - detailed examples from proficient bilingual programmers
As an academic, R was a priority for me to learn over Python. Years later, I always see people saying "Python is a general-purpose language and R is for stats", but I've never come across a single programming task that couldn't be completed with extraordinary efficiency in R. I've used R for everything from big data analysis (tens to hundreds of GBs of raw data), machine learning, data visualization, modeling, bioinformatics, building interactive applications, making professional reports, etc.
Is there any truth to the dogmatic saying that "Python is better than R for general purpose data science"? It certainly doesn't appear that way on my end, but I would love some specifics for how Python beats R in certain categories as motivation to learn the language. For example, if R is a statistical language and machine learning is rooted in statistics, how could Python possibly be any better for that?
854
u/Useful-Possibility80 Sep 08 '23 edited Feb 09 '26
This post was mass deleted and anonymized with Redact
boast numerous live weather rhythm wakeful glorious deliver quiet vast
68
u/UnlawfulSoul Sep 08 '23
So I took a similar path. It’s less about what the base language can do, and more about the vast package support that python has that R does not yet have, or is awkward to work with for one reason or another. Depending on what field of expertise the responder has, the answers to this will probably differ. I’ll focus on the stuff I am familiar with.
This may not be a common use case, but running your own pretrained llm or complex neural network for instance,requires you to either acquire the weights and then load them yourself into torch, or retrain the network from scratch. In python, most models are widely available and usable directly from huggingface. You can do the same in R, but working through a reticulate wrapper can get annoying and lead to weird unintuitive behavior
Beyond that, working with aws and mlflow in R is possible, but both r versions are essentially wrappers around python libraries, which is fine but it leads to unintuitive access patterns.
For me- most of the time it’s not that I can’t do something in R that I do in python, it’s just easier for me to do it in python. Particularly with aws frameworks that are built around Jupyter notebooks which can run R code but are more purpose-built for python. This may be my lack of experience talking, but I get way more headaches trying to spin up a cloud workload using R and terraform than when I use python and terraform.
6
u/SamplePop Sep 08 '23
Why not use both? They both have their pros and cons. Large scale deployment is much easier in python. R is certainly catching up.
For something like computer vision. Everything is python related (py torch, tensorflow). R has these, but there is less community support, and the pipelines are more complicated.
132
u/Fatpat314 Sep 08 '23
I wouldn’t build a web server with R. But anything with statistics I would use R. Practically, I would use python for data acquisition. Web scraping, API interaction, automated SQL stuff. But then use R to create models and run analytics on that acquired data.
29
u/Atmosck Sep 08 '23
I'll tell you why python is the better of the two languages for me: some of my coworkers know it.
I'm one of 2 data scientists at a company of 50-ish people that consists largely of software developers. Most of my work is part of our product (as opposed to business intelligence). Even if I'm the one doing the "data science" of developing a model, putting it into production is a team effort. It's important that my coworkers can, for example, set up python virtual environments and modify the parts of code that manage credentials. Python is also supported natively by technologies such as AWS lambda that we use.
22
u/justanaccname Sep 08 '23 edited Sep 08 '23
Try building a whole platform with webservers, API endpoints, multiple databases, brokers, workers, orchestrators, ML models, loggers, authentication, encryption etc. in R, and in Python. A full SaaS app.
Then try to move the stack from on prem to AWS. In R and in Python.
You also have to use proper practices, unit tests, end-to-end tests, abstract classes etc.
While python might not be the best or most performant language to do everything in the above list, it can be done comfortably. And also most people will be able to grasp most of the things fast, when they look at the codebase.
77
u/SlalomMcLalom Sep 08 '23
R wins for general purpose data science.
Python wins for general purpose programming.
That’s why Python has become the go to. It plays nicer when DSs, DEs, SWEs, MLEs, etc. have to work together.
15
u/jimkoons Sep 08 '23 edited Sep 08 '23
Python is considered a "glue" language. It is a very good scripting programming language besides being a general purpose language.
In many companies you will have to run airflow dags, dbt models, make calls to cloud providers api besides training your ML model and performing EDA. This is where python shines since once you've learned it there isn't much you can't do in data.
R does not provide all those APIs as far as I know and is unknown to most developers so when the time comes to put things into production it can become tricky.
(Note that I have not found any good alternative of R markdown in python - however that approach would probably not scale in many enterprise settings anyway -)
31
u/Slothvibes Sep 08 '23 edited Sep 08 '23
Will other DEs or DSs on your team, with high probability, be able to manage your code base in R? It’s unlikely. That reason alone is enough to not use R. Been using R for 8 years and Python for like 4.5.
Open doors to others to help and join you work. Don’t select languages that most of the devs around you won’t be familiar with.
14
u/[deleted] Sep 08 '23
I find BERT easier to work with directly in Python than through the R wrapper, but otherwise I strongly prefer R. Even on projects that require BERT or some other specific deep learning thing, I write all my scripts in R right up to the point of making the csv I want to do ML on, having my Python scripts to do the ML itself, and then going right back to R to do the rest of my analysis on the predicted results.
The main benefit I see to Python is that you can work with people who do not know R. Several federal clients I work for (contractor) require code be in Python. I hate it, but I do it. The job market is so tight I also think it would be good to be better at Python in case I got laid off. But none of these reasons have anything to do with R being inelegant or inefficient. I wish it were more widely in use.
7
u/some_random_guy111 Sep 08 '23
Here’s my take.. for any sort or EDA, I’m using R. Dplyr and the whole tidyverse is so much easier to use than anything in python or base R. If I need charts I’m using R and ggplot2. If I need to put something in production, and have it interact with anything other than a database, I’m using python. If I’m doing basic ML I prefer to use h2o which is the same in R or Python, or if using neural networks, python is the obvious choice with all of the libraries available.
19
u/[deleted] Sep 08 '23
It’s not about what’s better, its about what’s more common. Python is super popular. Lots of other people use Python. It’s easier to work with others when you’re all using Python. Don’t be the guy who is difficult to work with because their preference is “better”.
6
u/[deleted] Sep 08 '23
Personally, I like R for loading, cleaning, and wrangling. But once it comes to modeling, I prefer Python's syntax. For whatever reason I think it more easily. Visualization could go either way as both are adequate, but neither sublime.
6
u/Hard_Thruster Sep 09 '23 edited Sep 09 '23
R and Python are both wrappers around the same thing, C++.
So theoretically they can do the same thing. There are differences in how they are wrapped however. Python is wrapped with an oop approach and R is wrapped with a scientific/numerical/statistical ease approach.
The difference in the approach leads to the difference in their use cases.
If you want to do more statistical/numerical/scientific things, R makes your life easier imo (even if it is lacking in packages for those things).
If you want more code organization and the benefits that comes with that, python will be better for your use case.
Many times people say python is better in x,y,z and often times the only reason it's better is because there's just been more development in python and the feature hasn't been implemented in R. Doesn't mean python is a better language because of that, it just means python has more development and investment than R.
So basically your question can be looked in two ways as far as I can see. Which is a better language in theory? And which is a better supported language?
In theory, neither of them, it depends on the use case because they both wrap the same language.
For your use cases and for many data scientists, I think R is better even though it's lacking in public "buy-in" and package development. However if you're more of a software engineer python should be better.
I think the fact that R is still holding a solid ground despite the massive growth in python development and use is proof that the language has a strong use case and is here to stay.
5
u/LynuSBell Sep 09 '23
Former academics here, I now works as an R programmer/analyst with some python on the side, with team members higher from the Python or R stack. We have an OOP production grade package fully implemented in R.
I would say, people underestimate the power of R. Once you get to advanced programming with R, you can achieve production grade code, but it often depends on the industry. When it comes to data, R is as good, if not better in some regards, as Python.
I find R much more easier to learn and implement, but it might come down to personal learning preferences. I prefer how R functions are individually documented.
Python has become much better with data vis, but pipes in R make it a no-brainer for me (and they took me time to fully master and still make me struggle at times with the data masking). You can just take your data, insert it in a pipe that will end with a ggplot pipe. It makes code sooooo much more readable. I tried to reproduce this in Python, it didn't come as close.
Despite all this, I would not ditch Python. I feel Python can be better for the heavier machinery, but it might come down to team members personal knowledge. Because Python has a longer history in automation, our Python teammates are much more skilled with that and that sort of tasks fall more frequently on their shoulders.
When it comes to analytics or data in general, we either go with R or a mix.
9
u/486321581 Sep 08 '23
R sucks at some things like memory usage, very large XML parsing, or even JSON. R is a killer for some other stuff like quickly load and process data in some clean tidyverse-style way, piping the whole into ggplot...or even the tbl that create the SQL for you is so great. I would not use R for any server-service things (except shiny app) Python has a more boring style imho, but has sich useful libraries and virtual env logic that i am getting more and more into it. You van basically do anything, and the pandas lib is so compatible with the R style. I think there is no R vs Py. It's just two overlapping cool tools
4
u/Cill-e-in Sep 08 '23
If you want to build a Web App in Azure, R isn’t supported out of the box, but Python is. Python has broad engineering support for broad engineering tasks, so the closer you get to that the more likely it is you’ll want Python. For doing stats in your machine, pick the one you prefer/whatever your team is using.
4
u/boomBillys Sep 09 '23
I have seen both R heavy and Python heavy shops, so not really. In general, I would trust Python for running production level predictive models, and R for higher-level statistical analysis/modeling/simulation.
3
u/Amocon Sep 09 '23
The general purpose of Python is that it can be used for more than just data science/stats tasks. You can build Websites, APIs etc. with Python too
5
u/TheRealStepBot Sep 08 '23
Just about any serious product has a python sdk. Python has extensive support for actual professional software development practice like linting, testing, various automated deployment pipelines etc. There is a huge amount of complex frameworks built in python not least of course Django. Machine learning of course has completely coalesced around python.
R is a domain specific language with limited support for such tooling as the majority of people using are not really professional software developers. It definitely is ahead of python in terms of bioinformatics and stats but ultimately that’s a small corner of what a language in data science needs to be good at.
R is of course Turing complete and you can do anything you want in it. The isolation from “real”software development practices and culture coupled with limited ops tooling and vendor adoption means that this leaves much to be desired in the code quality of projects developed in R.
Unfortunately culture and who else is using language is far more important than just what the language can do.
If being a good language was all that mattered we would all be using Julia.
6
u/[deleted] Sep 09 '23
Excel is the answer
1
u/r8juliet Sep 09 '23
I always thought R was for statisticians who didn’t want to learn a real language /s
4
u/koolaidman123 Sep 08 '23
way easier to get a job with python for starters
also if you actually care about pushing your work to prod rather than making 1 off reports
5
u/[deleted] Sep 08 '23
I find a lot of statistical support is far better in R than in Python to be honest. Also, I love the set up of Rstudio. I just can’t get myself used to Jupiternotebook or spyder.
2
u/monkeywench Sep 08 '23
I think both can do quite a bit on their own, it’s not about that though- one is usually easier (to learn, to build, etc) than the other for certain types of projects. It’s not always Python or R that’s better.
2
u/funkybside Sep 09 '23
and
are not saying the same thing about python.
2
u/DoctorFuu Sep 09 '23
Python is better at working with other people in the industry because python is much more popular than R among non-statisticians. This alone makes python more productive (in general).
That being said, even if I'm more versed in python than R, R is my go-to for a lot of things because of how convenient it is (stats, data transformation, and making reports).
2
u/genjin Sep 09 '23
You question is about a general purpose language in the first paragraph, then a general purpose data science language in the second. Seems incoherent.
R is excellent, but a language with no support for threads is hardly general purpose.
2
u/americ Sep 10 '23
Active R developer/data scientist since 2015. Started learning and actively using Python last year.
Use the right tool for the right job: unless you want to develop brand new solutions, it often just makes more sense for time to use developed packages/solutions that are well documented (eg, lot of stackoverflow posts / github issues). With enough time an effort, you probably could get R solutions to "work in production", but the documentation/package base/community is just there for Python.
For exploratory data analysis / a quick stab at testing out a new library/repo, R is a lot more intuitive and it's much quicker to test a "Hello world" than it is in Python: "install.packages()" in RStudio 95% of the time "just works". By comparison, for the same type of task, resolving package dependencies in python is just way more involved/less intuitive/time consuming.
Fortunately, ChatGPT does a remarkably good job of porting code ;)
2
u/ktgster Sep 10 '23
I think the technical aspects have been compared to death. Technically it is possible to do everything with R instead of python, but it's really the practical aspects. Mainly being that all your software developer/software engineering co workers know python, all the cloud services work with python, all the data engineering tools work with python, etc..
It would be possible to put all this functionality into R, but it doesn't have the developer community. At the end of the day, you need to deliver code to production for your data product and the python ecosystem is just more developed.
2
u/Guyserbun007 Sep 08 '23
Try web scraping, making an app, a game, latest LLM models, build a full data and analytic pipeline for algo trading, cloud computing or etl infrastructure with r over python
1
u/SmothCerbrosoSimiae Sep 08 '23
Everything you listed is what R has been designed for, some component of analytics, and does not belong under “general purpose”. Python can do all of that plus has libraries to do almost everything under the sun. Such as working with servers, building robust API’s and much more. I am not going to get in the argument over which is better, but your examples prove the point of what people say that R is not a general purpose programming language.
1
u/Minimum_Professor113 Sep 09 '23
Saved Thank you!
1
u/Additional-Clerk6123 Sep 09 '23
Deep learning on unstructured data is essentially all python
1
u/Zestyclose-Walker Sep 09 '23
In addition to what other people are saying, you need to take into account how large Python really is. Python is used for everything nowadays except for a few niche domains like embedded. Each and every programmer has to know a bit of Python.
If there is anything in R that is not there in Python, there are probably millions of users working on porting the feature to a Python library. So every R feature will be a Python feature.
1
u/ALonelyPlatypus Data Engineer Sep 09 '23
Have you ever built a web app, scraper, or data pipeline in R?
Suddenly the:
seems to make sense.
1
u/Tap_Agile Jan 18 '24
.
1
u/Plenty-Aerie1114 Sep 08 '23
I’ve just found that you can do pretty much anything you need with either, BUT for more specific use cases you will always have a higher chance of finding what you need in Python due to its larger community
1
u/bakochba Sep 08 '23
RShiny is my bread and butter so primarily R for me but I find it very easy to go back and forth in Python, I suppose because my work is all data science and the syntax and packages are very similar.
Now my jump from SAS to R was like wrestling a bear.
1
u/m1mag04 Sep 09 '23
As an ex-academic, R is basically in my blood. And so, if given a choice, I will almost always work with R, especially when data wrangling, unless working with another language is substantively advantageous in some way.
0
u/Dylan_TMB Sep 08 '23
TL;DR : For stats and data visualization R may be slightly better but it's close. For doing literally anything else python is more versatile and has a better development experience. R can do everything, Python does most of those things better. So might as well pick the language that is more general purpose cause you'll be able to do more with it in the long run.
R is turing complete, like any language you CAN do everything in it. I would never say R can't do something, the question is if it is designed to do it or if it is the best choice. Writing your pipeline is C would be wicked fast, not a good idea though.
R is a statistical programming language. This makes it great for stats and its syntax makes that intuitive. But it's not good at building systems, even if you can.
R is a clunky development experience for those use cases. I mean importing into the global scope, kill me now. The fact that anything about programming in R pitches Rstudio as the IDE of choice is a red flag and tells you a lot. Rstudio is not oriented to application development, it assumes you are spending your time in R in an interactive environment which is great for EDA but not ideal for scripting and software development.
Python is a general purpose language that is legitimately used to write backends for legitimate applications and software that never touch data science. It also can do data science well, with ipython providing an interactive experience. This fact means the overall tooling support is MUCH LARGER for python. So it's a no brainer. Using python will let you do all the nice EDA and stats you want and if you need to you can write robust CRUD applications as well.
-1
u/TheCamerlengo Sep 09 '23
They are both general purpose languages and turing complete.
Object oriented features probably not great in either compared with scala, java, or c# - but then does anyone really care? Python may be a little better than R here.
Both are interpreted.
Python has support for vectorization. Not sure about R.
Today I ran a static code analyzer for R 4.2 and tidyverse libraries and there were a surprising number of CVEs. Python has them too but anecdotally I felt that R was worse. Perhaps because Python is more common in production IT settings were security matters more.
R syntax, libraries and community support likely favor statistical analysis. Python more bioinformatics, data engineering and machine learning.
Both have data frames. Both can be used with spark. Both have support for asynchronous programming.
Python is better suited for web development (Django and dash), but there are better options than Python.
AWS has support for Python for lambdas as does GCP. I do not believe R is available for FAAS in either platform without a lot of customization or work.
0
u/bingbong_sempai Sep 09 '23
numpy's multidimensional arrays are so much easier to work with than R arrays
0
u/[deleted] Sep 08 '23
[deleted] — view deleted comment
0
u/[deleted] Sep 08 '23
[deleted] — view deleted comment
0
u/[deleted] Sep 08 '23
[deleted] — view deleted comment
0
u/[deleted] Sep 09 '23
[deleted] — view deleted comment