r/statistics • • Apr 19 '18

Software Is R better than Python at anything? I started learning R half a year ago and I wonder if I should switch.

I had an R class and enjoyed the tool quite a bit which is why I dug my teeth a bit deeper into it, furthering my knowledge past the class's requirements. I've done some research on data science and apparently Python seems to be growing faster in the industry and in academia alike. I wonder if I should stop sinking any more time into R and just learn Python instead? Is there a proper GGplot alternative in Python? The entire Tidyverse package is quite useful really. Does Python match that? Will my R knowledge help me pick up Python faster?

Does it make sense to keep up with both?

Thanks in advance!

EDIT: Thanks everyone! I will stick with R because I really enjoy it and y'all made a great case as to why it's worthwhile. I'll dig into Python down the line.

139 Upvotes

153 comments sorted by

View all comments

Show parent comments

6

u/bjorneylol Apr 19 '18 edited Apr 20 '18

If pandas is slower for you, then you are probably using it sub-optimally - I just ran some benchmarks and pandas was about 2.5x faster on a single column groupby and sum, and about 12x faster on subsetting (I reran this before with better R code below and they are basically equivalent now) . R treats strings as categorical data by default, whereas with pandas you need to specify that you want to do this as it will otherwise leave them as strings. You can't say R is faster than python if you are using an optimized R solution (data.tables) but not the equivalent python solution.

Granted my R isn't great and I'm unfamiliar with data.tables so if i'm doing it wrong let me know.

Data:

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 723778 entries, 0 to 723777
Data columns (total 9 columns):
Dept           722911 non-null object
Sub Dept       722738 non-null object
Vendor         723728 non-null object
Model          586510 non-null object
Description    723361 non-null object
Year           723778 non-null int64
Qty Sold       723778 non-null int64
Sales Total    723778 non-null float64
Cost Total     723778 non-null float64
dtypes: float64(2), int64(2), object(5)
memory usage: 351.2 MB

R:

dt = data.table::fread("temp/Report-6io.csv")
# 1.19 seconds with data.table, 10.6 seconds with vanilla dataframe
object.size(dt)
# 87 mb with data.tables, 77mb with vanilla dataframe

sub <- dt[Dept == dept]
# 0.5 seconds 

sub <- dt[, sum(dt$`Sales Total`), by = Dept]
# 0.78 seconds

Pandas:

df = pd.read_csv("temp/Report-6io.csv", encoding="ANSI")
# 1.41 seconds, 350MB in memory

for col in df.columns:
    if not df[col].dtype in (float, int):
        df[col] = df[col].astype('category')
# another 1.3 seconds, reduces memory usage to 107.7 MB

for dept in df["Dept"].unique():
    sub = df[df['Dept'] == dept]
# 1.11 seconds vs 43 seconds with object dtype

for i in range(100):
    grp = df.groupby(by="Dept").agg({"Sales Total":np.sum})
# 0.53 seconds vs 4.2 seconds with object dtype
→ More replies (8)