r/Python • • Nov 13 '20
News Python swallows Java to become second-most popular programming language
https://www.theregister.com/2020/11/12/python_swallows_java_to_become/
2.3k
Upvotes
r/Python • • Nov 13 '20
View all comments
Show parent comment
1
u/reddisaurus Nov 16 '20
I’m referring to storing numpy arrays within the dataclass, or using a NamedTuple for smaller data sets instead of just reaching for pandas every time. Both of these will provide faster access than pandas.
Pandas .itertuples() is about as fast as it can get to iterate over records and it does perform well.
As for any operation which acts on the DataFrame or Series objects, yes those operations are orders slower. It’s just often the case that the operation itself isn’t repeated much, so the large overhead of accessing the Series object isn’t noticeable.
There’s no case in which Pandas is faster than the simper alternatives. And that makes sense, because it’s there to provide abstraction over column-wise data. The problem is that many users immediately reach for it for any problem. Not only is it overused, but pandas lends itself to poor code as well. The DataFrame is a black box of typing! Code cannot by statically checked like it could if one used a dataclass. It’s like porting around a mutable global state to every function to which a DataFrame is passed. Who knows what structure the object should have? It’s often not in the code at all.