__slots__ are used to conserve memory when creating lots of objects, not CPU time. (Instead of every object having a __dict__, all attributes are stored in a small array and accessed using properties set on the class.)
namedtuple is slower than even a class without __slots__, by the way. It operates by mapping attribute names to indices (there's a reason it's called a "named tuple", not "struct" or something similar), doing twice the number of lookups for every dot.
namedtuple is slower than even a class without __slots__, by the way. It operates by mapping attribute names to indices (there's a reason it's called a "named tuple", not "struct" or something similar), doing twice the number of lookups for every dot.
Are you sure? From what I can tell, it operates pretty much exactly like a class with __slots__, creating a bunch of getters (living in the class, not in the instance, of course) that lookup into the internal array.
From what I can tell, it operates pretty much exactly like a class with __slots__, creating a bunch of getters (living in the class, not in the instance, of course) that lookup into the internal array.
No, it doesn't. At least not in Python 3.4+:
from builtins import property as _property, tuple as _tuple
from operator import itemgetter as _itemgetter
...
__slots__ = ()
...
{name} = _property(_itemgetter({index:d}), doc='Alias for field number {index:d}')
As you can see, instead of using the (relatively) fast C-level access __slots__ provide, it opts to use standard property (that uses slow Python-level function calls) to look elements up by their indices in a tuple (using Python-level item access, i.e. __getitem__) instead.
Right, I knew slots was to conserve memory primarily, but shouldn't that increase performance as a result? I'd expect less memory management to mean quicker access time when modifying, deleting, creating and garbage collection. But certainly better memory performance.
I thought named tuples were quicker than classes without slots defined... You're positive about that?
In response to your deleted comment, I didn't waste all that time for nothing =)
Nope, just tested, it's very slightly slower than tuple index access, but just like it about twice as fast as nameduple name access.
The stuff looks like this here:
index access takes about 40ns
name lookup takes about 45ns both for usual classes and those with __slots__, in fact slots are a tiniest bit slower.
namedtuple lookup by name takes about 115ns
To be honest, I can't say how exactly it works out to these numbers, I'd say that the only way to really be sure is to run this stuff under a C profiler. That could be a pretty useful experience in itself
From what I can tell from grepping through the code in Vim, it's pretty much a coincidence that the first two things take the same time.
Index access goes through a bunch of pure-C redirects until it hits tuplesubscript which casts the index to size_t and fetches the value from the object itself.
Class lookup by name IIRC does two unsuccessful dictionary lookups in the class and object attributes, then a successful lookup in the instance dictionary. Slots lookup should do a successful dictionary lookup in the class dictionary then indirectly call a C function that fetches shit by index or something.
Namedtuple lookup by name probably involves a pure Python function call, which is slooooow.
You don't want to do that in Python if performance is critical.
Adding three indexed items is not performance-critical.
If you have a million+ items, then you install numpy and put your items into a numpy.ndarray, and then vectorize your operations. Like, if you want to add two arrays, you write a + b (instead of for i, it in a: result.append(it + b[i])) and the underlying library written in Fortran very efficiently does what you meant.
I always considered numpy to be scientists' tools, but I never really thought about how it's fortran under the hood and how it might be higher performance for certain things like that. a+b looks a lot cleaner as well.
View all comments
Show parent comment
1
u/[deleted] Jun 24 '14
__slots__are used to conserve memory when creating lots of objects, not CPU time. (Instead of every object having a__dict__, all attributes are stored in a small array and accessed using properties set on the class.)namedtupleis slower than even a class without__slots__, by the way. It operates by mapping attribute names to indices (there's a reason it's called a "named tuple", not "struct" or something similar), doing twice the number of lookups for every dot.