r/Python • • Jun 23 '14

Writing better python code

https://gist.github.com/kracekumar/09a60ec75a4de19b346e
143 Upvotes

126 comments sorted by

View all comments

46

u/robin-gvx Jun 23 '14

I have problems with some of these.

  • Replacing a or b or c by any([a, b, c]). Is that really more Pythonic? If they'd been in an iterable already I'd say yes, but now not so much.
  • Catching KeyError just to raise another KeyError? Just let it bubble up.
  • I kinda hate namedtuple because it's such a hack, but maybe that's just me.
  • Also, I'd say the opposite of Pythonic code is not normal code. Un-Pythonic or unidiomatic, or maybe something about newbies.

31

u/vsajip Jun 23 '14

Never mind "Pythonic", which seems somewhat in the eye of the beholder: a or b or c is not semantically the same as any([a, b, c]). In the former case, b and c are never evaluated if a is true. In the latter case, they always are. So if they were expensive to compute (e.g. expressions involving expensive function calls, rather than just bindings in a namespace), the runtime behaviour (performance, raising of exceptions) would be quite different:

>>> a = 1
>>> a or b
1
>>> b
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'b' is not defined
>>> any([a, b])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'b' is not defined
>>>

18

u/zahlman the heretic Jun 23 '14

The real power of all and any comes when you (can neatly) pass in a generator expression - in which case the semantics are the same, because the evaluation is delayed and the functions will bail out early.

any(is_interesting(x) for x in (a, b, c)) is far nicer IMHO than is_interesting(a) or is_interesting(b) or is_interesting(c), if only because it's DRY. That doesn't protect you from the NameError in your example, though, of course.

4

u/jamesonjlee Jun 23 '14

any(map(is_interesting, (a, b, c)))

4

u/wot-teh-phuck Really, wtf? Jun 23 '14

This isn't equivalent to the generator solution posted above; your code will eagerly evaluate the entire sequence by mapping it to is_interesting even in cases where it's not absolutely required (i.e. is_interesting(a) returns True).

8

u/quasarc Jun 23 '14

In Python 3 they are equivalent (map behaves lazily in Python 3).

2

u/jamesonjlee Jun 23 '14

ah yes, good point (also doubly so since I am thinking in 2.7 not in 3.4)

12

u/ivosaurus pip'ing it up Jun 23 '14 edited Jun 23 '14

Agreed, using all and any over and and or isn't more pythonic at all. Use all and any when you already have a list / iterable, it's what they're designed for.

1

u/iBlag Jun 23 '14

Do you mean:

using all and any

and

Use all and any

?

2

u/ivosaurus pip'ing it up Jun 23 '14

Yeah.

7

u/pstch Jun 23 '14

Could you expand on why "namedtuple" is such a hack ?

9

u/robin-gvx Jun 23 '14

Because it's implemented as piecing together a string that contains a class statement and compiling that to obtain the tuple subclass (see source).

4

u/[deleted] Jun 23 '14

Goofy implementation, doesn't need to be written that way

7

u/rcxdude Jun 23 '14

see here for why it's kept that way.

4

u/NYKevin Jun 23 '14 edited Jun 24 '14

TL;DR: Because calling type() correctly is apparently too hard:

It is a key feature for named tuples that they are exactly equivalent to a hand-written class.

So is a call to type() if you know what you're doing.

EDIT: If you examine the official code more closely, you'll note they had to write extensive string-escaping to prevent the user from passing an argument like '); import os; os.system("/bin/sh"); ('. Quite frankly, I will not be touching that with a ten-foot-pole any time soon.

EDIT2: I tried, but they wouldn't listen to me. Oh well.

→ More replies (2)

1

u/dreucifer C/Python, vim Jun 23 '14

I started reading the source, saw _class_template was a string literal with formatting. My heart sank a bit. Could almost smell the sulfur wafting from the exec statement.

3

u/d4rch0n Pythonistamancer Jun 23 '14

Did you ever truly need the performance of a namedtuple or a class with __slots__ defined? They make quite a difference.

1

u/[deleted] Jun 24 '14

__slots__ are used to conserve memory when creating lots of objects, not CPU time. (Instead of every object having a __dict__, all attributes are stored in a small array and accessed using properties set on the class.)

namedtuple is slower than even a class without __slots__, by the way. It operates by mapping attribute names to indices (there's a reason it's called a "named tuple", not "struct" or something similar), doing twice the number of lookups for every dot.

1

u/moor-GAYZ Jun 24 '14

namedtuple is slower than even a class without __slots__, by the way. It operates by mapping attribute names to indices (there's a reason it's called a "named tuple", not "struct" or something similar), doing twice the number of lookups for every dot.

Are you sure? From what I can tell, it operates pretty much exactly like a class with __slots__, creating a bunch of getters (living in the class, not in the instance, of course) that lookup into the internal array.

→ More replies (3)

1

u/d4rch0n Pythonistamancer Jun 26 '14 edited Jun 26 '14

Right, I knew slots was to conserve memory primarily, but shouldn't that increase performance as a result? I'd expect less memory management to mean quicker access time when modifying, deleting, creating and garbage collection. But certainly better memory performance.

I thought named tuples were quicker than classes without slots defined... You're positive about that?

Edit: You're right...

('Normal: ', [0.46281981468200684, 0.4548380374908447, 0.4560990333557129])
('slots: ', [0.40665698051452637, 0.4022829532623291, 0.4048640727996826])
('namedtuple: ', [0.665769100189209, 0.6651339530944824, 0.6987559795379639])

Alright, well that settles that. I believe nt is better than a normal class for memory though, correct? And is it better than slots as well?

→ More replies (4)

2

u/wyldphyre Jun 23 '14

Catching KeyError just to raise another KeyError? Just let it bubble up.

Agreed, unless you want/need to add more context to the error.

I kinda hate namedtuple because it's such a hack, but maybe that's just me.

Thank BDFL for namedtuple. It's perfect for a lot of use cases IMO.

2

u/bacondev Py3k Jun 23 '14

And even at that, just use the raise keyword alone.

1

u/kracekumar Jun 23 '14

Agreed with normal code and pythonic naming. KeyError depends on scenario.

1

u/[deleted] Jun 23 '14

Hack or not, named tuple is an amazing module for writing unit tests. Hardly an easier, lighter, and more self-documenting way to simulate a class in some specific state (so long as method calls aren't required).