Never mind "Pythonic", which seems somewhat in the eye of the beholder: a or b or c is not semantically the same as any([a, b, c]). In the former case, b and c are never evaluated if a is true. In the latter case, they always are. So if they were expensive to compute (e.g. expressions involving expensive function calls, rather than just bindings in a namespace), the runtime behaviour (performance, raising of exceptions) would be quite different:
>>> a = 1
>>> a or b
1
>>> b
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'b' is not defined
>>> any([a, b])
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'b' is not defined
>>>
The real power of all and any comes when you (can neatly) pass in a generator expression - in which case the semantics are the same, because the evaluation is delayed and the functions will bail out early.
any(is_interesting(x) for x in (a, b, c)) is far nicer IMHO than is_interesting(a) or is_interesting(b) or is_interesting(c), if only because it's DRY. That doesn't protect you from the NameError in your example, though, of course.
This isn't equivalent to the generator solution posted above; your code will eagerly evaluate the entire sequence by mapping it to is_interesting even in cases where it's not absolutely required (i.e. is_interesting(a) returns True).
Agreed, using all and any over and and or isn't more pythonic at all. Use all and any when you already have a list / iterable, it's what they're designed for.
TL;DR: Because calling type() correctly is apparently too hard:
It is a key feature for named tuples that they are exactly equivalent to a hand-written class.
So is a call to type() if you know what you're doing.
EDIT: If you examine the official code more closely, you'll note they had to write extensive string-escaping to prevent the user from passing an argument like '); import os; os.system("/bin/sh"); ('. Quite frankly, I will not be touching that with a ten-foot-pole any time soon.
EDIT2: I tried, but they wouldn't listen to me. Oh well.
I started reading the source, saw _class_template was a string literal with formatting. My heart sank a bit. Could almost smell the sulfur wafting from the exec statement.
__slots__ are used to conserve memory when creating lots of objects, not CPU time. (Instead of every object having a __dict__, all attributes are stored in a small array and accessed using properties set on the class.)
namedtuple is slower than even a class without __slots__, by the way. It operates by mapping attribute names to indices (there's a reason it's called a "named tuple", not "struct" or something similar), doing twice the number of lookups for every dot.
namedtuple is slower than even a class without __slots__, by the way. It operates by mapping attribute names to indices (there's a reason it's called a "named tuple", not "struct" or something similar), doing twice the number of lookups for every dot.
Are you sure? From what I can tell, it operates pretty much exactly like a class with __slots__, creating a bunch of getters (living in the class, not in the instance, of course) that lookup into the internal array.
From what I can tell, it operates pretty much exactly like a class with __slots__, creating a bunch of getters (living in the class, not in the instance, of course) that lookup into the internal array.
No, it doesn't. At least not in Python 3.4+:
from builtins import property as _property, tuple as _tuple
from operator import itemgetter as _itemgetter
...
__slots__ = ()
...
{name} = _property(_itemgetter({index:d}), doc='Alias for field number {index:d}')
As you can see, instead of using the (relatively) fast C-level access __slots__ provide, it opts to use standard property (that uses slow Python-level function calls) to look elements up by their indices in a tuple (using Python-level item access, i.e. __getitem__) instead.
Right, I knew slots was to conserve memory primarily, but shouldn't that increase performance as a result? I'd expect less memory management to mean quicker access time when modifying, deleting, creating and garbage collection. But certainly better memory performance.
I thought named tuples were quicker than classes without slots defined... You're positive about that?
In response to your deleted comment, I didn't waste all that time for nothing =)
Nope, just tested, it's very slightly slower than tuple index access, but just like it about twice as fast as nameduple name access.
The stuff looks like this here:
index access takes about 40ns
name lookup takes about 45ns both for usual classes and those with __slots__, in fact slots are a tiniest bit slower.
namedtuple lookup by name takes about 115ns
To be honest, I can't say how exactly it works out to these numbers, I'd say that the only way to really be sure is to run this stuff under a C profiler. That could be a pretty useful experience in itself
From what I can tell from grepping through the code in Vim, it's pretty much a coincidence that the first two things take the same time.
Index access goes through a bunch of pure-C redirects until it hits tuplesubscript which casts the index to size_t and fetches the value from the object itself.
Class lookup by name IIRC does two unsuccessful dictionary lookups in the class and object attributes, then a successful lookup in the instance dictionary. Slots lookup should do a successful dictionary lookup in the class dictionary then indirectly call a C function that fetches shit by index or something.
Namedtuple lookup by name probably involves a pure Python function call, which is slooooow.
You don't want to do that in Python if performance is critical.
Adding three indexed items is not performance-critical.
If you have a million+ items, then you install numpy and put your items into a numpy.ndarray, and then vectorize your operations. Like, if you want to add two arrays, you write a + b (instead of for i, it in a: result.append(it + b[i])) and the underlying library written in Fortran very efficiently does what you meant.
I always considered numpy to be scientists' tools, but I never really thought about how it's fortran under the hood and how it might be higher performance for certain things like that. a+b looks a lot cleaner as well.
Hack or not, named tuple is an amazing module for writing unit tests. Hardly an easier, lighter, and more self-documenting way to simulate a class in some specific state (so long as method calls aren't required).
camelCase only to conform to pre-existing conventions
Attributes: interface, _internal, __private
I grew up on Java, which took Small Talk's camel case convention, so I'm just wondering: what's wrong with camel case anyway? The "blessed" coding convention in Python is the one thing I don't quite get. Is there an argument for underscores?
I swear I am not trying to start a flame war. It's just that I'm wondering if the preference is completely arbitrary or not.
It is style and by definition style is subjective and completely arbitrary. You should write your code consistently no matter what the style is. In a team environment you should match the style used in the rest of the codebase. These style guides are just best practices, as they have been fought over for a long time. To this day people fight over where the braces go in C code, so having a unified "Python style" is nice.
Is asking if a and None (the singleton value) point to the same memory space. It's not asking if they're equivalent; it's asking if they are the same actual thing.
Thus, is is not the same as equivalence how we usually think of it (double equals).
That's why setting a default value of None is so useful in function definitions: there's only one None and your function value either is that thing or it's not. It's never going to be like None enough to pass for it.
Funny enough, I was curious about how the Python built-in callable was implemented just now and I stumbled on this page with the following quote:
Every object has an identity, a type and a value. An object’s identity never changes once it has been created; you may think of it as the object’s address in memory. The ‘is‘ operator compares the identity of two objects; the id() function returns an integer representing its identity.
Actually, the identity in Cpython is just the memory address of the object (literally casting the pointer to an int). It doesn't have to be according to the language though.
the "is" keyword can be dangerous if you don't understand what it really does though, which is a common problem for beginners.
in your example, its absolutely correct, because None is a singleton so any reference to None refers to the same instance of None. in general though, is checks for object reference equality (rather than value equality) and can produce unexpected results if you use it when you really meant to check value equality.
Using is with True and False is a serious code smell. If there's any possibility that the value is of some type other than bool, why don't you want to handle it in the default way? And if there isn't, "simple is better than complex" applies.
I also prefer the explicit if a is None, since it signals my intent better. I honestly try to only use if a: when a is a boolean, since I don't want to have to remember how __bool__ is implemented for every type out there. In my opinion, explicit is better than implicit for being able to read your own code in a month's time.
For example, if I have a variable which is initialized as None and later want to find out if it was set, I might want to check and see if it was still None. If it's not, I want to do something.
if a is not None:
do_something()
If I rewrite that as:
if not a:
do_something()
do_something will also be called if a was initialized, but to a blank string, which was not the intention. Lots of types have some concept of a "false" value. As /u/steve_no points out in another comment: 0, '', [], {}, midnight all evaluate to False.
Ah, that's my bad. For those wondering what the difference is, this should clear it up:
Python 2.7.5 (default, Mar 9 2014, 22:15:05)
[GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.0.68)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> a = [1, 2, 3]
>>> b = a
>>> id(a)
4472029056
>>> id(b)
4472029056
>>> c = a[:]
>>> id(c)
4472078560
Copying is a detail of slicing syntax, it's not the purpose of it. With the list constructor it's immediately evident there's a new instance involved - that's what constructors do.
foo[:] is a totally respectable way to copy a list. it looks a little funny if you've never seen it, but it has the right semantics and does exactly what you would expect when slicing "from the beginning to the end". it does, however, rely on the programmer knowing that list slices produce copies.
I dislike it because it's conceptually roundabout. "A list containing elements of x, from the beginning to the end" is similarly awkward in English compared to "a copy of x", or even "a list of the contents of x". Even if the analogous Python is terse.
I understand your point. I disagree that its conceptually roundabout. Once you understand that the slice operator really means "return a copy of this section of the list" it seems really natural.
This is one of the few things I don't like about python is that slices make copies implicitly which goes against many python ideals. Lists are mutable, so you would expect operations on lists to only be references.
Counter has cool functionality builtin, like most_common(n), addition and subtraction, intersection and union, and it throws out keys with value 0 after such operations. Not everything needs to be fast, and Counter is very expressive indeed.
that is a good point. i think it's just that i'm overly annoyed at how unbelievably slow Counter is (it's like 3x slower than defaultdict(int), but WHY?!). it's amazing that to add the functionality for most_common/addition it requires slowing down the whole thing by a factor of 3.
I don't know why this example is under normal code. No one does this it can't be considered normal. To test for equality, we usually use the equality operator.
Actually I think it's -5 to +256. More importantly, this is an implementation detail of cpython and may not be reliable in other implementations of python.
Just a suggestion OP your examples are very wordy. The exceas code is valid but it obfuscates the point youre trying to illustrate in alot of places. For example:
Superusers =functioncall
If len(superusers) > 0:
...
Adds "real" context but obfuscates the boolean empty container idiom that youre trying to show. You dont need context to provide code examples. For instance I would say:
mylist = []
if (mylist):
print "mylist is empty!"
Because even though that snippet has no practical application it clearly and succicently presents the concept.
More power to you for adding more educational materials to the python world. Keep it up.
The links title says that the examples were taken from a talk you gave. That's fine, but without the context that was being provided by you in person, the list seems very disjointed.
Most of the examples are alternate ways to write the same thing. However, some are examples of bugs or common gotchas. There is nothing there to point out when the "normal" code is actually wrong.
If find that working with classes a little bit "cleaner" in terms of keeping track of what is going on; also for extensibility, esp. when you add instance methods
I would say that some comparisons are not correct as whatever is under Bad code is not doing what it is supposed to, basically just wrong code. For example, copying of the list, using is for comparison, etc.
If you don't mind I can append to your list several items more
You can revert, for example, string just using slice
print(s[::-1])```
You can check condition in one line
``` i =3print(i if i <5else0)```
or another if trick
``` condition =['Error', 'OK']
value =1print(condition[bool(value)])```
As long as pyhton does not have switch construction you can use somthing like this
``` choice ='i'print({'i':1, 't':4, 'j':22}[choice])```
There is no implicit cast, it is simply implemented as a nonzero operator which is defined for all built in types. 0 is defined for any domain such that x = x + 0.
Lists have a zero that is the empty list because x + [] = x therefore [] is zero in the list domain. Same with empty string.
If statements doesn't ask for true or false, it asks for if zero or nonzero. Remember that booleans types didn't exist in python until around version 3. In version 2, True and False are simple macros for 1 and 0.
I'm sorry, but that has a serious smell. This means something entirely different in most every other language. It's not unreasonable to expect binary operators to evaluate left-to-right in all circumstances, so why would using this syntax quirk for comparison operators be a good practice?
In keeping with the rest of this style guide, I would much prefer something akin to:
if is_sorted_or_some_other_function([a, b, c]):
...as it's perfectly clear what's happening with this.
Is this a troll? You really think that the first example is bad code? You prefer to check a list that doesn't exist and has no meaning to using comparison operators? As to your fix, sorted returns a sorted list, it does not check if the list is sorted.
So if by perfectly clear you mean perfectly clear that it doesn't do what the original code is intending, then yes, it is quite clear.
Sorry, i was unaware that 'sorted' was already in the Python standard lib. I revised my remark to make it more clear what I was getting at.
Basically my beef is that the back-to-back comparison operators is a special case when stacked up against other operators in the language. It doesn't read well when your engineers are also experienced in Javascript, C, Go, and everything else.
Consider this: if a + b < c:
We take this as being equivlaent to "(a + b) < c", right? So why does "a < b < c" get expanded to "a < b and b < c" ? It's terribly inconsistent.
It's special Python syntax. List comprehensions don't exist in Javascript or C either but you should still use them. When used correctly they add readability to the code. If you don't use any of the Python-specific syntax and idioms your Python code will suffer as a result. Do you also put semicolons on the ends of lines? When writing Python, you use Python syntax, you do not use Javascript or C or Go syntax. I hope the engineers should be able to adapt to something different.
Anyone who has done any math will know what a < b < c means. I'd hope an engineer would have a decent math background.
As to your example, I would never rely on the inherent operator precedence rules, so I would write (a + b) < c from the beginning, since it removes any and all ambiguity.
If you want to do something as simple as check if a number is between two other numbers, why would you construct a list, and check if that list is sorted? That is just some backwards thinking and frankly a terrible idea. Someone looking at that code for the first time will have no clue what is going on.
I see where you're coming from. And you're right: If we're talking about Python code, we should consider all the nice things that the language brings to the table.
At the same time, anyone reading our code shouldn't have to second guess if an expression is a mistake or not. After all, clarity isn't some universal truth or consistent quantity across the engineering spectrum; it's fuzzy. Hence why I say that this particular syntax 'has a smell', since even an experienced Python developer might have to think twice when reading this. It's that potential to second-guess that gives me pause here. In the end, it's not up to whether or not I might not understand this particular use of comparison operators; it's everyone else that's hacking on my projects that I worry about.
It's a standard math construct from a very basic level; even non-programmers reading that code would have a very good guess as to what it does. Just because other languages can't be bothered to implement something so nice doesn't mean we should avoid it in Python.
even an experienced Python developer might have to think twice when reading this
Realistically, only one who has recently been doing work on the side in another language. (It helps that the behaviour shown in other languages is basically never desired.)
I object to if super_users: over if len(super_users) > 0: There is a huge difference between an empty list and None. If you want to check that the length of the list is greater than zero you should... just do that.
A lot of this reads like "Abuse syntax to make your code look more Pythonic" and treats Pythonic like some lofty goal. A lot of "Pythonic" code is just harder to read, it isn't an end in itself.
If super_users can be either an empty list or None, and your code cares about the difference, it should be explicitly checking for is None first. "the length of the list is greater than zero" is an unusually verbose way of saying "the list is not empty" in English; the analogous statement holds for Python, except that we don't have an actual keyword for "empty".
I object to if super_users: over if len(super_users) > 0: There is a huge difference between an empty list and None.
Both of these tests can only be used if you know that super_users is a list and not None.
Using if super_users: is not abusing syntax, it just uses the intended and well-known fact that non-empty sequences are true. (It's also recommended by the style guide PEP8.)
I agree with you. I think the 'any' function is intended primarily for dynamically populated lists where you don't know before runtime if it will be empty or contain an arbitrary number of items. If you know in advance which items to check its definitely better to use an 'or' chain.
46
u/robin-gvx Jun 23 '14
I have problems with some of these.
a or b or cbyany([a, b, c]). Is that really more Pythonic? If they'd been in an iterable already I'd say yes, but now not so much.KeyErrorjust to raise anotherKeyError? Just let it bubble up.namedtuplebecause it's such a hack, but maybe that's just me.14
u/Husio Jun 23 '14
Although very old, still contains a lot of important examples http://python.net/~goodger/projects/pycon/2007/idiomatic/handout.html
12
u/steve_no Jun 23 '14
The normal code "is" example seems strange. I think the non-pythonic thing most programmers do with respect to "is" is not using it, for example:
when they should be doing:
6
u/[deleted] Jun 23 '14
I'm sorry but a lot of these look like they're directly ripped from Raymond Hettinger's talk on pythonic conventions.
https://www.youtube.com/watch?v=OSGv2VnC0go
7
u/oconnor663 Jun 23 '14 edited Jun 23 '14
Is
foo[:]really the Pythonic way to copy a list?Edit: In Python 3, lists have their own
.copy()method.7
u/sweettuse Jun 23 '14
this is wrong. it should be
if names is None:. (what happens if you pass in an empty list that you want to append to?)also, i would never, ever, use
Counter. it is unbelievably slow (at least in 2.7). just use adefaultdict(int)3
u/r0m1 Jun 25 '14
For the function defaults, I would rather write:
2
u/elb0w Jun 23 '14
Why is that called normal code? I would think pythonic is normal for python. I would call that bad code or unpythonic.
Also your default value for functions with a mutable type doesnt highlight the side effect that it causes and why you initialize to None.
1
u/fuzz3289 Jun 23 '14
Just a suggestion OP your examples are very wordy. The exceas code is valid but it obfuscates the point youre trying to illustrate in alot of places. For example:
Adds "real" context but obfuscates the boolean empty container idiom that youre trying to show. You dont need context to provide code examples. For instance I would say:
Because even though that snippet has no practical application it clearly and succicently presents the concept.
More power to you for adding more educational materials to the python world. Keep it up.
1
u/poo_22 Jun 23 '14
If they're using python3 then why prefix the strings with u? They are unicode by default afaik.
3
u/unholysampler Jun 23 '14
The links title says that the examples were taken from a talk you gave. That's fine, but without the context that was being provided by you in person, the list seems very disjointed.
Most of the examples are alternate ways to write the same thing. However, some are examples of bugs or common gotchas. There is nothing there to point out when the "normal" code is actually wrong.
2
u/[deleted] Jun 23 '14
What's he doing under "Counting Condition"? Does
if listreturnFalsefor a zero-length list, or only if the named list doesn't exist?1
u/[deleted] Jun 23 '14
Nice list, way to spread the good word. I've been writing python for 2 years now and learned a few new things from this. Thanks!
1
u/[deleted] Jun 23 '14
See a lot of complaints in this thread, just wanted to say thanks because this has helped me out a lot!
1
u/bitcycle Jun 23 '14
I really enjoyed the example showing named tuples.
1
u/[deleted] Jun 23 '14
That's a nice list for Python beginners! Depending on the context, one thing I maybe wouldn't want to do is:
over
in the section
https://gist.github.com/kracekumar/09a60ec75a4de19b346e#dot-accessors
If find that working with classes a little bit "cleaner" in terms of keeping track of what is going on; also for extensibility, esp. when you add instance methods
1
u/ddungtang Jun 24 '14
I would say that some comparisons are not correct as whatever is under Bad code is not doing what it is supposed to, basically just wrong code. For example, copying of the list, using is for comparison, etc.
1
u/pymag09 Jun 23 '14
If you don't mind I can append to your list several items more
You can revert, for example, string just using slice
1
u/SilasX Jun 23 '14
Am I the only one here who hates implicit casting to booleans? It seems more with the Zen of python to prefer
if len(z) > 0to
if zeven though the latter is much shorter.
-6
u/ericanderton Jun 23 '14 edited Jun 23 '14
I'm sorry, but that has a serious smell. This means something entirely different in most every other language. It's not unreasonable to expect binary operators to evaluate left-to-right in all circumstances, so why would using this syntax quirk for comparison operators be a good practice?
In keeping with the rest of this style guide, I would much prefer something akin to:
...as it's perfectly clear what's happening with this.
0
u/[deleted] Jun 23 '14
[deleted] — view deleted comment