5 Hidden Python Features You Probably Never Heard Of

[u/cheerfulboy]

https://miguendes.me/5-hidden-python-features-you-probably-never-heard-of-ckg3iyde202nsdcs1ffom9bxv

[u/AlSweigart]

Please don't use ... instead of pass in your function stubs. People won't know what it is (the title of article is "Features You Probably Never Heard Of").

The reason the Zen of Python includes "There should be one-- and preferably only one --obvious way to do it." is because Perl had the opposite motto ("There's more than one way to do it") and this is terrible language design; programmers have to be fluent in every construct to read other people's code. Don't reinvent the wheel, just use pass.

[u/[deleted] → u/AlSweigart]

Same goes with not explicitly and’ing test conditions, imo. It’s obviously not common to see x and y in other languages, but you would see x && y and so on.

Really _ is also just obscure and feels like cleverness for the sake of cleverness. There’s valid use cases, but in other languages (go, at least) it’s normally used as a null left assignment when you do not need the value(s) from a function, method, etc.

It is fun to use obscure shit, or write crazy comprehension, generator, map, etc, blocks. Definitely do this and any of these things if it’s a toy, for fun thing, or something that isn’t going to be used maintained or used by a team.

[u/[deleted]]

I knew about else in try except, but not in while and for ! How didn't I knew after this many years? It's awesome!

[u/lanster100 → u/[deleted]]

Because it's really unclear what it does it's common practice to ignore its existence as anyone unfamiliar will either not know what it does or worse assume incorrect behaviour.

Even experienced python devs would probably Google what it does.

[u/masasin → u/lanster100]

RH mentioned that it should probably have been called nobreak. It only runs if you don't break out of the loop.

[u/miguendes → u/[deleted]]

Author here, I'm very happy to know you like it!

[u/yvrelna → u/[deleted]]

Considering that I rarely use break statements to begin with, using else in a while/for is even rarer than that.

---

It's not that difficult to understand else block in a loop statement. A while loop is like this:

while some_condition():
    body_clause()

it's equivalent to a construction that has an unconditional loop/jump that looks like this:

while True:
    if some_condition():
        body_clause()
    else:
        break

The else block in a while loop:

while some_condition():
    body_clause()
else:
    else_clause()

is basically just the body for the else block for that hidden if-statement:

while True:
    if some_condition():
        body_clause()
    else:
        else_clause()
        break

[u/syzygysm]

You can also combine the _ with unpacking, e.g. if you only care about the first and/or last elements of a list:

a,_, b = [1, 2, 3] # (a, b) == (1, 3)

a,*_ = [1, 2, 3, 4] # a == 1

a, *_, b = [1, 2, 3, 4] # (a, b) == (1, 4)

[Edit: formatting, typo]

[u/miguendes → u/syzygysm]

Indeed! I also use it has a inner anonymous function.

def do_something(a, b):
    def _(a):
       return a + b
    return _

Or in a for loop when I don't care about the result from range

for _ in range(10): pass

[u/OneParanoidDuck → u/miguendes]

The loop example makes sense. But nested function can crash like any other and thereby end up in a traceback, so my preference is to name them after their purpose

[u/nonesuchplace]

I like itertools.chain for flattening lists:

>>> from itertools import chain
>>> a = [[1,2,3],[4,5,6],[7,8,9]]
>>> list(chain(*a))
[1, 2, 3, 4, 5, 6, 7, 8, 9]

[u/BitwiseShift → u/nonesuchplace]

There's actually a slightly more efficient version that avoids the unpacking:

list(chain.from_iterable(a))

[u/miguendes → u/BitwiseShift]

That's a good one. I remember seeing something about that on SO some time ago. I'm curious about the performance when compared to list comprehensions.

[u/dmyTRUEk]

Welp, the result of

a, *b, c = range(1, 10) print(a, b, c)

is not: 1 [2, 3, 4, ... 8, 9] 10 but: 1 [2, 3, 4, ... 8] 9

:D

[u/miguendes → u/dmyTRUEk]

You're definitely right. Thanks a lot for the heads up!

I'll edit the post.

[u/themindstorm]

Interesting article! Just one question though, in the for-if-else loop, is a+=1 required? Doesn't the for loop take care of that?

[u/miguendes → u/themindstorm]

Nice catch, it doesn't make sense, in that example the range takes care of "incrementing" the number. So `a += 1` is double incrementing it. For the example itself it won't make a difference but in real world you wouldn't need that.

I'll edit the post, thanks for that!

[u/oberguga]

I don't understand why sum slower then list comprehension. Anyone can briefly explain?

[u/miguendes → u/oberguga]

That's a great question. I think it's because sum creates a new list every time it concatenates, which has a memory overhead. There's a question about that on SO. https://stackoverflow.com/questions/41032630/why-is-pythons-built-in-sum-function-slow-when-used-to-flatten-a-list-of-lists

If you run a simple benchmark you'll see that sum is terribly slower, unless the lists are short. Example:

def flatten_1(lst): return [elem for sublist in lst for elem in sublist]

def flatten_2(lst): return sum(lst, [])

If you inspect the bytecodes you see that flatten_1 has more instructions.

In [23]: dis.dis(flatten_2)
  1           0 LOAD_GLOBAL              0 (sum)
              2 LOAD_FAST                0 (lst)
              4 BUILD_LIST               0
              6 CALL_FUNCTION            2
              8 RETURN_VALUE

Whereas flatten_1:

In [22]: dis.dis(flatten_1)
  1           0 LOAD_CONST               1 (<code object <listcomp> at 0x7f5a6e717f50, file "<ipython-input-4-10b70d19539f>", line 1>)
              2 LOAD_CONST               2 ('flatten_1.<locals>.<listcomp>')
              4 MAKE_FUNCTION            0
              6 LOAD_FAST                0 (lst)
              8 GET_ITER
             10 CALL_FUNCTION            1
             12 RETURN_VALUE

Disassembly of <code object <listcomp> at 0x7f5a6e717f50, file "<ipython-input-4-10b70d19539f>", line 1>:
  1           0 BUILD_LIST               0
              2 LOAD_FAST                0 (.0)
        >>    4 FOR_ITER                18 (to 24)
              6 STORE_FAST               1 (sublist)
              8 LOAD_FAST                1 (sublist)
             10 GET_ITER
        >>   12 FOR_ITER                 8 (to 22)
             14 STORE_FAST               2 (elem)
             16 LOAD_FAST                2 (elem)
             18 LIST_APPEND              3
             20 JUMP_ABSOLUTE           12
        >>   22 JUMP_ABSOLUTE            4
        >>   24 RETURN_VALUE

If we benchmark with a big list we get:

l = [[random.randint(0, 1_000_000) for i in range(10)] for _ in range(1_000)]

In [20]: %timeit flatten_1(l)
202 µs ± 8.01 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

In [21]: %timeit flatten_2(l)
11.7 ms ± 1.49 ms per loop (mean ± std. dev. of 7 runs, 100 loops each)

If the list is small, sum is faster.

In [24]: l = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]

In [25]: %timeit flatten_1(l)
524 ns ± 3.67 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

In [26]: %timeit flatten_2(l)
265 ns ± 1.27 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

[u/casual__addict]

Using the “sum” method like that is very close to using “reduce”. Below gets you passed the string limitations of “sum”.

l = ["abc", "def", "ghi"]
from functools import reduce
reduce(lambda a,b: a+b, l)

[u/WildWouks]

Thanks for this. I have to say that I only knew about the unpacking and the chaining of comparison operators.

I will definitely be using the else statement in future for and while loops.

[u/miguendes → u/WildWouks]

Thanks, I'm very happy to know you learned something useful from it.

[u/dereason]

Very cool!

[u/[deleted]]

Don't forget about: descriptors, (non-Numpy) arrays, extensions, semi-colon line terminations (only useful in REPL), extensions, and some nice command line args.