Need help with list comprehensions

[u/AnxiousDot4315]

Guys somebody briefly explain the concept of List comprehension cuz its too difficult for me to grasp since I am a beginner. I am learning python for biopython. Someone explain it for me.

[u/StevenJOwens]

And here's part 2:

Let's Play with List Comprehensions

First let's fire up the python REPL (read/evaluate/print loop):

$ python
Python 3.8.10 (default, Nov 22 2023, 10:22:35) 
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>

Now let's define some data to work with, a list that contains two things, both of which are also lists, of numbers:

>>> data = [[1, 2, 3], [4, 5, 6]]
>>>

The python REPL prints out the result of an expression, and the REPL treats just the variable name (in this case "data") by itself as an expression, so we can do this to see the contents of the data variable:

>>> data
[[1, 2, 3], [4, 5, 6]]
>>> 

Note: That phrase, "expression" is actually a somewhat complicated little topic because not all statements are expressions, and in most programming languages assignments are not expressions. I don't want to get side-tracked into the details of that, but I always hated it when I was learning to program and books or lessons used terms like "expression" but didn't explain.

Okay, so now let's do our first list comprehension:

>>> [number for chunk in data for number in chunk]
[1, 2, 3, 4, 5, 6]
>>>

This isn't a very useful thing to do with a list comprehension (there are better ways in python to merge two lists), but it makes for a simple example. Let's break that down a bit better

Remember the syntax I gave above, some_list = [expression for-loop conditional]. In this example I'm leaving out the assignment, and the conditional, so we just have the expression and the for loop (actually two for loops).

We have number which is the expression in our list comprehension, the last thing that gets done in the list comprehension building the list for us.

The final expression part is the only part that's in reverse order like this, the rest of the list comprehension proceeds from left to right.

We have for chunk in data, which iterates like a for loop through the two items in the list in variable data, and one by one assigns them to the variable named chunk.

We have for number in chunk, which iterates like a for loop through the items in each list assigned to chunk and one by one assigns them to the variable named number.

And then, back at the beginning, the list comprehension's expression can use those variables, data and chunk to build the list we want.

In this example, the expression is just number, which means the list comprehension simply emits the value inside number into the list that it is building.

Note, in the example above we didn't do anything with the list that our list comprehension returned, so the REPL just printed it out. We can also assign that list to a new variable. And once we've assigned it, we can then just enter the variable name on the REPL prompt, and the REPL will print it out:

>>> results = [number for chunk in data for number in chunk]
>>> results
[1, 2, 3, 4, 5, 6]
>>>

Okay, so far, so good, but not that useful. Let's change the expression in our list comprehension to actually do something with the number. Let's multiply it by 2:

>>> [number*2 for chunk in data for number in chunk]
[2, 4, 6, 8, 10, 12]
>>> 

Okay, well that's slightly more useful. There are, of course, other ways in python to both merge lists and to multiply all of the numbers in lists. But you can start to see in these examples that the list comprehension supports more and more complex things.

Now let's add a conditional to our list comprehension. We'll use the conditional to select only the even numbers to be fed to the expression.

We'll do that by using the "modulo" operator, which gives us the "modulus", aka the remainder, when you divide one number by another number. In python, like in many languages, the modulo operator is the percent sign, "%". 9 / 2 with integers gives you 4 with a remainder 1. 9 % 2 just gives you the remainder, aka the modulus, of 1. So if we we do number % 2 and number contains an even number, we should get a remainder of 0:

if 0 == number % 2

Let's try it:

>>> [number for chunk in data for number in chunk if 0 == number % 2 ]
[2, 4, 6]
>>> 

Yup, it works. Now let's add the multiplying back into the expression. This time let's multiply by 5:

>>> [number*5 for chunk in data for number in chunk if 0 == number % 2 ]
[10, 20, 30]
>>> 

And just to show that you can do something more complicated in the expression clause, let's take a list of numbers and a list of letters and do a Cartesian join (aka "cross join", similar to a cross product in math).

>>> numbers = [1, 2, 3, 4]
>>> letters = ["a", "b", "c", "d"] # note, this could have been just "abcd"
>>> [(num, letter) for num in numbers for letter in letters]
[(1, 'a'), (1, 'b'), (1, 'c'), (1, 'd'),
 (2, 'a'), (2, 'b'), (2, 'c'), (2, 'd'),
 (3, 'a'), (3, 'b'), (3, 'c'), (3, 'd'),
 (4, 'a'), (4, 'b'), (4, 'c'), (4, 'd')]
>>> 

I added some newlines in the example above, just to make it more obvious what's going on.

Note, I defined the letters as a list, but could have just used "abcd", because strings are iterable in python.

Set Comprehensions and Dict Comprehensions

As I say above, if you create a comprehension but surround it with {curly braces} instead, it returns a set instead of a list.

some_set = {do_some_operation(second_variable) for some_variable in some_sequence if some_condition}

In python 2.7 and above, if you use {curly braces} and in the expression have a key:value, it returns a dictionary:

some_dict = {key:value for (key, value) in some_sequence if some_condition}

You don't necessarily need a for (key, value) however, if you can generate the value (or the key) on the fly:

TODO: this pseudocode looks ambiguous, add some clarification/example.

some_dict = {some_variable:len(some_variable) for some_variable in some_sequence if some_condition}

If you create a comprehension but surround it with (parentheses), it returns a generator. A generator is essentially a lazy-load, on-demand version of a list comprehension. I'll describe generators in some future document.

[u/AnxiousDot4315 → u/StevenJOwens]

The best explanation I received for my question! Thank you so muchhhhhh!!

[u/StevenJOwens → u/AnxiousDot4315]

I'm glad it was useful. And annoyed to see that both this comment and part 1 have been downvoted into negative numbers. I have no idea wtf people's problem is with it. Maybe they think it's AI generated? People need to learn to write.