Definition
Making a list out of another one is among the most frequent moves in Python, and among the most long winded: it takes an empty list, a loop that walks the source, an append on every pass. Three lines of plumbing for a single idea.
A list comprehension removes that plumbing. It builds a list in a single expression, out of an existing iterable, by stating what the resulting list holds rather than the moves that fill it.
# The loop describes the moves, in three steps
squares = []
for n in range(10):
squares.append(n * n)
# The comprehension states the result, in one expression
squares = [n * n for n in range(10)]
# [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]The second form reads almost as it stands: for every n taken from range(10), put n * n in the list. No half filled list living through the walk, no append call that can be forgotten inside a branch.
The result is a complete list, built in memory the moment the line runs. A comprehension therefore puts off no computation at all, and that is what separates it from the generator it otherwise resembles.
The full shape
A comprehension breaks down into three pieces, only one of them optional. The table takes them in the order you write them, which never changes.
| Piece | Role | Example |
|---|---|---|
| Expression | What lands in the list | n * n |
| Walk | The source of the values | for n in numbers |
| Filter | Rules out what must not enter | if n % 2 == 0 |
The expression is the only place where a value changes shape, and the walk accepts any iterable: a string, the keys of a dictionary, a file read line by line.
The filter, for its part, rules items out: the resulting list becomes shorter than the source, sometimes empty. You can no longer match its positions against those of the original.
The two if forms not to confuse
The word if can show up in two places, and it does a different job in each. Nothing in the word tells you which of the two you are writing, hence the most widespread mistake on the subject.
# The if after the walk filters: the list shrinks
evens = [n for n in numbers if n % 2 == 0]
# The if before the walk chooses: the list keeps its length
labels = ["even" if n % 2 == 0 else "odd" for n in numbers]In the first line, the if comes after the walk: it is a filter. It accepts no else, since a dropped item has no stand-in to offer. Out of ten numbers, four of them even, it holds four.
In the second, the if belongs to the expression, before the for: it is a ternary choice, which must produce a value and therefore requires its else. Nothing is dropped: out of ten numbers, it holds ten.
The position alone settles it: after the walk you sort, before the walk you transform.
An if without an else placed before the for filters nothing: it raises a SyntaxError. The line [n if n % 2 == 0 for n in numbers] never even starts, and the message will not say the filter is in the wrong place.
The neighbouring comprehensions
This grammar does not serve lists only: by changing the punctuation around it, it builds three other objects.
| Writing | What it produces |
|---|---|
[x for x in source] | A list, in the order of the walk |
{x for x in source} | A set, therefore without duplicates |
{x: compute(x) for x in source} | A dict, key and value split by a colon |
(x for x in source) | A generator, computing as it is read |
That last line is no decorative variant. On a log file of several million lines, the bracket version loads everything into memory before handing control back, while the parenthesis version delivers the values one at a time. Moving from sum([...]) to sum(...) gives the same total without the memory.
When the generator is the only argument of a function, its parentheses merge with those of the call: sum(n * n for n in numbers) is enough, no need to double them.
When a loop is still better
A comprehension draws its value from being readable in one glance. As soon as it stacks two walks and a filter, that quality is gone, and with it the only good reason to write it.
The second guardrail concerns side effects. A comprehension exists to produce a value, never to act.
# Never write this: the list being built serves no purpose
[save(row) for row in rows]
# The loop says exactly what it does
for row in rows:
save(row)The first version builds a list of None as long as the file, since save returns nothing, then throws it away. It costs memory for nothing and lies about the intent.
The same reasoning settles the contest between a comprehension and map, which demands a function to call, often anonymous and written for the occasion. The comprehension shows the transformation in plain sight.
Frequently asked questions
Is a comprehension faster than a loop?
Slightly, yes. Building the list is handed to a dedicated interpreter instruction, with no append call on every pass, which commonly saves about a third of the time. That is far too little to settle the matter: readability comes first.
Can several walks be nested in one comprehension?
Yes, and they read left to right, in the order they would take as nested loops: the first for is the outer one. Past two levels, the classic loop becomes the more honest form again.
Why does the walking variable no longer exist after the line?
Because a comprehension owns its scope since Python 3: the variable is born and dies inside the brackets, so an n defined above will never be overwritten. An ordinary for loop, by contrast, leaves its own one reachable after the block, one of the differences worked through in the Python course.