An iterable in Python: anything that can be walked item by item

Definition

Writing a loop that walks a list is easy. Writing a different one for an open file, another for a dictionary, another still for a range of numbers would be tedious, and nobody really bothers. Python avoids that duplication through a shared convention: any object able to hand over its contents one item at a time fits the same loop. That ability, rather than a particular type, has a name: an iterable. Nothing forces an iterable to be a list, it only has to answer when asked for its items.

A string, a dictionary, an open file and a range of numbers are all iterable, even though they have almost nothing else in common, as the code below shows.

for letter in "abc":
    print(letter)

for line in open("notes.txt"):
    print(line)

for key in {"name": "Ada", "city": "London"}:
    print(key)

The for loop knows none of these objects in particular, and that is what makes it reusable. It applies a protocol: it asks the object for an iterator, then requests items one by one until an end signal arrives. Any object able to answer becomes iterable as a result, and the rest of the language then treats it like any other collection.

The source and the cursor

Mixing up iterable and iterator explains half the surprises on this topic. The iterable is the source: a list, a string, a file. The iterator is the cursor built from that source, the one that remembers how far it has gone. The iter() function moves from one to the other, and every call produces a fresh cursor, placed at the start.

The table below places the two side by side.

It hides a useful asymmetry: every iterator is also iterable, since it can supply itself, while the reverse does not hold. A list, on the other hand, keeps no memory between two loops, which allows it to be walked ten times in a row without any precaution.

What is not iterable

Some objects flatly refuse to be walked, and the resulting error message is recognisable among all others: it always points at an object being handled like a collection when it is not one.

total = 0

for digit in 12345:
    total += digit

# TypeError: 'int' object is not iterable

An integer, a float, a boolean, None and a function are not iterable: they have nothing to hand out. The resulting TypeError is rarely fixed where it appears. The most frequent case is a function returning None on a forgotten path, an if with no else, whose result travels into a loop: the error points at the loop, the cause sits twenty lines above.

The single walk trap

Tools producing an iterator rather than a collection are everywhere since Python 3, and they can only be walked once. The second walk raises no error at all, it simply returns nothing, far harder to notice than a crash, as the example below shows.

pairs = zip([1, 2, 3], "abc")

print(list(pairs))  # [(1, 'a'), (2, 'b'), (3, 'c')]
print(list(pairs))  # []

The cursor reached the end, left its end signal there, the StopIteration, and nothing rewinds it. Two answers exist: convert once and for all into a list when the data fits in memory, or rebuild the object before every walk otherwise. That trade-off between memory and recomputation is the only real choice the topic imposes.

Building your own

An object becomes iterable as soon as it can answer iter(). The short road goes through yield: a function handing over its values instead of returning them produces a walkable object without any class being written. The explicit road means writing a class that delegates the work to an attribute that is already iterable.

class Basket:
    def __init__(self, items):
        self.items = items

    def __iter__(self):
        return iter(self.items)

for item in Basket(["bread", "salt"]):
    print(item)

Three lines are enough, and the object gains at once everything resting on that protocol: the loop, the conversion into a list, the membership test, unpacking, computing functions such as sum or max.

Frequently asked questions

By trying it rather than by asking about its type: a call to iter(object) either succeeds or raises a type error. This is a direct application of duck typing, the principle that an object is worth what it can do rather than the family it descends from.

Because being iterable says nothing about size. len requires the object to know its number of items in advance, which a generator cannot by construction. Counting them means walking through all of them, and therefore consuming them.

A string is iterable at the character level, and that is a design choice of the language rather than an accident. To walk words, split the string first with split(), then walk the result.