The Iterator Protocol: the secret to Python's for loops

Hiya,

Last week I shared some information with you on some of the more common looping constructs in Python. Let's dive deeper into how "for" loops work under the hood.

Python's "for" loops work differently than many other programming languages.

The rules that define "for" loops are described by something called the iterator protocol. For loops are powered by iterators and it's actually possible to implement a "for" loop using a "while" loop instead.

This "while" loop:

def print_each(iterable):
    iterator = iter(iterable)
    while True:
        try:
            item = next(iterator)
        except StopIteration:
            break  # Iterator exhausted: stop the loop
        else:
            print(item)

Is equivalent to this "for" loop:

def print_each(iterable):
    for item in iterable:
        print(item)

But why does this matter?

There are two primary reasons to learn about iterators in Python:

1. If you don't understand that Python's iterables are iterator-powered (and in fact iterators themselves are iterables) you may occasionally find yourself bit by an iterator-related Python oddity. I cover many iterator-related Python oddities in my article Loop better: a deeper look at iteration in Python (as well as a talk with the same title).

2. Thanks to iterators, it's possible to make iterables that generate data on-the-fly in Python. These are called generators and they're pretty neat! You can use generators to invent your own lazy looping helpers in Python, similar to the built-in enumerate and zip functions.

For more on iterators and generators, see my generator expression and generator function screencast series.

To get hands-on practice with iterators and generators, try the Intro to Generators and Iterators exercise path (included with a Python Morsels All Access subscription).

Happy lazy looping!