Member-only story
An Introduction to Python Generator Functions
Breaking down the mechanics of generators and working through examples
Python’s generator functions provide a powerful mechanism for managing data and computation resources, but for those new to Python, they are not necessarily intuitive. In this piece, I’ll break down the mechanics of generators, while also introducing what I hope is a motivating example: a small class for managing and streaming an S3 file resource.
Intro and History
Given how easy it is to get started with Python and write code that actually does something (e.g., iterate over a list of values, computing and/or printing those values), it may not dawn on the new or casual Python programmer that the language builds in the notion of procrastination, or deferred computation. This looseness — or laziness — that is built into the language itself may seem foreign to someone coming from a background of working with a compiled language (C++ for example).
Most programmers learn about “lazy evaluation” and are taught to write code to implement this practice. But Python’s native support of this in the language (achieved simply and elegantly through a single keyword) introduces power and expressivity that seems rare in other programming languages. It is not surprising that lazy…
