MSK Institute • Shikohabad
Loading Page Content...
Preparing courses, live labs, and learning resources.सामग्री लोड हो रही है, कृपया प्रतीक्षा करें...
Connecting serverPlease wait...
💻 Practical Labs🚀 Free Demo📜 Verified Certificates
Preparing courses, live labs, and learning resources.सामग्री लोड हो रही है, कृपया प्रतीक्षा करें...
Generators are more than simple lazy iterators—they are stateful execution frames capable of suspension, resumption, bidirectional communication, and clean lifecycle management. Under the hood, Python generators pause their execution context, retaining local variables, instruction pointers, and exception states on the CPython heap.
Understanding the internal state machine of generators unlocks advanced data streaming architectures, pipeline composition, and cooperative concurrency patterns.
At any given moment during runtime, a Python generator resides in one of four distinct states defined in the inspect module:
| State | Constant | Description |
|---|---|---|
| Created | GEN_CREATED | Waiting to start execution; no next() or send() has been called yet. |
| Running | GEN_RUNNING | Currently being executed by the interpreter. |
| Suspended | GEN_SUSPENDED | Paused at a yield expression; frame preserved in memory. |
| Closed | GEN_CLOSED | Execution finished (via return, unhandled exception, or .close()). |
When a regular function finishes, its stack frame is deallocated. In contrast, when a generator yields, CPython marks the frame as suspended and preserves its local variable array and instruction pointer (f_lasti) on the heap:
.close() and GeneratorExitWhen a generator is closed explicitly via gen.close(), or when it is garbage collected, CPython raises a GeneratorExit exception at the current yield point.
GeneratorExit, cleanup logic in finally blocks must execute cleanly. The generator must not yield any additional values; attempting to yield during GeneratorExit triggers a fatal RuntimeError: generator ignored GeneratorExit.The most scalable pattern for processing unbounded data streams is the Pipeline Pattern. Each stage is an independent generator that accepts an input iterable and yields transformed items:
| Lifecycle Event | Action | Next State |
|---|---|---|
gen = my_func() | Generator instantiation | GEN_CREATED |
next(gen) / gen.send(None) | Advances to next yield | GEN_SUSPENDED |
gen.close() | Raises GeneratorExit inside frame | GEN_CLOSED |
gen.throw(exc) | Injects exception at suspension point | Handled: GEN_SUSPENDED / Unhandled: GEN_CLOSED |
Reaching return / end | Raises StopIteration | GEN_CLOSED |
next() on an Active GeneratorCalling next() or .send() on a generator from inside itself (or concurrently from multiple threads) raises a ValueError: generator already executing.
Generators do not support indexing (gen[0]) or slicing (gen[:5]). Use itertools.islice to slice generator streams lazily.
Use the inspect module to inspect a generator's lifecycle states: GEN_CREATED, GEN_SUSPENDED, and GEN_CLOSED.
What is the state of a freshly instantiated generator object before next() or send() has been invoked on it? A. GEN_RUNNING B. GEN_SUSPENDED C. GEN_CREATED D. GEN_PENDING
GEN_CREATED state until execution is primed by calling next() or send(None).What exception is raised inside a suspended generator when gen.close() is called? A. StopIteration B. GeneratorExit C. KeyboardInterrupt D. SystemExit
GeneratorExit exception at the suspension point to allow the generator's finally or except GeneratorExit blocks to execute resource cleanup.What occurs if a generator catches GeneratorExit and attempts to yield another value with yield? A. The value is successfully emitted to the caller. B. Python ignores the yield and terminates silently. C. Python raises a RuntimeError: generator ignored GeneratorExit. D. A StopIteration exception is raised.
GeneratorExit. Violating this triggers a RuntimeError: generator ignored GeneratorExit.Where does CPython store the local variables and execution instruction pointer of a suspended generator? A. In CPU register cache $L1$. B. In an active heap-allocated frame object (gi_frame). C. On the operating system thread execution stack. D. In a temporary file on disk.
gi_frame), allowing them to resume execution safely across function calls.What is the primary operational characteristic of a generator pipeline where multiple generator functions are chained together? A. Each stage runs on a separate CPU core via multiprocessing. B. All data is eagerly transformed into memory before the next stage runs. C. Data flows lazily item-by-item through each stage on demand when the final consumer calls next(). D. The entire pipeline is converted to SQL queries.
In the next lesson, we will continue your advanced Python journey with Yield from Expression (5: Generators & Coroutines).
Continue learning with hands-on practice, examples, and exercises in the upcoming topic.
| Previous Lesson | Next Lesson |
|---|---|
| Project: Data Pipeline with Itertools | Yield from Expression |
Test your understanding of this lesson with 5 questions. Each question has one correct answer.