Implement a generator function to yield Fibonacci numbers.

A generator function for Fibonacci numbers leverages Python's yield keyword to produce numbers on demand, making it highly memory-efficient and suitable for potentially infinite sequences.

The core mechanic involves initializing two state variables, a and b, to 0 and 1. The while True loop ensures the generator can run indefinitely. Each iteration, yield a returns the current Fibonacci number to the caller without exiting the function, effectively pausing its execution. The function's local state (a and b) is preserved across these pauses. After yielding, a and b are updated to b and a+b respectively, preparing for the next number. This approach results in O(1) memory complexity because it only stores the two most recent Fibonacci numbers, unlike a list or array that would require O(N) memory to store N numbers.

This "lazy" evaluation is a fundamental benefit. Numbers are computed only when needed, which is crucial when dealing with very large or infinite sequences where pre-computing and storing all values would be impractical or impossible. In data engineering, this mirrors how systems like Spark process data partitions lazily (e.g., RDDs or DataFrames only compute transformations when an action is called) or how Kafka consumers process messages one by one from a stream, avoiding the need to load an entire dataset into memory.

def fib_generator():
    a, b = 0, 1
    while True:
        yield a
        a, b = b, a + b

To use it, you iterate over the generator. For example, for num in fib_generator(): if num > 100: break would print Fibonacci numbers until 100. For a specific number of elements, you can use next() repeatedly or itertools.islice. For instance, from itertools import islice; list(islice(fib_generator(), 10)) would yield the first 10 Fibonacci numbers. In production scenarios, you often "take" a specific number of elements from such an infinite stream, similar to how a data pipeline might process a fixed batch size from a continuous input.

In the interview, also mention:

The generator's O(1) memory footprint makes it ideal for handling unbounded data streams, a common pattern in real-time data processing and large-scale ETL, where memory is a critical resource.