Convert the list [1, [2, 3], 4, 5, 6, [7, 8, 9]] to a single list [1, 2, 3, 4, 5, 6, 7, 8, 9].

To flatten a nested list into a single, non-nested list, a common and elegant Pythonic approach involves recursion, often combined with a list comprehension and the sum() function. This method efficiently handles lists nested to arbitrary depths.

Mechanics and Why

The core idea is to iterate through the input list. For each element, check if it's a list itself using isinstance(item, list). If it is, recursively call the flattening function on that sub-list. If it's not a list, treat it as a single element by wrapping it in a new list [item]. The sum() function, when used with an empty list [] as its starting value, effectively concatenates all the resulting sub-lists (from recursive calls) and individual elements (wrapped in lists) into a single flat list. This leverages Python's list concatenation behavior.

Concrete Example and Key Trade-offs

For arbitrarily nested lists, a recursive solution is often the most concise and readable:

def flatten(nested_list):
    return sum([flatten(item) if isinstance(item, list) else [item] for item in nested_list], [])

While elegant, this recursive approach has trade-offs. Python has a default recursion depth limit (typically 1000), which can lead to a RecursionError for very deeply nested lists. For extremely large or deeply nested structures, an iterative approach (e.g., using a stack to manage pending items) or leveraging libraries like itertools.chain.from_iterable (for shallow nesting or when all sub-elements are iterable) might be more robust or memory-efficient. In data engineering, where memory and stack limits are critical for processing large datasets, these performance considerations are important.

In the interview, also mention…

Discuss how this problem relates to parsing complex JSON structures (e.g., using explode in Spark SQL or PySpark to flatten arrays of structs), denormalizing data, or handling semi-structured data in systems like Snowflake. Mention generator expressions for memory-efficient flattening without building intermediate lists, which is crucial for large datasets.