Explain the differences between multiprocessing and multithreading.

Multithreading involves multiple threads executing within a single process, sharing the same memory space. Multiprocessing, conversely, uses multiple independent processes, each with its own dedicated memory space. In Python, the Global Interpreter Lock (GIL) significantly impacts this distinction, limiting multithreading to concurrency rather than true parallelism for CPU-bound tasks.

Mechanics and the GIL

Multithreading creates lightweight threads that share the parent process's memory, making data access straightforward but requiring careful synchronization to prevent race conditions. Python's GIL is a mutex that ensures only one thread can execute Python bytecode at a time, even on multi-core systems. This means multithreading in Python is primarily beneficial for I/O-bound tasks. When a thread performs an I/O operation (like reading from a network or disk), the GIL can be released, allowing another thread to run concurrently while the first waits.

Multiprocessing creates separate, heavier processes, each with its own Python interpreter, memory space, and GIL. This isolation allows for true CPU parallelism across multiple cores, as each process can execute Python bytecode independently. Data sharing between processes requires explicit Inter-Process Communication (IPC) mechanisms like queues or pipes, which are more complex than shared memory but inherently safer from race conditions.

Key Trade-offs and Usage

For CPU-bound tasks (e.g., complex data transformations, heavy computations), multiprocessing is essential in Python to leverage multiple CPU cores. Libraries like concurrent.futures.ProcessPoolExecutor are ideal for this. For I/O-bound tasks (e.g., fetching data from many APIs, downloading files), concurrent.futures.ThreadPoolExecutor provides effective concurrency by allowing threads to yield the GIL during I/O waits.

import concurrent.futures
import time
def cpu_intensive_task(n):
    return sum(i*i for i in range(n))
# Use ProcessPoolExecutor for CPU-bound tasks to achieve true parallelism
with concurrent.futures.ProcessPoolExecutor(max_workers=4) as executor:
    results = list(executor.map(cpu_intensive_task, [10*7]4))

Multiprocessing incurs higher startup overhead and memory footprint compared to multithreading. Data engineering tools like Apache Spark leverage distributed processing, which is an extension of multiprocessing, where executors run as separate processes (or JVMs) on different nodes, enabling parallel computation on data partitions.

In the interview, also mention that while the GIL is a Python-specific constraint, the fundamental concepts of shared vs. isolated memory and concurrency vs. parallelism are universal in concurrent programming.