Reviewed by Aditya Kumar · Last reviewed 2026-03-24
**Why iteration matters**: ML, graph algorithms are iterative; disk I/O kills performance. **MapReduce**: Each iteration reads/writes disk; no in-memory reuse. **Spark**: Keeps data in memory across iterations; RDD lineage enables lazy + reuse. **Scalability trade-offs**: Spark...
Why iteration matters: ML, graph algorithms are iterative; disk I/O kills performance. MapReduce: Each iteration reads/writes disk; no in-memory reuse. Spark: Keeps data in memory across iterations; RDD lineage enables lazy + reuse. Scalability trade-offs: Spark = 10–100x faster for iterative; MapReduce = simpler, lower memory. Cost implications: Spark = less total compute time for iterative; MapReduce = more I/O cost. Best practice: Use Spark for iterative; consider MLlib, GraphX for domain-specific.
Red Flag: Not knowing why Spark is faster for iteration. Pro-Move: 'Spark keeps partitions in memory; MapReduce spills each iteration—orders of magnitude difference.'
According to DataEngPrep.tech, this is one of the most frequently asked Spark/Big Data interview questions, reported at 1 company. DataEngPrep.tech maintains an editor-reviewed database of 1,863 data engineering interview questions across 7 categories.