Reviewed by Aditya Kumar · Last reviewed 2026-03-25
**MapReduce**: Disk-based. Each job writes to HDFS between map and reduce. Multi-stage = multiple disk I/O. Latency: minutes to hours per stage. **Spark**: In-memory (when cached). DAG with lazy evaluation. RDD/DataFrame caching. 10–100x faster for iterative (ML, graph). **Why...
MapReduce: Disk-based. Each job writes to HDFS between map and reduce. Multi-stage = multiple disk I/O. Latency: minutes to hours per stage.
Spark: In-memory (when cached). DAG with lazy evaluation. RDD/DataFrame caching. 10–100x faster for iterative (ML, graph).
Why Spark Won: Iterative algorithms (e.g., PageRank, K-means) run 10–100x faster. Unified batch + streaming. DataFrame API + Catalyst.
Scalability Trade-offs: Spark needs more memory. MapReduce more fault-tolerant per stage (writes to disk). Spark handles failures via lineage.
Cost Implications: Spark = fewer cluster hours for same work. MapReduce for legacy or disk-bound workloads.
Pro-Move: 'We migrated 50 MR jobs to Spark; 80% runtime reduction, same data.' Red Flag: 'MapReduce is obsolete'—still used in some Hadoop-only shops.
According to DataEngPrep.tech, this is one of the most frequently asked Spark/Big Data interview questions, reported at 1 company. DataEngPrep.tech maintains an editor-reviewed database of 1,863 data engineering interview questions across 7 categories.