Reviewed by Aditya Kumar · Last reviewed 2026-03-25
**#1 Bottleneck: Shuffle**. Network I/O for join, groupBy. Resolve: (1) Broadcast small tables. (2) Column pruning. (3) Partition by key. (4) Fix skew (salting). (5) Tune shuffle partitions. (6) AQE coalesce. **Other Bottlenecks**: GC (reduce executor memory, G1), data skew,...
#1 Bottleneck: Shuffle. Network I/O for join, groupBy. Resolve: (1) Broadcast small tables. (2) Column pruning. (3) Partition by key. (4) Fix skew (salting). (5) Tune shuffle partitions. (6) AQE coalesce.
Other Bottlenecks: GC (reduce executor memory, G1), data skew, too few executors, source slow.
Why Shuffle Dominates: Moves data across network; serialization; often 70–90% of time.
Scalability Trade-offs: Each fix has limits. Combine. Profile first.
Cost Implications: Shuffle fix = 50–80% runtime reduction. First place to look.
Pro-Move: 'Spark UI → Stages → Shuffle read/write; we target <input size.' Red Flag: Adding executors when shuffle is bottleneck—doesn't help.
According to DataEngPrep.tech, this is one of the most frequently asked Spark/Big Data interview questions, reported at 1 company. DataEngPrep.tech maintains an editor-reviewed database of 1,863 data engineering interview questions across 7 categories.