Reviewed by Aditya Kumar · Last reviewed 2026-03-24
**Sources**: Spark UI (tasks, stages, shuffle); executor logs (memory, GC, CPU); driver logs; iostat, top. **Signals**: Skew (one task 10x slower); memory (spill, OOM); network (shuffle); stragglers. Tools: Spark UI, Prometheus....
Sources: Spark UI (tasks, stages, shuffle); executor logs (memory, GC, CPU); driver logs; iostat, top. Signals: Skew (one task 10x slower); memory (spill, OOM); network (shuffle); stragglers. Tools: Spark UI, Prometheus. Add custom metrics; correlate with logic; profile (JVM, Python); document fixes in runbook.
Pro-Move: 'Spark UI showed 1 task 45 min, others 2 min—skew. Salted join key; all tasks ~3 min. Added skew alert.'
According to DataEngPrep.tech, this is one of the most frequently asked General/Other interview questions, reported at 1 company. DataEngPrep.tech maintains an editor-reviewed database of 1,863 data engineering interview questions across 7 categories.