Reviewed by Aditya Kumar · Last reviewed 2026-03-24
**Why Explode:** Nested JSON (items array per order) → one row per item for joins/aggregation. Normalization for star schema. **Functions:** explode()—one-to-many; null/empty arrays drop row. explode_outer()—keeps row, null for empty. posexplode()—adds position index. For...
Why Explode: Nested JSON (items array per order) → one row per item for joins/aggregation. Normalization for star schema.
Functions: explode()—one-to-many; null/empty arrays drop row. explode_outer()—keeps row, null for empty. posexplode()—adds position index. For multiple array cols: arrays_zip then explode.
Scalability Risk: Large arrays cause row explosion—1 row with 10K items → 10K rows. Can cause skew. Use explode then repartition or broadcast the small side for joins. Monitor for OOM.
df.withColumn('item', explode('items'))
df.withColumn('item', explode_outer('items')) # keep nulls
Red Flag: Exploding without considering row multiplication. Pro-Move: 'We sample array lengths in prod—flag partitions with avg>100 for separate handling to avoid skew.'
According to DataEngPrep.tech, this is one of the most frequently asked Python/Coding interview questions, reported at 1 company. DataEngPrep.tech maintains an editor-reviewed database of 1,863 data engineering interview questions across 7 categories.