Reviewed by Aditya Kumar · Last reviewed 2026-03-24
Beyond repartition/cache: (1) Broadcast small tables for joins. (2) Predicate pushdown—filter early. (3) Projection pruning—select only needed columns. (4) Partition pruning for partitioned reads. (5) Coalesce before writes to reduce files. (6) AQE (Adaptive Query Execution)....
Beyond repartition/cache: (1) Broadcast small tables for joins. (2) Predicate pushdown—filter early. (3) Projection pruning—select only needed columns. (4) Partition pruning for partitioned reads. (5) Coalesce before writes to reduce files. (6) AQE (Adaptive Query Execution). (7) Salting for skewed joins. (8) Avoid shuffle when possible. (9) Use bucketing. (10) Tune executor memory, parallelism. Why it matters: Design choices compound at scale—wrong approach can cause 100× overhead. Scalability trade-offs: Profile before optimizing; validate on sample then full. Cost implications: Suboptimal choices multiply at billion-row scale.
Red Flag: Generic textbook answers. Pro-Move: 'At scale we measured X, implemented Y, achieved Z%—validated and iterated.'
According to DataEngPrep.tech, this is one of the most frequently asked SQL interview questions, reported at 1 company. DataEngPrep.tech maintains an editor-reviewed database of 1,863 data engineering interview questions across 7 categories.