Reviewed by Aditya Kumar · Last reviewed 2026-03-25
**Format**: Parquet for data files + JSON transaction log in _delta_log. **Parquet Benefits**: Columnar; compression (snappy, zstd); predicate pushdown; schema in footer. **Delta Additions**: Transaction log for ACID; versioning; MERGE/UPDATE/DELETE; compaction...
Format: Parquet for data files + JSON transaction log in _delta_log.
Parquet Benefits: Columnar; compression (snappy, zstd); predicate pushdown; schema in footer.
Delta Additions: Transaction log for ACID; versioning; MERGE/UPDATE/DELETE; compaction (checkpoint.parquet).
Why Both: Parquet gives storage efficiency; log gives transactional semantics. Best of both.
Scalability Trade-offs: Log grows; checkpoint files aggregate. VACUUM and OPTIMIZE manage.
Cost Implications: Parquet compression ~5–10x. Delta log ~1–5% overhead. Combined = efficient storage + ACID.
Pro-Move: 'Zstd compression; 30% smaller than snappy, same read speed.' Red Flag: Deleting checkpoint files—slows log replay.
According to DataEngPrep.tech, this is one of the most frequently asked Spark/Big Data interview questions, reported at 1 company. DataEngPrep.tech maintains an editor-reviewed database of 1,863 data engineering interview questions across 7 categories.