Spark & Big Data questions from Citi data engineering interviews.
These spark & big data questions are sourced from Citi data engineering interviews. Each includes an expert-level answer. This set leans toward senior-level depth (10 of 16 are tagged hard). Recurring themes are partition, spark, and optimization — these patterns appear most often in real interviews and reward the deepest preparation. Many of these questions also surface at BCG and Dunnhumby, so the preparation transfers across companies. Average answer is around 1 minute of reading — plan roughly 1 hour to work through the full set thoughtfully.
This collection contains 16 curated questions: 4 easy, 2 medium, and 10 hard. The distribution skews toward harder problems, reflecting the depth expected in senior-level interviews.
The most frequently tested areas in this set are partition (9), spark (9), optimization (9), python (4), airflow (4), and sql (3). Focusing on these topics will give you the highest return on your preparation time.
Start with the easy questions to warm up and solidify fundamentals. Medium-difficulty questions form the bulk of real interviews — spend the most time here and practice explaining your reasoning out loud. Hard questions often appear in senior and staff-level rounds; attempt them after you're comfortable with the basics. For each question, try answering before revealing the solution. Use our AI Mock Interview to simulate real interview conditions and get instant feedback on your responses.
What is the difference between repartition and coalesce in Apache Spark?
What is the difference between SparkSession and SparkContext in Spark?
What strategies can you use to handle skewed data in Spark?
What is the difference between Managed and External tables in Hive/Spark?
Explain the concept of checkpointing in Spark and why it is important.
Describe how to pass data between tasks in Airflow using XComs.
Explain the concept of RDD, DataFrame, and Dataset in PySpark.
Explain the concept of consumer groups in Kafka. How do they affect message processing?
Explain the difference between TriggerDagRunOperator and ExternalTaskSensor in Airflow.
How do you ensure data quality and consistency across different stages of a data pipeline?
How do you handle failures in Airflow tasks, and what retry strategies can you use?
How do you optimize a join operation in Spark for large datasets?
How would you design a Kafka-based pipeline for processing streaming data in real-time?
Methods to avoid duplicates in PySpark/Scala?
Usage of UDFs?
What is a DAG in Apache Airflow, and how is it used for scheduling workflows?
Get full access to 1,800+ expert answers, AI mock interviews, and personalized progress tracking.