Real questions from top companies in Spark/Big Data · easy
What is the difference between Managed and External tables in Hive/Spark?
When would you architecturally choose Dataset[T] over DataFrame in a Scala Spark pipeline, and what are the scalability and portability trade-offs? Include type-safety benefits vs. operational constraints.
What is the difference between Managed and External Tables in Databricks?
A JSON file with evolving schema needs to be ingested into a DataFrame. How would you handle new fields dynamically in PySpark without breaking the job for previous structures?
A task intermittently fails due to external API limitations. How would you configure Airflow retries and alerts to manage this situation efficiently?
Accumulator and Broadcast Variables - explain
Approaches to handling multiple tasks within a sprint?
Cache() vs Persist(): Explain the difference and use cases for caching and persisting data in Spark with memory levels.
Can you explain dynamic resource allocation in Spark? How does it help optimize job performance?
Can you explain the concept of incremental loading in Sqoop and how to use it for job processing?
Can you give a use case where Delta Live Tables would be ideal?
Can you share a time when you had to shift focus due to urgent tasks?
Cluster Resource Allocation in Spark
Compare HDFS and cloud-based storage systems in terms of scalability and performance.
Compare ORC and Parquet
Compare Spark SQL vs. Hive Performance.
Compare Spark and MapReduce for iterative workloads
Concatenate Columns in PySpark
Controlling mappers in MapReduce
Create a DataFrame with default column types
Type or paste your answer to any of these questions and our AI Coach scores it, highlights gaps, and rewrites it at FAANG quality. Free to try.