FedEx Dataworks

Explain wide vs. narrow transformations and how they drive shuffle cost, failure domains, and pipeline design. When would you intentionally add a wide transformation, and how do you minimize its impact?

Architecturally, how do Job–Stage–Task boundaries in Spark's execution model impact cluster sizing, shuffle cost, and when would you deliberately collapse or split stages?

What are the key components of the Spark execution model (Job, Stage, Task)?

Create Spark Session, read CSV, join, and write as table. Provide example code.

How do you give permission to a notebook to other users in Databricks?

How does Autoscaling work in Databricks and what are its benefits?

Provide example code for Drop Duplicates in PySpark.