Reviewed by Aditya Kumar · Last reviewed 2026-03-25
**Steps**: (1) Install Spark (SPARK_HOME). (2) `pip install pyspark`. (3) `spark-submit --master yarn --deploy-mode client /path/script.py`. Or `--master local[*]` without YARN. (4) `--conf spark.executor.memory=4g` for config. (5) `--py-files` for zip dependencies. **Why...
Steps: (1) Install Spark (SPARK_HOME). (2) pip install pyspark. (3) spark-submit --master yarn --deploy-mode client /path/script.py. Or --master local[*] without YARN. (4) --conf spark.executor.memory=4g for config. (5) --py-files for zip dependencies.
Why Care: EC2 = raw VMs; no managed Spark. EMR provides managed Spark on EC2.
Scalability Trade-offs: Local mode = single machine. YARN = multi-node. Bootstrap scripts for repeatable EC2 setup.
Cost Implications: EC2 + self-managed vs. EMR: EMR adds ~20% for management; saves ops time.
Pro-Move: 'Bootstrap installs Spark + deps; spot fleet for workers; 50% cost cut.' Red Flag: spark-submit without --py-files—ImportError in cluster.
According to DataEngPrep.tech, this is one of the most frequently asked Spark/Big Data interview questions, reported at 1 company. DataEngPrep.tech maintains an editor-reviewed database of 1,863 data engineering interview questions across 7 categories.