Explain how you would use repartition or coalesce effectively to optimize processing when analyzing data only for a specific region.

Architectural Logic: Filter first; then repartition for parallelism or coalesce to reduce output. Repartition: Increase partitions when skewed or under-partitioned; df.filter(col("region")=="US").repartition(8, "region"). Coalesce: Reduce partitions after filter when data small; avoids empty partitions and many small files. Why Order: Filter early to reduce data; repartition for downstream parallelism; coalesce before write. Scalability: Tune by cluster size; monitor partition sizes for skew. Cost: Repartition causes shuffle; coalesce is cheap. Best Practice: Filter before repartition; coalesce before write to avoid small files.