Write the PySpark code to find the second highest salary in each department.

Logic: Partition by department → order by salary DESC → dense_rank → filter rank=2. Code: from pyspark.sql.window import Window; from pyspark.sql import functions as F; windowSpec = Window.partitionBy("department").orderBy(F.desc("salary")); ranked = df.withColumn("rank", F.dense_rank().over(windowSpec)); result = ranked.filter(F.col("rank") == 2).select("department", "salary"). Why DENSE_RANK: Handles ties (e.g., two #1 salaries) so rank 2 is the true second-highest. Scalability trade-off: Window functions require shuffle for partitionBy; large departments can cause skew. Consider bucketing or pre-filtering if departments are very unbalanced. Cost implication: Full shuffle by department; ensure Z-ORDER or partitioning on department for large tables.