What is the difference between map and flatMap in Spark, and when would you use each?

map: 1 input element → 1 output element; output collection has same cardinality. flatMap: 1 input element → 0 or more output elements; results are flattened into a single collection. Why it matters: flatMap controls cardinality explosion (e.g., tokenizing a line into N words) and avoids nested structures that would require a separate explode step. Scalability trade-off: flatMap can dramatically increase partition size if one input yields many outputs (e.g., huge JSON arrays); consider repartitioning or controlling output to avoid skew. Cost implication: flatMap is a narrow transformation (no shuffle); both map and flatMap are pipeline-friendly. Example: rdd.map(lambda x: (x, 1)) vs rdd.flatMap(lambda x: x.split()). In DataFrames: flatMap ≈ explode. Use map for 1:1 transforms; flatMap for splitting, exploding, tokenization, or when returning empty for filtering.