An application processes a large dataset. One optimization technique is to use partitioning to reduce the number of partitions from 10 to fewer. What partitioning strategy would you use and why?

To reduce the number of partitions from 10 to a smaller number, you would use the coalesce transformation in Spark. Coalesce merges existing partitions without shuffling data, which is efficient when you only need to reduce the number of partitions. For example, df.coalesce(5) will combine the 10 partitions into 5, preserving data locality and minimizing data movement. If you need to increase partitions or repartition based on a key, you would use repartition or repartitionByRange. Coalesce is preferred for shrinking because it avoids a full shuffle, reduces network I/O, and keeps the job faster and less memory‑intensive. This strategy is especially useful when the dataset is large and you want to reduce the overhead of many small tasks.