Is coalesce the same as repartition? What is the difference between them?

Coalesce and repartition both change the number of partitions, but they differ in how they do it. Coalesce reduces the number of partitions by merging existing ones without shuffling data, which is fast but can lead to uneven partition sizes if the data is skewed. Repartition can either increase or decrease partitions and always performs a full shuffle, redistributing data evenly across the new partitions. Use coalesce when shrinking partitions and when you can tolerate some imbalance; use repartition when you need a larger number of partitions or a balanced distribution.