If one executor is taking 90% of the execution time while the rest handle only 10%, how would you handle this imbalance?

This imbalance usually indicates data skew or an uneven partitioning strategy. Steps to resolve:

1. Identify the skewed key – use Spark UI or logs to find which partition is large.
2. Salting – prepend a random number to the skewed key before the shuffle, then remove it after the join.
3. Broadcast join – if the skewed side is small, broadcast it.
4. Repartition – increase the number of partitions or use a custom partitioner to spread data.
5. Adaptive Query Execution (AQE) – enable spark.sql.adaptive.enabled=true and spark.sql.adaptive.skewJoin.enabled=true to let Spark automatically handle skew.
6. Resource tuning – increase executor memory or cores if the heavy partition still dominates.

By applying these techniques, the workload can be balanced across executors, reducing the 90/10 split.