How does a broadcast join work in Spark?

A broadcast join in Spark is a join strategy where one of the datasets (the smaller one) is sent to all worker nodes as a broadcast variable. Each executor then performs a local join between the broadcasted dataset and its partition of the larger dataset. This eliminates the need for shuffling the larger dataset across the network, which is the most expensive part of a join. The broadcasted dataset is cached in memory on each executor, so subsequent joins can reuse it. Broadcast joins are most effective when the broadcasted dataset is small enough to fit in memory on each executor, typically a few hundred megabytes. If the dataset is too large, Spark will fall back to a shuffle hash join or sort‑merge join. The key benefit is reduced network I/O and faster join performance for skewed or small‑to‑large joins.