Can you explain how a broadcast join works?

A broadcast join is an optimization technique used in distributed data processing frameworks like Spark. In a broadcast join, the smaller dataset is sent (broadcast) to all worker nodes, so each node has a local copy of it. The larger dataset is then joined locally on each node with the broadcasted data. This eliminates the need for shuffling the larger dataset across the network, which can be expensive. The join type is typically a broadcast‑hash join. It works well when one side of the join is small enough to fit in memory on each executor. The framework automatically decides to broadcast based on size thresholds or you can force it with a broadcast hint. This leads to faster join performance for skewed or small‑to‑large joins.