It's because of joins. How would you identify whether the issue is in the join keys, aggregation logic, or source code?

Start by isolating the problem scope. First, compare row counts before and after each join to detect cardinality changes; a sudden increase indicates a join key issue. Next, verify that the join keys are unique and correctly typed by inspecting the schema and using sample data to confirm equality. If row counts are correct but aggregated results are off, examine the aggregation logic: check GROUP BY columns, aggregation functions, and any filters applied after aggregation. Use Spark UI or query plan to see if the aggregation is performed on the correct dataset. Finally, if both join and aggregation look correct, review the source code for data transformations that might alter values (e.g., null handling, type casting, or incorrect joins). Logging intermediate results and using unit tests on small subsets can help pinpoint the exact layer where the data diverges.