In my production job, the source schema has changed with the deletion of two columns. How would you ensure that this does not affect the production job?

When a source schema changes, the first step is to assess compatibility. If columns are deleted, you need to determine whether downstream transformations or queries rely on them. Use a schema registry or versioned schema files to track changes. Implement backward‑compatible transformations: keep the old columns as optional or defaulted values, or use a dynamic schema approach where the job reads only the columns that exist. In Spark, you can use the option spark.sql.sources.schemaEvolution.enabled to allow reading newer schemas. Additionally, test the job against a staging environment with the new schema before promoting to production. If the job fails, roll back to the previous schema or use a fallback data source. Monitoring and alerting should be in place to detect schema drift early.