What will happen if your Spark job fails after 50% completion?

Spark’s fault‑tolerance is built on lineage and checkpointing. If a job fails after 50% completion, Spark will first mark the failed stage(s) as needing recomputation. All tasks that succeeded remain in memory or on disk if persisted, so they are not lost. Spark will then recompute only the failed stages by re‑executing the transformations that produced the missing RDD partitions, using the lineage graph to regenerate data from the original source. If the job has checkpointing enabled, Spark can truncate the lineage to the checkpoint and recompute from that point, which reduces recomputation time. If no checkpoint is available, Spark will recompute from the original source, which may be expensive. In either case, the driver will retry the failed tasks a configurable number of times before giving up. Thus, a failure after 50% completion does not mean the entire job must restart; Spark efficiently recovers by recomputing only the missing parts.