What are the components of Spark architecture?

Spark architecture is built around a driver program, a cluster manager, and worker nodes that run executors. The driver program, created by SparkContext, orchestrates the entire job: it parses the user code, builds a directed acyclic graph (DAG) of stages, and schedules tasks. The cluster manager (YARN, Mesos, or Spark's standalone mode) allocates resources across the cluster. Executors run on worker nodes; each executor is a JVM process that executes tasks, stores partitions in memory or disk, and reports progress back to the driver. Key components include: SparkContext (entry point), SparkConf (configuration), RDDs (Resilient Distributed Datasets), DataFrames/Datasets (typed APIs), DAG Scheduler (creates stages), Task Scheduler (schedules tasks to executors), Storage Level (caching strategies), and the Shuffle Manager (handles data redistribution). Together, these components enable fault‑tolerant, in‑memory distributed processing.