Home › Interview Questions › Explain the difference between an executor and a d…

Explain the difference between an executor and a driver in Apache Spark.

🟔 Medium Conceptual Mid level
1Times asked
May 2026Last seen
May 2026First seen

šŸ’” Model Answer

In Spark, the driver is the master node that orchestrates the entire job. It parses the user program, creates a directed acyclic graph (DAG) of stages, schedules tasks, and tracks progress. Executors are worker processes launched on cluster nodes; each executor runs multiple tasks in parallel. The driver sends serialized tasks to executors, which execute them on local data partitions. Executors store intermediate data in memory or disk (RDD partitions) and return results to the driver. Key differences: the driver manages job control and metadata, while executors perform the actual computation. The driver runs in a single JVM, whereas executors can be many, each with its own JVM. If the driver fails, the whole job fails; executor failures are handled by rescheduling tasks. Understanding this separation is crucial for tuning performance, memory usage, and fault tolerance in Spark applications.

This answer was generated by AI for study purposes. Use it as a starting point — personalize it with your own experience.

šŸŽ¤ Get questions like this answered in real-time

Assisting AI listens to your interview, captures questions live, and gives you instant AI-powered answers — invisible to screen sharing.

Get Assisting AI — Starts at ₹500