Which orchestration tools are you using in your projects?

In my projects I have primarily used Apache Airflow for batch workflow orchestration. Airflow’s DAG model, rich operator ecosystem, and strong community support make it a go‑to choice for scheduling ETL jobs. I also experimented with Prefect for its simpler API and built‑in retry logic, especially for data science pipelines that require dynamic task generation. For Kubernetes‑native workflows, I used Argo Workflows, which integrates tightly with Helm charts and allows me to run containerized tasks directly on the cluster. In smaller, lightweight scenarios, I have used Luigi for its dependency resolution and ease of deployment. Each tool has its strengths: Airflow excels at complex DAGs and monitoring, Prefect offers a more Pythonic experience, Argo provides native Kubernetes integration, and Luigi is great for quick prototyping. I choose the tool based on the team’s skill set, the complexity of the workflow, and the deployment environment.