Is your experience with it compared to what you've done with Glue workflows?

I have used AWS Glue for serverless ETL jobs that run on a schedule or trigger. Glue’s integration with S3, Redshift, and Athena makes it convenient for data lake pipelines, and its Glue Workflows let me chain jobs with simple dependencies. However, Glue’s flexibility is limited compared to tools like Airflow or Prefect. With Airflow, I can write custom operators, use dynamic DAG generation, and run on Kubernetes for better scalability. Glue also has a higher learning curve for complex logic because it relies on Spark notebooks or Glue scripts. In terms of cost, Glue charges per DPU‑hour, which can be efficient for sporadic workloads, whereas Airflow on a managed cluster incurs a fixed cost. Overall, I use Glue for straightforward, fully managed ETL tasks and Airflow for complex, multi‑step pipelines that require fine‑grained control over scheduling, retries, and monitoring.