Create Dynamic DAGs with Apache Airflow
Unlock the full potential of your data workflows and elevate your data engineering skills with dynamic DAGs in Apache Airflow. This course will teach you what a dynamic DAG is and how to build dynamic DAGs.
What you'll learn
Data pipelines are a vital part of any data platform. Ensuring these pipelines perform well and are easy to maintain is crucial for a successful data platform.
In this course, Create Dynamic DAGs with Apache Airflow, you’ll gain the ability to identify and build dynamic DAGs using Apache Airflow.
First, you’ll explore how to define a dynamic DAG.
Next, you’ll discover building dynamic DAGs using variables and macros.
Finally, you’ll learn how to understand the difference between dynamic and static DAGs.
When you’re finished with this course, you’ll have the skills and knowledge of dynamic DAGs in Apache Airflow needed to identify and build dynamic DAGs for your data platform.
Table of contents
About the author
Matt Stenzel is a Data and AI Cloud Solution Architect at Microsoft. He has 15+ years of experience in the data and analytics industry working in various roles including a software developer, data engineer, DW developer, DBA and integration architect. He has spent the last 8 years working at Microsoft helping companies across the US successfully implement Azure solutions that provide business value.