-
Notifications
You must be signed in to change notification settings - Fork 15.9k
Description
Apache Airflow version
Other Airflow 2/3 version (please specify below)
If "Other Airflow 2/3 version" selected, which one?
3.1.0
What happened?
We have a DAG which each time it is triggered would create 3 DAG versions. Question is why is this happening , as the impact is that the serialized_dag table is growing and would require clean up
What you think should happen instead?
would like to understand how does DAG version being triggered , we would not like to have multiple versions
How to reproduce
in this example i am not sure what triggered the DAG versions being increased , but basically we are using some custom operators and looping through a list to generate python operators
Operating System
RHEL
Versions of Apache Airflow Providers
apache-airflow==3.1.0rc2
apache-airflow-core==3.1.0rc2
apache-airflow-providers-amazon==9.14.0
apache-airflow-providers-celery==3.12.3
apache-airflow-providers-cncf-kubernetes==10.5.0
apache-airflow-providers-common-compat==1.7.4
apache-airflow-providers-common-io==1.6.3
apache-airflow-providers-common-messaging==2.0.0
apache-airflow-providers-common-sql==1.28.1
apache-airflow-providers-docker==4.4.3
apache-airflow-providers-elasticsearch==6.3.3
apache-airflow-providers-fab==2.4.3
apache-airflow-providers-ftp==3.13.2
apache-airflow-providers-git==0.0.8
apache-airflow-providers-google==18.0.0
apache-airflow-providers-grpc==3.8.2
apache-airflow-providers-hashicorp==4.3.2
apache-airflow-providers-http==5.3.4
apache-airflow-providers-microsoft-azure==12.7.1
apache-airflow-providers-microsoft-mssql==4.3.2
apache-airflow-providers-mysql==6.3.4
apache-airflow-providers-odbc==4.10.2
apache-airflow-providers-openlineage==2.7.1
apache-airflow-providers-postgres==6.3.0
apache-airflow-providers-redis==4.3.1
apache-airflow-providers-sendgrid==4.1.3
apache-airflow-providers-sftp==5.4.0
apache-airflow-providers-slack==9.3.0
apache-airflow-providers-smtp==2.2.1
apache-airflow-providers-snowflake==6.5.4
apache-airflow-providers-ssh==4.1.4
apache-airflow-providers-standard==1.8.0
apache-airflow-task-sdk==1.1.0rc2
Deployment
Official Apache Airflow Helm Chart
Deployment details
No response
Anything else?
this issue occurs everytime DAG is triggered ( scheduled daily at midnight ) and will generate 3 versions
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct