-
Notifications
You must be signed in to change notification settings - Fork 15.4k
Closed
Labels
affected_version:2.0Issues Reported for 2.0Issues Reported for 2.0area:Schedulerincluding HA (high availability) schedulerincluding HA (high availability) schedulerkind:bugThis is a clearly a bugThis is a clearly a bugpriority:mediumBug that should be fixed before next release but would not block a releaseBug that should be fixed before next release but would not block a release
Milestone
Description
I can observe the same problem with version 2.0.2:
- Tasks fail, because a DAG/task has gone missing (we are using dynamically created DAGs, and they can go missing)
- The scheduler keeps those queued
- The pool gradually fills up with these queued tasks
- The whole operation stops, because of this behaviour
My current remedy:
- Manually remove those queued tasks
My desired solution:
When a DAG/task goes missing while it is queued, it should end up in a failed state.
Originally posted by @lukas-at-harren in #13542 (comment)
Naraujo13
Metadata
Metadata
Assignees
Labels
affected_version:2.0Issues Reported for 2.0Issues Reported for 2.0area:Schedulerincluding HA (high availability) schedulerincluding HA (high availability) schedulerkind:bugThis is a clearly a bugThis is a clearly a bugpriority:mediumBug that should be fixed before next release but would not block a releaseBug that should be fixed before next release but would not block a release