Hello Everyone,

I'm looking for help/advise on understanding error we are facing in our airflow environment couple of times a day.

We are hosting airflow in azure kubernetes , for backend database we were using postgres (pgbouncer enabled) that comes with helm chart in production. As per prod recommondations we are planning to use external postgres (azure flexible server with pbbouncer) , currently its being tested in our non-prod environment.

In both prod and non-prod enviornments and irrespective of type of postgres , we are facing following error. It happens when task instance is queued but never starts execution and celery throws state mismtach error.

Executor CeleryExecutor(parallelism=48) reported that the task instance <TaskInstance: DAGNAME.TASK [queued]> finished with state failed, but the task instance's state attribute is queued. Learn more: https://airflow.apache.org/docs/apache-airflow/stable/troubleshooting.html#task-state-changed-externally

airflow image : 2.11.0
cache : redis/valkey
executor: Celery

Please advise possible root cause for these errors.