-
Notifications
You must be signed in to change notification settings - Fork 15.9k
Closed
Labels
area:APIAirflow's REST/HTTP APIAirflow's REST/HTTP APIarea:autharea:corekind:bugThis is a clearly a bugThis is a clearly a bugpriority:highHigh priority bug that should be patched quickly but does not require immediate new releaseHigh priority bug that should be patched quickly but does not require immediate new release
Description
Apache Airflow version
3.1.1
If "Other Airflow 2/3 version" selected, which one?
No response
What happened?
History begins here.
Log in / log out works right after deploy, but after some time, it starts failing (log in is impossible).
Re-creation of the API server k8s pod helps - it starts working again.
API Server log (several attempts to log in):
INFO: 172.25.119.245:58858 - "GET /api/v2/version HTTP/1.1" 200 OK
2025-10-30T07:07:47.868081Z [error [] Exception on /oauth-authorized/azure [GET[] [airflow.providers.fab.www.app] loc=app.py:1744
Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1967, in _exec_single_context
self.dialect.do_execute(
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/default.py", line 951, in do_execute
cursor.execute(statement, parameters)
File "/home/airflow/.local/lib/python3.12/site-packages/MySQLdb/cursors.py", line 179, in execute
res = self._query(mogrified_query)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/MySQLdb/cursors.py", line 330, in _query
db.query(q)
File "/home/airflow/.local/lib/python3.12/site-packages/MySQLdb/connections.py", line 280, in query
_mysql.connection.query(self, query)
MySQLdb.OperationalError: (2006, 'Server has gone away')
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.12/site-packages/flask/app.py", line 2529, in wsgi_app
response = self.full_dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/flask/app.py", line 1825, in full_dispatch_request
rv = self.handle_user_exception(e)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/flask/app.py", line 1823, in full_dispatch_request
rv = self.dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/flask/app.py", line 1799, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/flask_appbuilder/security/views.py", line 713, in oauth_authorized
user = self.appbuilder.sm.auth_user_oauth(userinfo)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/providers/fab/auth_manager/security_manager/override.py", line 2031, in auth_user_oauth
user = self.find_user(username=username)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/providers/fab/auth_manager/security_manager/override.py", line 1426, in find_user
return self.session.scalars(
^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 2459, in scalars
return self._execute_internal(
^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 2249, in _execute_internal
result: Result[Any] = compile_state_cls.orm_execute_statement(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/orm/context.py", line 306, in orm_execute_statement
result = conn.execute(
^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1419, in execute
return meth(
^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/sql/elements.py", line 526, in _execute_on_connection
return connection._execute_clauseelement(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1641, in _execute_clauseelement
ret = self._execute_context(
^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1846, in _execute_context
return self._exec_single_context(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1986, in _exec_single_context
self._handle_dbapi_exception(
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 2355, in _handle_dbapi_exception
raise sqlalchemy_exception.with_traceback(exc_info[2]) from e
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1967, in _exec_single_context
self.dialect.do_execute(
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/default.py", line 951, in do_execute
cursor.execute(statement, parameters)
File "/home/airflow/.local/lib/python3.12/site-packages/MySQLdb/cursors.py", line 179, in execute
res = self._query(mogrified_query)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/MySQLdb/cursors.py", line 330, in _query
db.query(q)
File "/home/airflow/.local/lib/python3.12/site-packages/MySQLdb/connections.py", line 280, in query
_mysql.connection.query(self, query)
sqlalchemy.exc.OperationalError: (MySQLdb.OperationalError) (2006, 'Server has gone away')
[SQL: SELECT ab_user.id, ab_user.first_name, ab_user.last_name, ab_user.username, ab_user.password, ab_user.active, ab_user.email, ab_user.last_login, ab_user.login_count, ab_user.fail_login_count, ab_user.created_on, ab_user.changed_on, ab_user.created_by_fk, ab_user.changed_by_fk
FROM ab_user
WHERE lower(ab_user.username) = lower(%s)]
[parameters: ('4dac55b4-dc30-42f5-8e4f-4846ab968868',)]
(Background on this error at: https://sqlalche.me/e/20/e3q8)
INFO: 172.25.120.253:55062 - "GET /auth/oauth-authorized/azure?code=1.AQMA9KHHarFf....dacd5d2b HTTP/1.1" 500 Internal Server Error
INFO: 172.25.120.253:55062 - "GET /favicon.ico HTTP/1.1" 200 OK
INFO: 172.25.119.245:38882 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:38884 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:33248 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:33260 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:35764 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:35760 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:39466 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:39468 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:33598 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:33584 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:59828 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:59826 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:32796 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:32790 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:53710 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:53712 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:36588 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:36590 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:34038 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:34036 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:36746 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:36744 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:37786 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:37788 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:50022 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:50036 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:48980 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:48976 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.120.253:41384 - "GET / HTTP/1.1" 200 OK
INFO: 172.25.120.253:41384 - "GET /ui/config HTTP/1.1" 401 Unauthorized
INFO: 172.25.120.253:41384 - "GET /api/v2/auth/login?next=https%3A%2F%2F<our_airflow_host>%2F HTTP/1.1" 307 Temporary Redirect
INFO: 172.25.117.209:50626 - "GET /ui/auth/menus HTTP/1.1" 401 Unauthorized
INFO: 172.25.117.209:50630 - "GET /ui/config HTTP/1.1" 401 Unauthorized
INFO: 172.25.120.253:41388 - "GET /api/v2/plugins HTTP/1.1" 401 Unauthorized
INFO: 172.25.120.253:41384 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.117.209:50626 - "GET /api/v2/auth/login?next=https%3A%2F%2F<our_airflow_host>%2F HTTP/1.1" 307 Temporary Redirect
INFO: 172.25.120.253:41384 - "GET /api/v2/auth/login?next=https%3A%2F%2F<our_airflow_host>%2F HTTP/1.1" 307 Temporary Redirect
INFO: 172.25.120.253:41384 - "GET /auth/login/?next=https://<our_airflow_host> HTTP/1.1" 200 OK
INFO: 172.25.120.253:41408 - "GET /auth/static/appbuilder/css/fontawesome/solid.min.css HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41384 - "GET /auth/static/appbuilder/css/bootstrap.min.css HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41402 - "GET /auth/static/appbuilder/css/fontawesome/regular.min.css HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41388 - "GET /auth/static/appbuilder/css/fontawesome/fontawesome.min.css HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41388 - "GET /auth/static/appbuilder/css/fontawesome/brands.min.css HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41402 - "GET /auth/static/appbuilder/css/bootstrap-datepicker/bootstrap-datepicker3.min.css HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41388 - "GET /auth/static/appbuilder/css/fontawesome/v4-shims.min.css HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41388 - "GET /auth/static/appbuilder/css/select2/select2.min.css HTTP/1.1" 304 Not Modified
INFO: 172.25.117.209:50626 - "GET /auth/static/appbuilder/css/select2/select2-bootstrap.min.css HTTP/1.1" 304 Not Modified
INFO: 172.25.117.209:50632 - "GET /auth/static/appbuilder/css/flags/flags16.css HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41388 - "GET /auth/static/appbuilder/css/ab.css HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41402 - "GET /auth/static/dist/airflowDefaultTheme.ff5a35f322070b094aa2.css HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41402 - "GET /auth/static/dist/materialIcons.3e67dd6fbfcc4f3b5105.css HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41402 - "GET /auth/static/dist/loadingDots.2e5f555f0753107b0300.css HTTP/1.1" 304 Not Modified
INFO: 172.25.117.209:50626 - "GET /auth/static/dist/jquery-ui.min.css HTTP/1.1" 304 Not Modified
INFO: 172.25.117.209:50632 - "GET /auth/static/dist/main.3cf3be1a0c5439bb640d.css HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41388 - "GET /auth/static/appbuilder/js/ab_filters.js HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41402 - "GET /auth/static/dist/flash.5583a9e0cf11f2be93da.css HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41402 - "GET /auth/static/appbuilder/js/jquery-latest.js HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41402 - "GET /auth/static/appbuilder/js/ab_actions.js HTTP/1.1" 304 Not Modified
INFO: 172.25.117.209:50626 - "GET /auth/static/appbuilder/js/bootstrap-datepicker/bootstrap-datepicker.min.js HTTP/1.1" 304 Not Modified
INFO: 172.25.117.209:50632 - "GET /auth/static/appbuilder/js/bootstrap.min.js HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41402 - "GET /auth/static/appbuilder/js/select2/select2.min.js HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41402 - "GET /auth/static/appbuilder/js/ab.js HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41402 - "GET /auth/static/dist/jquery-ui.min.js HTTP/1.1" 304 Not Modified
INFO: 172.25.117.209:50632 - "GET /auth/static/dist/main.3cf3be1a0c5439bb640d.js HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41402 - "GET /auth/static/dist/moment.9baee5ec3d7639a10897.js HTTP/1.1" 304 Not Modified
INFO: 172.25.117.209:50632 - "GET /auth/static/appbuilder/css/webfonts/fa-brands-400.woff2 HTTP/1.1" 304 Not Modified
INFO: 172.25.117.209:50632 - "GET /auth/login/azure?next=https%3A//<our_airflow_host>/ HTTP/1.1" 302 Found
2025-10-30T07:10:14.906063Z [error [] Exception on /oauth-authorized/azure [GET[] [airflow.providers.fab.www.app] loc=app.py:1744
Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.12/site-packages/flask/app.py", line 2529, in wsgi_app
response = self.full_dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/flask/app.py", line 1825, in full_dispatch_request
rv = self.handle_user_exception(e)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/flask/app.py", line 1823, in full_dispatch_request
rv = self.dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/flask/app.py", line 1799, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/flask_appbuilder/security/views.py", line 713, in oauth_authorized
user = self.appbuilder.sm.auth_user_oauth(userinfo)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/providers/fab/auth_manager/security_manager/override.py", line 2031, in auth_user_oauth
user = self.find_user(username=username)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/providers/fab/auth_manager/security_manager/override.py", line 1426, in find_user
return self.session.scalars(
^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 2459, in scalars
return self._execute_internal(
^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 2249, in _execute_internal
result: Result[Any] = compile_state_cls.orm_execute_statement(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/orm/context.py", line 306, in orm_execute_statement
result = conn.execute(
^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1419, in execute
return meth(
^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/sql/elements.py", line 526, in _execute_on_connection
return connection._execute_clauseelement(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1641, in _execute_clauseelement
ret = self._execute_context(
^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1813, in _execute_context
conn = self._revalidate_connection()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 673, in _revalidate_connection
self._invalid_transaction()
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 663, in _invalid_transaction
raise exc.PendingRollbackError(
sqlalchemy.exc.PendingRollbackError: Can't reconnect until invalid transaction is rolled back. Please rollback() fully before proceeding (Background on this error at: https://sqlalche.me/e/20/8s2b)
INFO: 172.25.117.209:50632 - "GET /auth/oauth-authorized/azure?code=1.AQMA9KHH...mCp_QE&session_state=009a7329-bd24-d99e-9121-bb58dacd5d2b HTTP/1.1" 500 Internal Server Error
INFO: 172.25.117.209:50632 - "GET /favicon.ico HTTP/1.1" 200 OK
INFO: 172.25.119.245:39872 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:39870 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:59258 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:59260 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:41694 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:41680 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:48448 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:48450 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.117.209:50632 - "GET / HTTP/1.1" 200 OK
INFO: 172.25.117.209:50632 - "GET /ui/config HTTP/1.1" 401 Unauthorized
INFO: 172.25.117.209:50632 - "GET /api/v2/auth/login?next=https%3A%2F%2F<our_airflow_host>%2F HTTP/1.1" 307 Temporary Redirect
INFO: 172.25.117.209:50632 - "GET /ui/config HTTP/1.1" 401 Unauthorized
INFO: 172.25.120.253:41402 - "GET /ui/auth/menus HTTP/1.1" 401 Unauthorized
INFO: 172.25.120.253:41388 - "GET /api/v2/plugins HTTP/1.1" 401 Unauthorized
INFO: 172.25.120.253:41384 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.117.209:50632 - "GET /auth/login/?next=https://<our_airflow_host>/ HTTP/1.1" 200 OK
INFO: 172.25.120.253:41384 - "GET /api/v2/auth/login?next=https%3A%2F%2F<our_airflow_host>%2F HTTP/1.1" 307 Temporary Redirect
INFO: 172.25.120.253:41384 - "GET /api/v2/auth/login?next=https%3A%2F%2F<our_airflow_host>%2F HTTP/1.1" 307 Temporary Redirect
INFO: 172.25.120.253:41384 - "GET /auth/login/?next=https://<our_airflow_host>/ HTTP/1.1" 200 OK
INFO: 172.25.120.253:41384 - "GET /auth/static/appbuilder/css/bootstrap.min.css HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41402 - "GET /auth/static/appbuilder/css/fontawesome/regular.min.css HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41388 - "GET /auth/static/appbuilder/css/fontawesome/fontawesome.min.css HTTP/1.1" 304 Not Modified
INFO: 172.25.117.209:50632 - "GET /auth/static/appbuilder/css/fontawesome/solid.min.css HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41388 - "GET /auth/static/appbuilder/css/bootstrap-datepicker/bootstrap-datepicker3.min.css HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41402 - "GET /auth/static/appbuilder/css/fontawesome/v4-shims.min.css HTTP/1.1" 304 Not Modified
INFO: 172.25.117.209:50632 - "GET /auth/static/appbuilder/css/select2/select2.min.css HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41402 - "GET /auth/static/appbuilder/css/select2/select2-bootstrap.min.css HTTP/1.1" 304 Not Modified
INFO: 172.25.117.209:50632 - "GET /auth/static/appbuilder/css/ab.css HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41402 - "GET /auth/static/appbuilder/css/flags/flags16.css HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41388 - "GET /auth/static/appbuilder/css/fontawesome/brands.min.css HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41384 - "GET /auth/static/dist/materialIcons.3e67dd6fbfcc4f3b5105.css HTTP/1.1" 304 Not Modified
INFO: 172.25.117.209:50632 - "GET /auth/static/dist/main.3cf3be1a0c5439bb640d.css HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41384 - "GET /auth/static/dist/airflowDefaultTheme.ff5a35f322070b094aa2.css HTTP/1.1" 304 Not Modified
INFO: 172.25.117.209:50632 - "GET /auth/static/dist/loadingDots.2e5f555f0753107b0300.css HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41388 - "GET /auth/static/dist/jquery-ui.min.css HTTP/1.1" 304 Not Modified
INFO: 172.25.117.209:50626 - "GET /auth/static/appbuilder/js/bootstrap.min.js HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41388 - "GET /auth/static/dist/flash.5583a9e0cf11f2be93da.css HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41384 - "GET /auth/static/appbuilder/js/ab_actions.js HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41402 - "GET /auth/static/appbuilder/js/ab_filters.js HTTP/1.1" 304 Not Modified
INFO: 172.25.117.209:50632 - "GET /auth/static/appbuilder/js/jquery-latest.js HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41402 - "GET /auth/static/appbuilder/js/ab.js HTTP/1.1" 304 Not Modified
INFO: 172.25.117.209:50632 - "GET /auth/static/appbuilder/js/bootstrap-datepicker/bootstrap-datepicker.min.js HTTP/1.1" 304 Not Modified
INFO: 172.25.117.209:50626 - "GET /auth/static/appbuilder/js/select2/select2.min.js HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41388 - "GET /auth/static/dist/jquery-ui.min.js HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41402 - "GET /auth/static/dist/moment.9baee5ec3d7639a10897.js HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41384 - "GET /auth/static/dist/main.3cf3be1a0c5439bb640d.js HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41388 - "GET /auth/static/appbuilder/css/webfonts/fa-brands-400.woff2 HTTP/1.1" 304 Not Modified
INFO: 172.25.119.245:56250 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:56248 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.120.253:41388 - "GET /auth/login/azure?next=https%3A//<our_airflow_host>/ HTTP/1.1" 302 Found
2025-10-30T07:11:01.580516Z [error [] Exception on /oauth-authorized/azure [GET[] [airflow.providers.fab.www.app] loc=app.py:1744
Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.12/site-packages/flask/app.py", line 2529, in wsgi_app
response = self.full_dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/flask/app.py", line 1825, in full_dispatch_request
rv = self.handle_user_exception(e)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/flask/app.py", line 1823, in full_dispatch_request
rv = self.dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/flask/app.py", line 1799, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/flask_appbuilder/security/views.py", line 713, in oauth_authorized
user = self.appbuilder.sm.auth_user_oauth(userinfo)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/providers/fab/auth_manager/security_manager/override.py", line 2031, in auth_user_oauth
user = self.find_user(username=username)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/providers/fab/auth_manager/security_manager/override.py", line 1426, in find_user
return self.session.scalars(
^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 2459, in scalars
return self._execute_internal(
^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 2249, in _execute_internal
result: Result[Any] = compile_state_cls.orm_execute_statement(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/orm/context.py", line 306, in orm_execute_statement
result = conn.execute(
^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1419, in execute
return meth(
^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/sql/elements.py", line 526, in _execute_on_connection
return connection._execute_clauseelement(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1641, in _execute_clauseelement
ret = self._execute_context(
^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1813, in _execute_context
conn = self._revalidate_connection()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 673, in _revalidate_connection
self._invalid_transaction()
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 663, in _invalid_transaction
raise exc.PendingRollbackError(
sqlalchemy.exc.PendingRollbackError: Can't reconnect until invalid transaction is rolled back. Please rollback() fully before proceeding (Background on this error at: https://sqlalche.me/e/20/8s2b)
INFO: 172.25.120.253:41388 - "GET /auth/oauth-authorized/azure?code=1.AQMA9KHHarF...E2DyWwE-IXOp4&session_state=009a7329-bd24-d99e-9121-bb58dacd5d2b HTTP/1.1" 500 Internal Server Error
INFO: 172.25.120.253:41388 - "GET /favicon.ico HTTP/1.1" 200 OK
INFO: 172.25.119.245:40900 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:40898 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:36386 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:36388 - "GET /api/v2/version HTTP/1.1" 200 OK
What you think should happen instead?
No response
How to reproduce
Not sure when, but after several hours after the deployment of v3.1.1, it can be reproduced (but if the API server pod was restarted for some reason, the problem will not appear, need to wait).
Operating System
linux
Versions of Apache Airflow Providers
No response
Deployment
Official Apache Airflow Helm Chart
Deployment details
Chart v1.18, deployed to AWS EKS, using Azuer OAuth.
Anything else?
No response
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct
Metadata
Metadata
Assignees
Labels
area:APIAirflow's REST/HTTP APIAirflow's REST/HTTP APIarea:autharea:corekind:bugThis is a clearly a bugThis is a clearly a bugpriority:highHigh priority bug that should be patched quickly but does not require immediate new releaseHigh priority bug that should be patched quickly but does not require immediate new release