Keycloak provider not refreshing auth token properly #56614
Description
Apache Airflow Provider(s)
keycloak
Versions of Apache Airflow Providers
apache-airflow-providers-keycloak==0.1.0
Apache Airflow version
3.1.0
Operating System
MacOS 15.7.1
Deployment
Docker-Compose
Deployment details
Docker file used to build container image
ARG AIRFLOW_VERSION=3.1.0
FROM apache/airflow:${AIRFLOW_VERSION}
RUN pip install --no-cache-dir \
apache-airflow-providers-keycloak \
psycopg2-binary
Airflow parameters set in docker compose (raw values are provided via the .env file)
# Airflow Core DB Connection (uses Postgres service name)
AIRFLOW__DATABASE__SQL_ALCHEMY_CONN: postgresql://${POSTGRES_USER}:${POSTGRES_PASSWORD}@postgres:5432/${POSTGRES_DB}
# Airflow Core Configuration
AIRFLOW__CORE__EXECUTOR: LocalExecutor
AIRFLOW__CORE__FERNET_KEY: ${_AIRFLOW_FERNET_KEY}
AIRFLOW__WEBSERVER__SECRET_KEY: ${_AIRFLOW_SECRET_KEY}
AIRFLOW__CORE__LOAD_EXAMPLES: 'false'
AIRFLOW__WEBSERVER__RBAC: 'true'
# Keycloak Authorization Manager Configuration
AIRFLOW__CORE__AUTH_MANAGER: 'airflow.providers.keycloak.auth_manager.keycloak_auth_manager.KeycloakAuthManager'
AIRFLOW__KEYCLOAK_AUTH_MANAGER__CLIENT_ID: ${KC_CLIENT_ID}
AIRFLOW__KEYCLOAK_AUTH_MANAGER__CLIENT_SECRET: ${KC_CLIENT_SECRET}
AIRFLOW__KEYCLOAK_AUTH_MANAGER__REALM: ${KC_REALM}
# Point the Airflow containers to the Keycloak service name
AIRFLOW__KEYCLOAK_AUTH_MANAGER__SERVER_URL: 'http://10.10.0.141:8080'
# Required for Docker file permissions
AIRFLOW_UID: ${AIRFLOW_UID}
AIRFLOW_GID: ${AIRFLOW_GID}
# Authentication must be disabled locally, as it's delegated to Keycloak
_AIRFLOW_WWW_USER_USERNAME: ''
_AIRFLOW_WWW_USER_PASSWORD: ''
What happened
The keycloak token lifetime is set to 5 minutes. I can log in properly to the Airflow, it requests correct permissions but after the token lifetime expires the UI starts to show 500 Internal server error. What I see in the logs is the message with Invalid bearer token
Whole stack trace from the logs
INFO: 192.168.65.1:39410 - "GET /ui/dashboard/historical_metrics_data?start_date=2025-10-13T09%3A41%3A55.870Z HTTP/1.1" 500 Internal Server Error
ERROR: Exception in ASGI application
Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.12/site-packages/uvicorn/protocols/http/httptools_impl.py", line 409, in run_asgi
result = await app( # type: ignore[func-returns-value]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/fastapi/applications.py", line 1082, in __call__
await super().__call__(scope, receive, send)
File "/home/airflow/.local/lib/python3.12/site-packages/starlette/applications.py", line 113, in __call__
await self.middleware_stack(scope, receive, send)
File "/home/airflow/.local/lib/python3.12/site-packages/starlette/middleware/errors.py", line 186, in __call__
raise exc
File "/home/airflow/.local/lib/python3.12/site-packages/starlette/middleware/errors.py", line 164, in __call__
await self.app(scope, receive, _send)
File "/home/airflow/.local/lib/python3.12/site-packages/starlette/middleware/gzip.py", line 29, in __call__
await responder(scope, receive, send)
File "/home/airflow/.local/lib/python3.12/site-packages/starlette/middleware/gzip.py", line 130, in __call__
await super().__call__(scope, receive, send)
File "/home/airflow/.local/lib/python3.12/site-packages/starlette/middleware/gzip.py", line 46, in __call__
await self.app(scope, receive, self.send_with_compression)
File "/home/airflow/.local/lib/python3.12/site-packages/starlette/middleware/cors.py", line 85, in __call__
await self.app(scope, receive, send)
File "/home/airflow/.local/lib/python3.12/site-packages/starlette/middleware/exceptions.py", line 63, in __call__
await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
File "/home/airflow/.local/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
raise exc
File "/home/airflow/.local/lib/python3.12/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
await app(scope, receive, sender)
File "/home/airflow/.local/lib/python3.12/site-packages/starlette/routing.py", line 716, in __call__
await self.middleware_stack(scope, receive, send)
File "/home/airflow/.local/lib/python3.12/site-packages/starlette/routing.py", line 736, in app
await route.handle(scope, receive, send)
File "/home/airflow/.local/lib/python3.12/site-packages/starlette/routing.py", line 290, in handle
await self.app(scope, receive, send)
File "/home/airflow/.local/lib/python3.12/site-packages/starlette/routing.py", line 78, in app
await wrap_app_handling_exceptions(app, request)(scope, receive, send)
File "/home/airflow/.local/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
raise exc
File "/home/airflow/.local/lib/python3.12/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
await app(scope, receive, sender)
File "/home/airflow/.local/lib/python3.12/site-packages/starlette/routing.py", line 75, in app
response = await f(request)
^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/fastapi/routing.py", line 298, in app
solved_result = await solve_dependencies(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/fastapi/dependencies/utils.py", line 648, in solve_dependencies
solved = await run_in_threadpool(call, **solved_result.values)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/starlette/concurrency.py", line 38, in run_in_threadpool
return await anyio.to_thread.run_sync(func)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/anyio/to_thread.py", line 56, in run_sync
return await get_async_backend().run_sync_in_worker_thread(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/anyio/_backends/_asyncio.py", line 2485, in run_sync_in_worker_thread
return await future
^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/anyio/_backends/_asyncio.py", line 976, in run
result = context.run(func, *args)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/api_fastapi/core_api/security.py", line 125, in inner
_requires_access(
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/api_fastapi/core_api/security.py", line 462, in _requires_access
if not is_authorized_callback():
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/api_fastapi/core_api/security.py", line 126, in <lambda>
is_authorized_callback=lambda: get_auth_manager().is_authorized_dag(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/providers/keycloak/auth_manager/keycloak_auth_manager.py", line 152, in is_authorized_dag
return self._is_authorized(
^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/providers/keycloak/auth_manager/keycloak_auth_manager.py", line 307, in _is_authorized
raise AirflowException(f"Unexpected error: {resp.status_code} - {resp.text}")
airflow.exceptions.AirflowException: Unexpected error: 401 - {"error":"invalid_grant","error_description":"Invalid bearer token"}
It looks like for some reasons Airflow is not using a refresh token to get a new authorization token
What you think should happen instead
The token should be renewed using refresh token mechanism
How to reproduce
Deploy airflow with keycloak provider.
Configure keycloak with Standard flow, needed resources and simple policies that grant access to everything.
Anything else
No response
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct