PostgreSQL Connection URI from API Server UI #51309

raipier8818 · 2025-06-02T15:11:48Z

raipier8818
Jun 2, 2025

Hi

I'd like to get a postgres connection uri from the connection saved in api server ui, but it saved the connection uri to postgres://. So if I extract and use it without replacement, then connection can't work throwing this error:

[2025-06-03, 00:00:01] INFO - DAG bundles loaded: dags-folder: source="airflow.dag_processing.bundles.manager.DagBundlesManager"
[2025-06-03, 00:00:01] INFO - Filling up the DagBag from /opt/airflow/dags/fetch_floating_population_data.py: source="airflow.models.dagbag.DagBag"
[2025-06-03, 00:00:01] INFO - Secrets backends loaded for worker: count=1: backend_classes=["EnvironmentVariablesBackend"]: source="supervisor"
[2025-06-03, 00:00:01] INFO - Connection Retrieved '***_db': source="airflow.hooks.base"
[2025-06-03, 00:00:01] ERROR - Task failed with exception: source="task"
NoSuchModuleError: Can't load plugin: sqlalchemy.dialects:postgres
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/sdk/execution_time/task_runner.py", line 838 in run
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/sdk/execution_time/task_runner.py", line 1130 in _execute_task
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/sdk/bases/operator.py", line 408 in wrapper
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/sdk/bases/decorator.py", line 251 in execute
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/sdk/bases/operator.py", line 408 in wrapper
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/providers/standard/operators/python.py", line 212 in execute
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/providers/standard/operators/python.py", line 235 in execute_callable
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/sdk/execution_time/callback_runner.py", line 81 in run
File "/opt/airflow/dags/fetch_floating_population_data.py", line 24 in get_chunk
File "/opt/airflow/plugins/client/***_db.py", line 17 in __init__
File "<string>", line 2 in create_engine
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/util/deprecations.py", line 375 in warned
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/create.py", line 518 in create_engine
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/url.py", line 662 in _get_entrypoint
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/util/langhelpers.py", line 343 in load

So, I have to replace the prefix of connection to postresql:// like next code, then it works.

class DBConnection:
    def __init__(self):
        self.conn = BaseHook.get_connection("postgresql_connection")

        conn_str = self.conn.get_uri()
        if conn_str.startswith("postgres://"):
            conn_str = conn_str.replace("postgres://", "postgresql://", 1)

        self.engine = create_engine(conn_str, echo=True)
        self.session = Session(self.engine)
        ...

I'd like to know about the reason that postgreSQL connection must have that prefix in connection and if there is a proper way to use postgreSQL connection in airflow not my way.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

PostgreSQL Connection URI from API Server UI #51309

Uh oh!

{{title}}

Uh oh!

Replies: 0 comments

Select a reply

Uh oh!

PostgreSQL Connection URI from API Server UI #51309

Uh oh!

raipier8818 Jun 2, 2025

Replies: 0 comments

raipier8818
Jun 2, 2025