这是indexloc提供的服务，不要输入任何密码
Skip to content

Build correct SQLAlchemy URI in hooks based on DbApiHook #38195

New issue
New issue
Open
Open
Build correct SQLAlchemy URI in hooks based on DbApiHook#38195
Assignees
Pyasma
Labels
good first issuekind:metaHigh-level information important to the communityprovider:apache-hiveprovider:apache-impalaprovider:common-sqlprovider:databricksprovider:exasolprovider:jdbcprovider:microsoft-mssqlprovider:mysqlprovider:postgresprovider:prestoprovider:sqliteprovider:trinoprovider:vertica
@Taragolis

Description

@Taragolis
Taragolis
opened on Mar 15, 2024

Body

Original discussion: https://lists.apache.org/thread/8rhmz3qh30hvkondct4sfmgk4vd07mn5

tl;dr; DbApiHook assumes that Airflow Connection URI representation is a valid SA URI, but it is not true and never was, it might work in simple cases, however it will fail in some complex one.

airflow/airflow/providers/common/sql/hooks/sql.py

Lines 184 to 192 in 2ab6081

def get_uri(self) -> str:
"""
Extract the URI from the connection.
:return: the extracted uri.
"""
conn = self.get_connection(getattr(self, self.conn_name_attr))
conn.schema = self.__schema or conn.schema
return conn.get_uri()

This task intend to track fixes into the hooks which based on DbApiHook.

Hooks which don't overwrite invalid DbApiHook.get_uri implementation

  • MySqlHook from mysql provider. This hook provide support for two different drivers.
  • OracleHook from oracle provider
  • JdbcHook from jdbc provider
  • TrinoHook from trino provider
  • HiveServer2Hook from apache.hive provider
  • ImpalaHook from apache.impala provider
  • DatabricksSqlHook from databricks provider
  • ExasolHook from exasol provider
  • PrestoHook from presto provider
  • VerticaHook from vertica provider
  • TeradataHook from teradata provider

Hooks which uses Connection.get_uri for construct SQLAlchemy URI

Committer

  • I acknowledge that I am a maintainer/committer of the Apache Airflow project.

Metadata

Metadata