-
Notifications
You must be signed in to change notification settings - Fork 15.4k
Closed
Labels
good first issuekind:bugThis is a clearly a bugThis is a clearly a bugprovider:googleGoogle (including GCP) related issuesGoogle (including GCP) related issues
Description
Apache Airflow version: 1.10.12
Environment: Centos 7 Host
- Cloud provider or hardware configuration:
- OS (e.g. from /etc/os-release): Ubuntu 20.04 Docker Container
- Kernel (e.g.
uname -a
): Linux c6c6e8230c17 3.10.0-1127.18.2.el7.x86_64 Improving the search functionality in the graph view #1 SMP Sun Jul 26 15:27:06 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux - Install tools: Docker Compose
- Others ?
What happened:
The schema file generated by MSSQLToGCSOperator specifies that a field defined as bit in MSSQL, is defined as INTEGER. However, the JSON file defines the value as true/false. When attempting to import into BigQuery using the GCSOperatorToBigQuery operator, the job fails:
Error while reading data, error message: JSON parsing error in row starting at position 0: Could not convert value 'boolean_value: false' to integer. Field: manager; Value: 0
What you expected to happen:
Value should be defined as BOOLEAN in the schema.
How to reproduce it:
Ensure dbo.Customers
table has a bit column:
export_customers = MsSqlToGoogleCloudStorageOperator(
task_id='export_customers',
sql='SELECT * FROM dbo.Customers;',
bucket='mssql-export',
filename='data/customers/export.json',
schema_filename='schemas/export.json',
mssql_conn_id='mssql_default',
google_cloud_storage_conn_id='google_cloud_default',
dag=dag
)
Metadata
Metadata
Assignees
Labels
good first issuekind:bugThis is a clearly a bugThis is a clearly a bugprovider:googleGoogle (including GCP) related issuesGoogle (including GCP) related issues