Unexpected casting order of List[Union[float, int]] when importing Pandas (1.2.4) #2835
Description
Checks
- I added a descriptive title to this issue
- I have searched (google, github) for similar issues and couldn't find anything
- I have read and followed the docs and still think this is a bug
Bug
Output of python -c "import pydantic.utils; print(pydantic.utils.version_info())":
pydantic version: 1.8.2
pydantic compiled: True
install path: /Users/wfranceys/tmp/pydantic-casting/.venv/lib/python3.7/site-packages/pydantic
python version: 3.7.4 (default, Sep 13 2019, 10:50:17) [Clang 10.0.1 (clang-1001.0.46.4)]
platform: Darwin-19.6.0-x86_64-i386-64bit
optional deps. installed: ['typing-extensions']
Output of python -c "import pandas as pd; print(pd.__version__)"
1.2.4
# main.py
from pydantic import BaseModel
from typing import List, Union
# Toggle import on and off to get different result
import pandas as pd
class Model(BaseModel):
data: List[Union[float, int]]
data = [0.1234, 0.4567]
print(Model(data=data).dict())What I would expect based on the behaviour of Unions:
{'data': [0.1234, 0.4567]}
What I see running the script above (with the import pandas as pd):
$ python main.py
{'data': [0, 0]}
What I see when removing the pandas import from above:
$ python main.py
{'data': [0.1234, 0.4567]}
Notes
Dockerfile to reproduce
Dockerfile:
FROM python:slim-buster
RUN python -m pip install pydantic==1.8.2 pandas==1.2.4
# See above
COPY main.py .
CMD python main.py
Results:
$ docker build -t pydantic-test .
$ docker run -t pydantic-test
{'data': [0, 0]}
Weirdness with pandas - what is causing the issue?
Now the part which looses me completely
Added into this collapsible section as it seems like a red herring..
- Create a virtual environment (
.venv):python -m venv .venv - Activate the environment:
source .venv/bin/activate - Install dependencies:
pip install pandas==1.2.4 pydantic==1.8.2 - Run the main.py script above:
$ python main.py
{'data': [0, 0]}
- Go to
.venv/lib/python3.7/site-packages/pandas/io/formats/format.py - Go to ~L1565, it should look like
def format_percentiles(
percentiles: Union[
np.ndarray, List[Union[int, float]], List[float], List[Union[str, float]]
]
) -> List[str]:
- Remove the types on the input, it should now look like
def format_percentiles(
percentiles
) -> List[str]:
- Now rerun the above script:
$ python main.py
{'data': [0.1234, 0.4567]}
And we now have the correct parsing again 🤔
Toggle the types on and off to get the correct results