load_all()/select_related() pydantic validation errors for ormar.JSON() field type #1391
Description
Describe the bug
When running load_all() or select_related() on model with foreign model with ormar.JSON() field, we see a pydantic validation error.
To Reproduce
import asyncio
import databases
import ormar
import sqlalchemy
DATABASE_URL = "sqlite:///db.sqlite"
base_ormar_config = ormar.OrmarConfig(
database=databases.Database(DATABASE_URL),
metadata=sqlalchemy.MetaData(),
engine=sqlalchemy.create_engine(DATABASE_URL),
)
class Author(ormar.Model):
ormar_config = base_ormar_config.copy(tablename="authors")
id: int = ormar.Integer(primary_key=True)
class Book(ormar.Model):
ormar_config = base_ormar_config.copy(tablename="books")
id: int = ormar.Integer(primary_key=True)
author: Author = ormar.ForeignKey(Author, name="author_id")
my_data: dict = ormar.JSON(nullable=True)
async def main():
base_ormar_config.metadata.drop_all(base_ormar_config.engine)
base_ormar_config.metadata.create_all(base_ormar_config.engine)
author = await Author.objects.create()
book = await Book.objects.create(
author=author,
my_data={}
)
# this errors out
authors = await Author.objects.select_related(Author.books).all()
# this also errors out
authors = await Author.objects.select_all().all()
asyncio.run(main())Error traceback
Traceback (most recent call last):
File "/Users/tonyyao/workspace/ormar-bug/main.py", line 50, in <module>
asyncio.run(main())
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
return future.result()
File "/Users/tonyyao/workspace/ormar-bug/main.py", line 47, in main
authors = await Author.objects.select_all().all()
File "/Users/tonyyao/.local/pipx/.cache/f0aff6fe1a77366/lib/python3.9/site-packages/ormar/queryset/queryset.py", line 1084, in all
result_rows = await self._process_query_result_rows(rows)
File "/Users/tonyyao/.local/pipx/.cache/f0aff6fe1a77366/lib/python3.9/site-packages/ormar/queryset/queryset.py", line 189, in _process_query_result_rows
self.model.from_row(
File "/Users/tonyyao/.local/pipx/.cache/f0aff6fe1a77366/lib/python3.9/site-packages/ormar/models/model_row.py", line 104, in from_row
instance = cast("Model", cls(**item))
File "/Users/tonyyao/.local/pipx/.cache/f0aff6fe1a77366/lib/python3.9/site-packages/ormar/models/newbasemodel.py", line 138, in __init__
self.__pydantic_validator__.validate_python(
pydantic_core._pydantic_core.ValidationError: 12 validation errors for Author
books.int
Input should be a valid integer [type=int_type, input_value=Book({'id': 1, 'author': ...040>]}), 'my_data': {}}), input_type=Book]
For further information visit https://errors.pydantic.dev/2.5/v/int_type
books.Book.my_data
JSON input should be string, bytes or bytearray [type=json_type, input_value={}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.5/v/json_type
books.PkOnlyBookrybjdp
Input should be a valid dictionary or instance of PkOnlyBookrybjdp [type=model_type, input_value=Book({'id': 1, 'author': ...040>]}), 'my_data': {}}), input_type=Book]
For further information visit https://errors.pydantic.dev/2.5/v/model_type
books.`list[nullable[union[int,Book,...]]]`.0.int
Input should be a valid integer [type=int_type, input_value=('id', 1), input_type=tuple]
For further information visit https://errors.pydantic.dev/2.5/v/int_type
books.`list[nullable[union[int,Book,...]]]`.0.Book
Input should be a valid dictionary or object to extract fields from [type=model_attributes_type, input_value=('id', 1), input_type=tuple]
For further information visit https://errors.pydantic.dev/2.5/v/model_attributes_type
books.`list[nullable[union[int,Book,...]]]`.0.PkOnlyBookrybjdp
Input should be a valid dictionary or instance of PkOnlyBookrybjdp [type=model_type, input_value=('id', 1), input_type=tuple]
For further information visit https://errors.pydantic.dev/2.5/v/model_type
books.`list[nullable[union[int,Book,...]]]`.1.int
Input should be a valid integer [type=int_type, input_value=('author', Author({'id': ...Book at 0x102391040>]})), input_type=tuple]
For further information visit https://errors.pydantic.dev/2.5/v/int_type
books.`list[nullable[union[int,Book,...]]]`.1.Book
Input should be a valid dictionary or object to extract fields from [type=model_attributes_type, input_value=('author', Author({'id': ...Book at 0x102391040>]})), input_type=tuple]
For further information visit https://errors.pydantic.dev/2.5/v/model_attributes_type
books.`list[nullable[union[int,Book,...]]]`.1.PkOnlyBookrybjdp
Input should be a valid dictionary or instance of PkOnlyBookrybjdp [type=model_type, input_value=('author', Author({'id': ...Book at 0x102391040>]})), input_type=tuple]
For further information visit https://errors.pydantic.dev/2.5/v/model_type
books.`list[nullable[union[int,Book,...]]]`.2.int
Input should be a valid integer [type=int_type, input_value=('my_data', {}), input_type=tuple]
For further information visit https://errors.pydantic.dev/2.5/v/int_type
books.`list[nullable[union[int,Book,...]]]`.2.Book
Input should be a valid dictionary or object to extract fields from [type=model_attributes_type, input_value=('my_data', {}), input_type=tuple]
For further information visit https://errors.pydantic.dev/2.5/v/model_attributes_type
books.`list[nullable[union[int,Book,...]]]`.2.PkOnlyBookrybjdp
Input should be a valid dictionary or instance of PkOnlyBookrybjdp [type=model_type, input_value=('my_data', {}), input_type=tuple]
For further information visit https://errors.pydantic.dev/2.5/v/model_type
Expected behavior
Unless I'm using these functions the wrong way, I expect either of these functions to not hit any pydantic errors.
Versions (please complete the following information):
# /// script
# dependencies = [
# "databases[aiosqlite]==0.7.0",
# "pydantic==2.5.3",
# "ormar==0.20.1",
# "sqlalchemy==1.4.52"
# ]
# ///
Additional context
- does NOT hit validation errors if json field is optional and not populated
- does NOT hit validation errors for other field types
- does NOT hit validation errors if using
prefetch_relatedinstead ofselected_related - one mitigation to not hit validation errors is the following
author = await author.load(); books = await author.books.all()
Thanks in advance!