这是indexloc提供的服务,不要输入任何密码
Skip to content

serialize query responses according to schema types #53

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 26 commits into from
Apr 26, 2024

Conversation

hallettj
Copy link
Collaborator

@hallettj hallettj commented Apr 25, 2024

Describe your changes

This changes response serialization to use simple JSON instead of Extended JSON (except in cases of fields with the type ExtendedJSON).

Mutation responses have not been updated with this change yet - that will be in a follow-up PR.

This could be greatly simplified if we had an internal query request type that used copies of object type definitions in all places instead of object type names. I've got some thoughts on changing v3_to_v2_query_request to work with an internal type like that instead of with the v2 query request type. But that's a big change that will have to wait until later.

Issue ticket number and link

MDB-96

@hallettj hallettj self-assigned this Apr 25, 2024
Comment on lines -22 to +21
pub aggregates: Option<Option<::std::collections::HashMap<String, crate::Aggregate>>>,
pub aggregates: Option<::std::collections::HashMap<String, crate::Aggregate>>,
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Flattening these double-option wrappings was not strictly necessary. But the double wrapping doesn't do anything for us, and I got tired of dealing with it.

Comment on lines -16 to +19
rating: 7.1
rating:
$numberDouble: "7.1"
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The strict Extended JSON serialization for rating is expected. In the sample database ratings are a mixture of double and int values. Because we have two different types in the same position type unification resolves the type to ExtendedJSON

@@ -1269,222 +1261,4 @@ mod tests {
assert_eq!(v2_request, expected);
Ok(())
}

fn make_scalar_types() -> BTreeMap<String, ScalarType> {
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I moved these into a shared test_helpers module

@hallettj hallettj marked this pull request as ready for review April 26, 2024 01:16
@hallettj
Copy link
Collaborator Author

hallettj commented Apr 26, 2024

Ready to review!

I realized that I could probably improve performance substantially by hoisting logic that's currently done per result row.

Aggregates are still serialized to extjson, and aggregates from relations aren't serialized. I'd like to fix that soon, but we don't need aggregates to work until they're supported by the engine.

Edit: I did that hoisting, and I filed https://hasurahq.atlassian.net/browse/MDB-130 to follow up on aggregate serialization

@codedmart
Copy link
Collaborator

I realized that I could probably improve performance substantially by hoisting logic that's currently done per result row.

This is follow up work? Or you did this in this PR?

@@ -95,6 +95,30 @@ impl From<Type> for ndc_models::Type {
}
}

// Should only be used for testing
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have any way of ensuring this for all these?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed these

@@ -105,6 +105,7 @@ fn find_object_field<'a>(
ConversionError::UnknownObjectTypeField {
object_type: object_type.name.to_string(),
field_name: field_name.to_string(),
path: Default::default(), // TODO: set a path for more helpful error reporting
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this TODO for later or missed in this PR?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Saving this for later

|(key, _aggregate_definition)| match aggregate_values.remove_entry(key) {
Some((owned_key, value)) => Ok((
owned_key,
// TODO: bson_to_json
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also later TODO or for this PR?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Collaborator

@codedmart codedmart left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left a few comments. Thanks!

@hallettj hallettj merged commit c654ec2 into main Apr 26, 2024
@hallettj hallettj deleted the jesse/response-serialization branch April 26, 2024 22:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants