support $project stage in native query pipeline type inference #126
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
hallettj
commented
Nov 19, 2024
crates/cli/src/native_query/tests.rs
Outdated
Comment on lines
327
to
439
| let result_type_name = native_query.result_document_type; | ||
| let result_type = &native_query.object_types[&result_type_name]; | ||
|
|
||
| expect_false!(result_type.fields.contains_key("title")); | ||
|
|
||
| let tomatoes_type_name = match result_type.fields.get("tomatoes") { | ||
| Some(ObjectField { | ||
| r#type: Type::Object(name), | ||
| .. | ||
| }) => ObjectTypeName::from(name.clone()), | ||
| _ => panic!("tomatoes field does not have an object type"), | ||
| }; | ||
| let tomatoes_type = &native_query.object_types[&tomatoes_type_name]; | ||
| expect_false!(tomatoes_type.fields.contains_key("title")); | ||
| expect_that!( | ||
| tomatoes_type.fields.keys().collect_vec(), | ||
| unordered_elements_are![&&FieldName::from("viewer"), &&FieldName::from("critic")] | ||
| ); | ||
| expect_eq!( | ||
| tomatoes_type.fields["viewer"].r#type, | ||
| Type::Object("TomatoesCriticViewer".into()), | ||
| ); | ||
|
|
||
| let critic_type_name = match tomatoes_type.fields.get("critic") { | ||
| Some(ObjectField { | ||
| r#type: Type::Object(name), | ||
| .. | ||
| }) => ObjectTypeName::from(name.clone()), | ||
| _ => panic!("tomatoes.critic field does not have an object type"), | ||
| }; | ||
| let critic_type = &native_query.object_types[&critic_type_name]; | ||
| expect_eq!( | ||
| critic_type.fields, | ||
| object_fields([("numReviews", Type::Scalar(BsonScalarType::Int))]), | ||
| ); | ||
|
|
||
| Ok(()) | ||
| } | ||
|
|
||
| #[googletest::test] | ||
| fn supports_project_stage_in_inclusion_mode() -> Result<()> { | ||
| let config = mflix_config(); | ||
|
|
||
| let pipeline = Pipeline::new(vec![Stage::Project(doc! { | ||
| "title": 1, | ||
| "tomatoes.critic.rating": true, | ||
| "tomatoes.critic.meter": true, | ||
| "tomatoes.lastUpdated": true, | ||
| "releaseDate": "$released", | ||
| })]); | ||
|
|
||
| let native_query = | ||
| native_query_from_pipeline(&config, "inclusion", Some("movies".into()), pipeline)?; | ||
|
|
||
| expect_eq!(native_query.result_document_type, "inclusion_project".into()); | ||
|
|
||
| expect_eq!( | ||
| native_query.object_types, | ||
| [ | ||
| ( | ||
| "inclusion_project".into(), | ||
| ObjectType { | ||
| fields: object_fields([ | ||
| ("_id", Type::Scalar(BsonScalarType::ObjectId)), | ||
| ("title", Type::Scalar(BsonScalarType::String)), | ||
| ("tomatoes", Type::Object("inclusion_project_tomatoes".into())), | ||
| ("releaseDate", Type::Scalar(BsonScalarType::Date)), | ||
| ]), | ||
| description: None | ||
| } | ||
| ), | ||
| ( | ||
| "inclusion_project_tomatoes".into(), | ||
| ObjectType { | ||
| fields: object_fields([ | ||
| ( | ||
| "critic", | ||
| Type::Object("inclusion_project_tomatoes_critic".into()) | ||
| ), | ||
| ("lastUpdated", Type::Scalar(BsonScalarType::Date)), | ||
| ]), | ||
| description: None | ||
| } | ||
| ), | ||
| ( | ||
| "inclusion_project_tomatoes_critic".into(), | ||
| ObjectType { | ||
| fields: object_fields([ | ||
| ("rating", Type::Scalar(BsonScalarType::Double)), | ||
| ("meter", Type::Scalar(BsonScalarType::Int)), | ||
| ]), | ||
| description: None | ||
| } | ||
| ) | ||
| ] | ||
| .into(), | ||
| ); | ||
|
|
||
| Ok(()) | ||
| } |
There was a problem hiding this comment.
These tests demonstrates inference for $project in its two possible modes.
e9ee52a to
9ed82c6
Compare
Base automatically changed from
jessehallett/eng-1248-type-inference-for-more-match-operators
to
main
November 19, 2024 20:10
16177c7 to
4952d73
Compare
…when-generating-native-query-configs
codedmart
reviewed
Nov 22, 2024
| // | ||
| // TODO: This implementation does not fully account for uses of $$REMOVE. It does correctly select | ||
| // inclusion mode if $$REMOVE is used. A complete implementation would infer a nullable type for | ||
| // a projection that conditionally resolves to $$REMOVE. |
There was a problem hiding this comment.
Have you filed a ticket for this so we can come back to it at some point?
There was a problem hiding this comment.
codedmart
approved these changes
Nov 22, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is work an an in-progress feature that is gated behind a feature flag,
native-query-subcommand.This change allows the native query configuration generator to infer the output type of
$projectstages in pipelines. It also enables inference for parameters used in$limitand$skipstages.$projectis a complicated feature, but it's important to support it because it's widely used. It allows adding, removing, or modifying document fields in the document-processing pipeline. It has two modes: it can either remove fields, or it can select a subset of fields to keep while optionally adding more or changing values of kept fields. It also has dotted-path notation for easily manipulating nested structures.