这是indexloc提供的服务,不要输入任何密码
Skip to content

arguments for read-write native queries #16

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 40 commits into from
Mar 27, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
1ef17e1
add configuration for native queries; report queries in schema response
hallettj Mar 14, 2024
029ff0f
read schema and native queries from separate files
hallettj Mar 14, 2024
9ebfc75
rename metadata to schema
hallettj Mar 14, 2024
b6f6258
update fixture configuration
hallettj Mar 14, 2024
039c840
succeed parsing configuration if no native_queries are present
hallettj Mar 14, 2024
12e7b33
add mode property to NativeQuery
hallettj Mar 14, 2024
5962ae5
represent native queries as functions and procedures in schema
hallettj Mar 15, 2024
b995bee
Merge branch 'main' into jesse/native-queries
hallettj Mar 15, 2024
178695a
execute native query
hallettj Mar 18, 2024
c608d17
updated fixtures with a very basic native query
hallettj Mar 18, 2024
e9f042f
add more context to configuration read errors
hallettj Mar 18, 2024
f5d4f62
add objectTypes field to native queries
hallettj Mar 18, 2024
5385363
update fixtures with object types
hallettj Mar 18, 2024
1238cc6
Merge branch 'main' into jesse/native-queries-in-query-handler
hallettj Mar 18, 2024
e6b32b1
fix a typo
hallettj Mar 18, 2024
0d098d3
check for duplicate names when parsing configuration
hallettj Mar 18, 2024
4100fd7
implementation for read-write native queries
hallettj Mar 19, 2024
6af42b6
remove fields from Job if we're not going to use it
hallettj Mar 19, 2024
a092d72
add example mutation, insertArtist, in fixtures
hallettj Mar 19, 2024
9189bdf
Merge branch 'main' into jesse/mutations-via-native-queries
hallettj Mar 20, 2024
3c7e9df
fixture was inserting two documents
hallettj Mar 20, 2024
fae1840
wip:
hallettj Mar 21, 2024
795c59d
add json_to_bson with implementations for scalar and object types
hallettj Mar 22, 2024
2c556af
implement array conversion in json_to_bson
hallettj Mar 22, 2024
e3fcc1b
convert nullable types
hallettj Mar 22, 2024
a452ca4
cleanup
hallettj Mar 22, 2024
251ea63
Merge branch 'main' into arguments-for-native-queries
hallettj Mar 25, 2024
6c75265
wip
hallettj Mar 25, 2024
3dbb723
updates for switch to map types
hallettj Mar 25, 2024
f21e75d
rename
hallettj Mar 25, 2024
4b78a1a
Merged origin/main into arguments-for-native-queries
hallettj Mar 26, 2024
5a382bd
add Procedure type that resolves arguments, and executes command
hallettj Mar 26, 2024
d6bf2dd
connect arguments to procedure in mutation handler
hallettj Mar 26, 2024
104fbeb
update native query fixture to use arguments
hallettj Mar 26, 2024
fa6bedd
convert native query fixtures to json
hallettj Mar 26, 2024
e6d12b0
remove error variants that I didn't end up using
hallettj Mar 26, 2024
01fcded
Merge branch 'main' into arguments-for-native-queries
hallettj Mar 27, 2024
33fa050
finish the incomplete tests
hallettj Mar 27, 2024
01b5cee
lint fixes for test code
hallettj Mar 27, 2024
078baf2
Merge branch 'main' into arguments-for-native-queries
hallettj Mar 27, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 17 additions & 7 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

41 changes: 39 additions & 2 deletions crates/configuration/src/native_queries.rs
Original file line number Diff line number Diff line change
Expand Up @@ -22,13 +22,50 @@ pub struct NativeQuery {
/// `schema.json`.
pub result_type: Type,

/// Arguments for per-query customization
/// Arguments to be supplied for each query invocation. These will be substituted into the
/// given `command`.
///
/// Argument values are standard JSON mapped from GraphQL input types, not Extended JSON.
/// Values will be converted to BSON according to the types specified here.
#[serde(default)]
pub arguments: BTreeMap<String, ObjectField>,
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Technically this field should be called parameters. Is it worth changing?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If parameters is what we use elsewhere for native queries then it's probably worth changing, given that users will need to write these.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ndc-postgres uses the name arguments which is what I copied here. It's just that technically they're wrong: parameters are input variables, arguments are the values that bind to parameters. But I can see the argument that consistency is more important than technical correctness.


/// Command to run expressed as a BSON document
/// Command to run via MongoDB's `runCommand` API. For details on how to write commands see
/// https://www.mongodb.com/docs/manual/reference/method/db.runCommand/
///
/// The command is read as Extended JSON. It may be in canonical or relaxed format, or
/// a mixture of both.
/// See https://www.mongodb.com/docs/manual/reference/mongodb-extended-json/
///
/// Keys and values in the command may contain placeholders of the form `{{variableName}}`
/// which will be substituted when the native query is executed according to the given
/// arguments.
///
/// Placeholders must be inside quotes so that the command can be stored in JSON format. If the
/// command includes a string whose only content is a placeholder, when the variable is
/// substituted the string will be replaced by the type of the variable. For example in this
/// command,
///
/// ```json
/// json!({
/// "insert": "posts",
/// "documents": "{{ documents }}"
/// })
/// ```
///
/// If the type of the `documents` argument is an array then after variable substitution the
/// command will expand to:
///
/// ```json
/// json!({
/// "insert": "posts",
/// "documents": [/* array of documents */]
/// })
/// ```
///
#[schemars(with = "Object")]
pub command: bson::Document,
// TODO: test extjson deserialization

/// Determines which servers in a cluster to read from by specifying read preference, or
/// a predicate to apply to candidate servers.
Expand Down
12 changes: 12 additions & 0 deletions crates/configuration/src/schema/database.rs
Original file line number Diff line number Diff line change
Expand Up @@ -60,3 +60,15 @@ pub struct ObjectField {
#[serde(default)]
pub description: Option<String>,
}

impl ObjectField {
pub fn new(name: impl ToString, r#type: Type) -> (String, Self) {
(
name.to_string(),
ObjectField {
r#type,
description: Default::default(),
},
)
}
}
2 changes: 2 additions & 0 deletions crates/mongodb-agent-common/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ futures = "0.3.28"
futures-util = "0.3.28"
http = "^0.2"
indexmap = { version = "1", features = ["serde"] } # must match the version that ndc-client uses
indent = "^0.1"
itertools = "^0.10"
mongodb = "2.8"
mongodb-support = { path = "../mongodb-support" }
Expand All @@ -25,6 +26,7 @@ regex = "1"
schemars = { version = "^0.8.12", features = ["smol_str"] }
serde = { version = "1.0", features = ["derive"] }
serde_json = { version = "1.0", features = ["preserve_order"] }
serde_with = { version = "^3.7", features = ["base64", "hex"] }
thiserror = "1"
time = { version = "0.3.29", features = ["formatting", "parsing", "serde"] }
tracing = "0.1"
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
use std::collections::BTreeMap;

use configuration::native_queries::NativeQuery;
use configuration::{native_queries::NativeQuery, schema::ObjectType};
use mongodb::Client;

#[derive(Clone, Debug)]
Expand All @@ -11,4 +11,5 @@ pub struct MongoConfig {
pub database: String,

pub native_queries: BTreeMap<String, NativeQuery>,
pub object_types: BTreeMap<String, ObjectType>,
}
1 change: 1 addition & 0 deletions crates/mongodb-agent-common/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ pub mod health;
pub mod interface_types;
pub mod mongodb;
pub mod mongodb_connection;
pub mod procedure;
pub mod query;
pub mod scalar_types_capabilities;
pub mod schema;
Expand Down
22 changes: 22 additions & 0 deletions crates/mongodb-agent-common/src/procedure/error.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
use mongodb::bson::Bson;
use thiserror::Error;

use crate::query::arguments::ArgumentError;

#[derive(Debug, Error)]
pub enum ProcedureError {
#[error("error executing mongodb command: {0}")]
ExecutionError(#[from] mongodb::error::Error),

#[error("a required argument was not provided, \"{0}\"")]
MissingArgument(String),

#[error("found a non-string argument, {0}, in a string context - if you want to use a non-string argument it must be the only thing in the string with no white space around the curly braces")]
NonStringInStringContext(String),

#[error("object keys must be strings, but got: \"{0}\"")]
NonStringKey(Bson),

#[error("could not resolve arguments: {0}")]
UnresolvableArguments(#[from] ArgumentError),
}
Loading