这是indexloc提供的服务,不要输入任何密码
Skip to content

Conversation

@paf31
Copy link
Contributor

@paf31 paf31 commented Dec 27, 2019

Description

This is a precursor to #3533. It is an early WIP, but I'd like to put up this PR for review. In particular, I still need to:

  • investigate whether I can avoid passing around as many aliases for CTEs
  • add more documentation
  • fix up the tests and write new tests
  • test the CLI changes
  • (if we decide it's the right approach), add the err(text) function to hdb_views or hdb_catalog and create a migration for it

Affected components

  • Server

Related Issues

#3710

Solution and Design

Steps to test and verify

Limitations, known bugs & workarounds

Server checklist

Catalog upgrade

Does this PR change Hasura Catalog version?

  • No
  • Yes
    • Updated docs with SQL for downgrading the catalog

Metadata

Does this PR add a new Metadata feature?

  • No
  • Yes
    • Does run_sql auto manages the new metadata through schema diffing?
      • Yes
      • Not required
    • Does run_sql auto manages the definitions of metadata on renaming?
      • Yes
      • Not required
    • Does export_metadata/replace_metadata supports the new metadata added?
      • Yes
      • Not required

GraphQL

  • No new GraphQL schema is generated
  • New GraphQL schema is being generated:
    • New types and typenames are correlated

Breaking changes

  • No Breaking changes

  • There are breaking changes:

    1. Metadata API

      Existing query types:

      • Modify args payload which is not backward compatible
      • Behavioural change of the API
      • Change in response JSON schema
      • Change in error code
    2. GraphQL API

      Schema Generation:

      • Change in any NamedType
      • Change in table field names

      Schema Resolve:-

      • Change in treatment of null value for any input fields
    3. Logging

      • Log JSON schema has changed
      • Log type names have changed

@paf31 paf31 requested a review from lexi-lambda as a code owner December 27, 2019 00:53
@netlify
Copy link

netlify bot commented Dec 27, 2019

Deploy preview for hasura-docs ready!

Built with commit 9f0a5d0

https://deploy-preview-3598--hasura-docs.netlify.com

@paf31 paf31 force-pushed the remove-hdb_views branch 4 times, most recently from b6acea3 to ea97bfe Compare December 27, 2019 22:11
@hasura hasura deleted a comment from hasura-bot Dec 27, 2019
@hasura-bot
Copy link
Contributor

Review app for commit ea0cab9 deployed to Heroku: https://hge-ci-pull-3598.herokuapp.com
Docker image for server: hasura/graphql-engine:pull3598-ea0cab92

@hasura hasura deleted a comment from hasura-bot Dec 27, 2019
@hasura-bot
Copy link
Contributor

Review app for commit ef9d1ee deployed to Heroku: https://hge-ci-pull-3598.herokuapp.com
Docker image for server: hasura/graphql-engine:pull3598-ef9d1ee5

@hasura-bot
Copy link
Contributor

Review app for commit 5fe1cce deployed to Heroku: https://hge-ci-pull-3598.herokuapp.com
Docker image for server: hasura/graphql-engine:pull3598-5fe1cce3

@paf31 paf31 changed the title WIP: Remove hdb_views for inserts Remove hdb_views for inserts Dec 28, 2019
@paf31
Copy link
Contributor Author

paf31 commented Dec 28, 2019

Tests are now passing 🎉

Copy link
Contributor

@lexi-lambda lexi-lambda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally speaking, these changes look reasonable to me, but I’ll be honest: I don’t really understand them. Now, to be fair, I don’t think I really understood the old approach either, so that probably says less about the code and more about the abstract problem, but I think it would be great if some of the “why?” could make it into some comments somewhere.

The comment on insertPermsCheck seems very relevant, and indeed very helpful, but I think I’m missing a few pieces of context. Maybe just extending that comment slightly would be sufficient? I’ll leave it up to you.

@0x777
Copy link
Member

0x777 commented Jan 3, 2020

@paf31 I looked through the changes, from what I understand, the generated insert statement is of this form:

WITH 
  "<table>__mutation_result_alias" AS ( insert statement returning * ),
  "<table>__inserted" AS ( SELECT .. ),
  "<table>__checks" AS ( SELECT bool_and( .. WHEN check_expression ..) ) 
SELECT
  "<table>__inserted".*
FROM
  "<table>__inserted",
  "<table>__checks"

I experimented with a check condition which involves a condition on another table through a relationship, in such cases the following insert statement has a better plan (check expression is computed in returning):

WITH 
  "<table>__mutation_result_alias" AS 
    ( insert statement returning *, check_expression AS _check_expression_value ),
  "<table>__inserted" AS ( SELECT .. ),
  "<table>__checks" AS ( SELECT bool_and( .. WHEN _check_expression_value ..) ) 
SELECT
  "<table>__inserted".*
FROM
  "<table>__inserted",
  "<table>__checks"

The two statements that I used are as follows:

Statement 1
explain analyze with "address_mutation" as (
  update
    address a
  set
    location = 'blr'
  where
    id = 1 returning a.*
),
"address_check" as (
  select
    bool_and(
      case
        when (
          exists (
            select
              1
            from
              "user" u
            where
              u.id = a.user_id
          )
        ) THEN NULL
        ELSE "hdb_catalog"."check_violation"()
      end
    )
  from
    "address_mutation" a
),
"address_return" as (
  select
    json_agg(a.*) as returning
  from
    "address_mutation" a
)
select
  a.*
from
  "address_return" a,
  "address_check";
Statement 2
explain analyze with "address_mutation" as (
  update
    address a
  set
    location = 'blr'
  where
    id = 1 returning a.*,
    (
      exists (
        select
          1
        from
          "user" u
        where
          u.id = a.user_id
      )
    ) as _check_condition
),
"address_check" as (
  select
    bool_and(
      case
        when (a._check_condition) THEN NULL
        else "hdb_catalog"."check_violation"()
      end
    )
  from
    "address_mutation" a
),
"address_return" as (
  select
    json_agg(a.*) as returning
  from
    "address_mutation" a
)
select
  a.*
from
  "address_return" a,
  "address_check";

Regarding the check_violation function defined in hdb_catalog, it would be best not to rely on functions/views defined in hdb_catalog in the light of #3533. The function body can be inlined in the statement itself?

During this refactor, I was also hoping that we would end up with simplified logic and fix some issues:

Better error messages

It would be good to have the object on which the constraint failed in the returned error message instead of the current "check constraint failed" (maybe something like "the following object '{"location":"blr"}' does not satisfy the check constraint"). This is particularly useful when you are inserting more than one objects or when inserting related data (through relationships).

Different code paths based on relationships in selection set

Assume the following schema:

user:
- id
- name
- addresses (array relationship to address table)
address:
- id
- user_id 
- location 
- user (object relationship to user table)

Let's say we issued a mutation such as this:

mutation insert_address {
  insert_address(objects: {location: "blr"}) {
    affected_rows
    returning {
      user {
        addresses {
          id
          location
      }
    }
  }
}

Now, ideally you would expect to generate SQL such as this:

with "insert_result" AS ( INSERT INTO .. )
select .. from "insert_result"

However, this doesn't work as expected (the address inserted as a part of this mutation will not be returned in the addresses relationship) because of how CTEs behave :

All the statements are executed with the same snapshot (see Chapter 13), so they cannot “see” one another's effects on the target tables

So, whenever there are relationships in returning's selection set, what we do is something different:

BEGIN
inserted rows <- INSERT INTO ...
WITH (inserted rows) AS "insert_result" SELECT .. from "insert_result"
COMMIT

This also applies in case of deletes and updates. I was wondering if having the same code path would reduce some of the issues that we have been seeing with inserts (#3148, #3375, #3609).

I have to admit, I'm not 100% familiar with the current insert mutations implementation so I do not know the implication of the changes that I'm proposing. @rakeshkky thoughts?

@paf31
Copy link
Contributor Author

paf31 commented Jan 4, 2020

@0x777 I'm not sure how I would inline the function body into the call site since it contains a RAISE statement.

I'm all for consolidating the different code paths. To be honest, I found the current way quite confusing, and I was trying to preserve the existing behavior.

@hasura-bot
Copy link
Contributor

Review app for commit 4c79b84 deployed to Heroku: https://hge-ci-pull-3598.herokuapp.com
Docker image for server: hasura/graphql-engine:pull3598-4c79b84a

@hasura-bot
Copy link
Contributor

Review app for commit acf0ce5 deployed to Heroku: https://hge-ci-pull-3598.herokuapp.com
Docker image for server: hasura/graphql-engine:pull3598-acf0ce5c

@hasura-bot
Copy link
Contributor

Review app for commit dd2cac2 deployed to Heroku: https://hge-ci-pull-3598.herokuapp.com
Docker image for server: hasura/graphql-engine:pull3598-dd2cac29

@hasura-bot
Copy link
Contributor

Review app for commit 52f1070 deployed to Heroku: https://hge-ci-pull-3598.herokuapp.com
Docker image for server: hasura/graphql-engine:pull3598-52f10702

@hasura-bot
Copy link
Contributor

Review app for commit b563972 deployed to Heroku: https://hge-ci-pull-3598.herokuapp.com
Docker image for server: hasura/graphql-engine:pull3598-b5639727

@hasura-bot
Copy link
Contributor

Review app for commit 6c4768b deployed to Heroku: https://hge-ci-pull-3598.herokuapp.com
Docker image for server: hasura/graphql-engine:pull3598-6c4768b5

@paf31 paf31 requested a review from lexi-lambda January 14, 2020 02:31
Copy link
Contributor

@lexi-lambda lexi-lambda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These code changes LGTM, though I’m not sure I really understand what this code does well enough to thoroughly review it. @0x777 or @rakeshkky, could I ask one of you to review this as well?

More generally: I think it would be great to add some kind of high-level explanatory comment somewhere that explains the general idea. I don’t understand enough to know where it belongs or what it should say, though.

@hasura-bot
Copy link
Contributor

Review app for commit 9a8af83 deployed to Heroku: https://hge-ci-pull-3598.herokuapp.com
Docker image for server: hasura/graphql-engine:pull3598-9a8af83e

0x777
0x777 previously approved these changes Jan 15, 2020
@hasura-bot
Copy link
Contributor

Review app for commit e825e64 deployed to Heroku: https://hge-ci-pull-3598.herokuapp.com
Docker image for server: hasura/graphql-engine:pull3598-e825e641

:depth: 1
:local:

From 29 to 28
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Catalog version as per this PR is 30

Shouldn't we have 30 to 29 downgrade instructions?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, thanks. I forgot to change this when I merged master (which bumped from 29 to 30).

@shahidhk shahidhk changed the title Remove hdb_views for inserts remove hdb_views for inserts Jan 16, 2020
@hasura-bot
Copy link
Contributor

Review app for commit 9f0a5d0 deployed to Heroku: https://hge-ci-pull-3598.herokuapp.com
Docker image for server: hasura/graphql-engine:pull3598-9f0a5d07

Copy link
Member

@shahidhk shahidhk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@0x777 and @lexi-lambda had already LGTM-ed

@shahidhk shahidhk merged commit 9ed8f71 into hasura:master Jan 16, 2020
@hasura-bot
Copy link
Contributor

Review app https://hge-ci-pull-3598.herokuapp.com is deleted

polRk pushed a commit to polRk/graphql-engine that referenced this pull request Feb 12, 2020
* WIP: Remove hdb_views for inserts

* Show failing row in check constraint error

* Revert "Show failing row in check constraint error"

This reverts commit dd2cac2.

* Use the better query plan

* Simplify things

* fix cli test

* Update downgrading.rst

* remove 1.1 asset for cli
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants