这是indexloc提供的服务,不要输入任何密码
Skip to content

Conversation

@0x777
Copy link
Member

@0x777 0x777 commented Jul 1, 2020

This PR introduces changes that should reduce memory consumption (which could result in a leak) on both graphql-engine and Postgres for a mutation heavy workload.

Description

With mutations we end up with several 'single-use' prepared SQL statements, i.e, statements which cannot be reused for multiple mutations. The underlying reason for ending up with 'single-use' prepared statements is we embed values inside the statement instead of parameterizing them to reduce the complexity around handling relationships in returning clause. The problem with single-use prepared statements is that we end up leaking memory both on graphql-engine (we store a mapping of "SQL prepared statement" to "postgres's prepared statement id" in each PGConn) and on Postgres (no limits on number of prepared statements per connection).

Changelog

  • CHANGELOG.md is updated with user-facing content relevant to this PR. If no changelog is required, then add the no-changelog-required label.

Affected components

  • Server

Related Issues

#2012
#3388
#4077

Solution and Design

Prepared statements are not used for mutations any more. We might potentially end up with slightly slower but the benefit of predictable memory usage outweighs the potential degradation.

Steps to test and verify

@tirumaraiselvan has been running some benchmarks with a mutation heavy workload. This PR will be updated with the findings from those benchmarks.

Does this PR add a new Metadata feature?

  • No

GraphQL

  • No new GraphQL schema is generated

Breaking changes

  • No Breaking changes

@0x777 0x777 requested review from jberryman and rakeshkky July 1, 2020 13:30
@netlify
Copy link

netlify bot commented Jul 1, 2020

Deploy preview for hasura-docs ready!

Built with commit 2a088d6

https://deploy-preview-5255--hasura-docs.netlify.app

@0x777 0x777 marked this pull request as ready for review July 1, 2020 13:33
@0x777 0x777 requested a review from a team as a code owner July 1, 2020 13:33
@hasura-bot
Copy link
Contributor

Review app for commit 4b43f88 deployed to Heroku: https://hge-ci-pull-5255.herokuapp.com
Docker image for server: hasura/graphql-engine:pull5255-4b43f88f

@tirumaraiselvan
Copy link
Contributor

Ran the benchmark with nested mutations:

On this build:

Hasura: 215MB
PG: 135MB (right after benchmark)

On v1.3.0-beta.3:

Hasura: 600MB
PG: 2.2G (right after benchmark), 50MB (after 5 minutes)

Copy link
Collaborator

@jberryman jberryman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agree with Tiru's suggested edit, otherwise awesome!

@0x777 0x777 merged commit d464208 into hasura:master Jul 2, 2020
@hasura-bot
Copy link
Contributor

Review app https://hge-ci-pull-5255.herokuapp.com is deleted

@hasura-bot
Copy link
Contributor

Review app for commit cfffade deployed to Heroku: https://hge-ci-pull-5255.herokuapp.com
Docker image for server: hasura/graphql-engine:pull5255-cfffade1

@nirvdrum
Copy link
Contributor

nirvdrum commented Jul 7, 2020

I have no familiarity with talking to PostgreSQL from Haskell or even how Hasura takes user input to pass on via SQL, so apologies in advance if this is a misguided question, but how are SQL injection attacks guarded against? I had assumed that prepared statements were being used to prevent SQL injection issues and so seeing them removed caught my eye.

@bitjson bitjson mentioned this pull request Oct 11, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants