这是indexloc提供的服务,不要输入任何密码
Skip to content

Conversation

@nizar-m
Copy link
Contributor

@nizar-m nizar-m commented Apr 1, 2019

Description

Affected components

  • Tests

Related Issues

#1868

Invocation

Examples

  1. pytest --hge-urls "http://127.0.0.1:8080" --pg-urls "postgresql://admin@127.0.0.1:5432/hge_tests" -vv
  2. pytest --hge-urls "http://127.0.0.1:8080" "http://127.0.0.1:8081" --pg-urls "postgresql://admin@127.0.0.1:5432/hge_tests" "postgresql://admin@127.0.0.1:5432/hge_tests2" -vv

Solution and Design

Reducing execution time of tests

  • The Schema setup and teardown, which were earlier done per test method, usually takes around 1 sec.
  • For mutations, the model has now been changed to only do schema setup and teardown once per test class.
  • A data setup and teardown will be done once per test instead (usually takes ~10ms).
  • For the test class to get this behaviour, one can can extend the class DefaultTestMutations.
    • The function dir() should be define which returns the location of the configuration folder.
    • Inside the configuration folder, there should be
      • Files <conf_dir>/schema_setup.yaml and <conf_dir>/schema_teardown.yaml, which has the metadata query executed during schema setup and teardown respectively
      • Files named <conf_dir>/values_setup.yaml and <conf_dir>/values_teardown.yaml. These files are executed to setup and remove data from the tables respectively.

Running Graphql queries on both http and websockets

  • Each GraphQL query/mutation is run on the both HTTP and websocket protocols
  • Pytests test parameterisation is used to achieve this
  • The errors over websockets are slightly different from that on HTTP
    • The code takes care of converting the errors in HTTP to errors in websockets

Parallel executation of tests.

  • The plugin pytest-xdist helps in running tests on parallel workers.
  • We are using this plugin to group tests by file and run on different workers.
  • Parallel test worker processes operate on separate postgres databases(and separate graphql-engines connected to these databases). Thus tests on one worker will not affect the tests on the other worker.
  • With two workers, this decreases execution times by half, as the tests on event triggers usually takes a long time, but does not consume much CPU.

Steps to test and verify

Limitations, known bugs & workarounds

@nizar-m nizar-m requested review from 0x777 and shahidhk as code owners April 1, 2019 11:25
@netlify
Copy link

netlify bot commented Apr 1, 2019

Deploy preview for hasura-docs ready!

Built with commit f2d8363

https://deploy-preview-1921--hasura-docs.netlify.com

@nizar-m nizar-m changed the title Run GraphQL queries on both http and websocket (close #1868) Run GraphQL tests on both http and websocket (close #1868) Apr 1, 2019
@hasura-bot
Copy link
Contributor

Review app for commit 0b24b30 deployed to Heroku: https://hge-ci-pull-1921.herokuapp.com
Docker image for server: hasura/graphql-engine:pull1921-0b24b30

@shahidhk shahidhk added c/server Related to server c/build-system Related to the build-release system labels Apr 2, 2019
@hasura-bot
Copy link
Contributor

Review app for commit 6810ae0 deployed to Heroku: https://hge-ci-pull-1921.herokuapp.com
Docker image for server: hasura/graphql-engine:pull1921-6810ae0

@hasura-bot
Copy link
Contributor

Review app for commit 9d093c7 deployed to Heroku: https://hge-ci-pull-1921.herokuapp.com
Docker image for server: hasura/graphql-engine:pull1921-9d093c7

2) readme about structure of pytests
3) Update Contributing.md
ecthiender
ecthiender previously approved these changes Apr 3, 2019
Copy link
Contributor

@ecthiender ecthiender left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. We just need to just document why we split the schema setup and the data setup into separate steps/scopes, before merging so that other ppl writing tests knows how to do this.

@hasura-bot
Copy link
Contributor

Review app for commit 68c5269 deployed to Heroku: https://hge-ci-pull-1921.herokuapp.com
Docker image for server: hasura/graphql-engine:pull1921-68c5269

@nizar-m nizar-m added the s/ok-to-merge Status: This pull request can be merged to master label Apr 8, 2019
@hasura-bot
Copy link
Contributor

Review app for commit 4a53cd6 deployed to Heroku: https://hge-ci-pull-1921.herokuapp.com
Docker image for server: hasura/graphql-engine:pull1921-4a53cd6

@shahidhk shahidhk changed the title Run GraphQL tests on both http and websocket (close #1868) run graphql tests on both http and websocket (close #1868) Apr 8, 2019
@shahidhk shahidhk changed the title run graphql tests on both http and websocket (close #1868) run graphql tests on both http and websocket; adds parallelism (close #1868) Apr 8, 2019
@shahidhk shahidhk changed the title run graphql tests on both http and websocket; adds parallelism (close #1868) run graphql tests on both http and websocket; add parallelism (close #1868) Apr 8, 2019
@hasura-bot
Copy link
Contributor

Review app for commit f2d8363 deployed to Heroku: https://hge-ci-pull-1921.herokuapp.com
Docker image for server: hasura/graphql-engine:pull1921-f2d8363

Copy link
Member

@shahidhk shahidhk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@shahidhk shahidhk merged commit a40bf10 into hasura:master Apr 8, 2019
@hasura-bot
Copy link
Contributor

Review app https://hge-ci-pull-1921.herokuapp.com is deleted

wawhal pushed a commit to wawhal/graphql-engine that referenced this pull request Apr 11, 2019
…asura#1868) (hasura#1921)

Examples 
1)  `
pytest --hge-urls "http://127.0.0.1:8080" --pg-urls "postgresql://admin@127.0.0.1:5432/hge_tests" -vv
`
2)  `pytest --hge-urls "http://127.0.0.1:8080"   "http://127.0.0.1:8081" --pg-urls "postgresql://admin@127.0.0.1:5432/hge_tests"  "postgresql://admin@127.0.0.1:5432/hge_tests2" -vv
`
### Solution and Design
<!-- How is this issue solved/fixed? What is the design? -->
<!-- It's better if we elaborate -->
#### Reducing execution time of tests
- The Schema setup and teardown, which were earlier done per test method, usually takes around 1 sec. 
- For mutations, the model has now been changed to only do schema setup and teardown once per test class.
-  A data setup and teardown will be done once per test instead (usually takes ~10ms).
- For the test class to get this behaviour, one can can extend the class `DefaultTestMutations`. 
    - The function  `dir()` should be define which returns the location of the configuration folder.
    - Inside the configuration folder, there should be 
        - Files `<conf_dir>/schema_setup.yaml` and `<conf_dir>/schema_teardown.yaml`, which has the metadata query executed during schema setup and teardown respectively
        - Files named `<conf_dir>/values_setup.yaml` and `<conf_dir>/values_teardown.yaml`. These files are executed to setup and remove data from the tables respectively. 

#### Running Graphql queries on both http and websockets
- Each GraphQL query/mutation is run on the both HTTP and websocket protocols
- Pytests test parameterisation is used to achieve this
- The errors over websockets are slightly different from that on HTTP
   - The code takes care of converting the errors in HTTP to errors in websockets

#### Parallel executation of tests.
- The plugin pytest-xdist helps in running tests on parallel workers.
- We are using this plugin to group tests by file and run on different workers.
- Parallel test worker processes operate on separate postgres databases(and separate graphql-engines connected to these databases). Thus tests on one worker will not affect the tests on the other worker.
- With two workers, this decreases execution times by half, as the tests on event triggers usually takes a long time, but does not consume much CPU.
tirumaraiselvan pushed a commit to tirumaraiselvan/graphql-engine that referenced this pull request Apr 11, 2019
* added feedback on migration commands (close hasura#1410) (hasura#1931)

* firebase2graphql (v0.0.2): support array types in firebase json (close hasura#1829) (hasura#1900)

* improve console error page (close hasura#272) (hasura#1890)

* add nextjs 8 jwt sample app (hasura#1943)

* persist queries in GraphiQL (closes hasura#1894) (hasura#1903)

* add realtime chat sample app with vue (hasura#1885)

* fix update_event_trigger query not persisting the changes (hasura#1950)

fix hasura#1949

* mention required key length for HMAC encryption to user (hasura#1956)

Following on from a discussion in discord, a small tweak to make the required key length for a HMAC key more obvious

* fix input argument types for custom SQL functions (close hasura#1952) (hasura#1953)

* run graphql tests on both http and websocket; add parallelism (close hasura#1868) (hasura#1921)

Examples 
1)  `
pytest --hge-urls "http://127.0.0.1:8080" --pg-urls "postgresql://admin@127.0.0.1:5432/hge_tests" -vv
`
2)  `pytest --hge-urls "http://127.0.0.1:8080"   "http://127.0.0.1:8081" --pg-urls "postgresql://admin@127.0.0.1:5432/hge_tests"  "postgresql://admin@127.0.0.1:5432/hge_tests2" -vv
`
### Solution and Design
<!-- How is this issue solved/fixed? What is the design? -->
<!-- It's better if we elaborate -->
#### Reducing execution time of tests
- The Schema setup and teardown, which were earlier done per test method, usually takes around 1 sec. 
- For mutations, the model has now been changed to only do schema setup and teardown once per test class.
-  A data setup and teardown will be done once per test instead (usually takes ~10ms).
- For the test class to get this behaviour, one can can extend the class `DefaultTestMutations`. 
    - The function  `dir()` should be define which returns the location of the configuration folder.
    - Inside the configuration folder, there should be 
        - Files `<conf_dir>/schema_setup.yaml` and `<conf_dir>/schema_teardown.yaml`, which has the metadata query executed during schema setup and teardown respectively
        - Files named `<conf_dir>/values_setup.yaml` and `<conf_dir>/values_teardown.yaml`. These files are executed to setup and remove data from the tables respectively. 

#### Running Graphql queries on both http and websockets
- Each GraphQL query/mutation is run on the both HTTP and websocket protocols
- Pytests test parameterisation is used to achieve this
- The errors over websockets are slightly different from that on HTTP
   - The code takes care of converting the errors in HTTP to errors in websockets

#### Parallel executation of tests.
- The plugin pytest-xdist helps in running tests on parallel workers.
- We are using this plugin to group tests by file and run on different workers.
- Parallel test worker processes operate on separate postgres databases(and separate graphql-engines connected to these databases). Thus tests on one worker will not affect the tests on the other worker.
- With two workers, this decreases execution times by half, as the tests on event triggers usually takes a long time, but does not consume much CPU.

* update manifests to v1.0.0-alpha42

* update regex to delete event triggers in edit-pg-dump (close hasura#1959) (hasura#1960)

* support composite foreign keys and primary keys through console (close hasura#1707) (hasura#1854)

* edit-pg-dump: fix a missing semi-colon

* fix styles in browse rows, relationship and permission tables (hasura#1969)

* update console permissions ui (hasura#1924)

* Make permissions sections as collapsibles with tooltips
* Remove 'use same permission as select' type options from row permissions section
* Added clone section to apply same permission to any table-role-action
* Disable other subsections till row permissions are set

* re-add same as other action options in permissions (hasura#1970)

* fix excessive memory consumption with bulk queries (hasura#1972)

Use 'Strict' State monad instead of 'Lazy' to avoid unevaluated memory thunks

* add placeholder for bool type fields while inserting rows (hasura#1962)

* Use UTF-8 encoding for stack hpc report generation (hasura#1974)

* fix save function in console's retry config editor (fix hasura#1975) (hasura#1976)

* Wip

* add scroll to literal blocks in docs (close hasura#1954) (hasura#1978)

* remote relationship ui v1

* wip

* ui for creating remote relationships

* wip

* ui fixes

* ui fixes

* functional ui

* namespace ui wip

* add functionality for dropping relationship

* show exact error in notification

* fix state inconsistencies

* update reldef for namespaced fields

* lint
polRk pushed a commit to polRk/graphql-engine that referenced this pull request Feb 12, 2020
…asura#1868) (hasura#1921)

Examples 
1)  `
pytest --hge-urls "http://127.0.0.1:8080" --pg-urls "postgresql://admin@127.0.0.1:5432/hge_tests" -vv
`
2)  `pytest --hge-urls "http://127.0.0.1:8080"   "http://127.0.0.1:8081" --pg-urls "postgresql://admin@127.0.0.1:5432/hge_tests"  "postgresql://admin@127.0.0.1:5432/hge_tests2" -vv
`
### Solution and Design
<!-- How is this issue solved/fixed? What is the design? -->
<!-- It's better if we elaborate -->
#### Reducing execution time of tests
- The Schema setup and teardown, which were earlier done per test method, usually takes around 1 sec. 
- For mutations, the model has now been changed to only do schema setup and teardown once per test class.
-  A data setup and teardown will be done once per test instead (usually takes ~10ms).
- For the test class to get this behaviour, one can can extend the class `DefaultTestMutations`. 
    - The function  `dir()` should be define which returns the location of the configuration folder.
    - Inside the configuration folder, there should be 
        - Files `<conf_dir>/schema_setup.yaml` and `<conf_dir>/schema_teardown.yaml`, which has the metadata query executed during schema setup and teardown respectively
        - Files named `<conf_dir>/values_setup.yaml` and `<conf_dir>/values_teardown.yaml`. These files are executed to setup and remove data from the tables respectively. 

#### Running Graphql queries on both http and websockets
- Each GraphQL query/mutation is run on the both HTTP and websocket protocols
- Pytests test parameterisation is used to achieve this
- The errors over websockets are slightly different from that on HTTP
   - The code takes care of converting the errors in HTTP to errors in websockets

#### Parallel executation of tests.
- The plugin pytest-xdist helps in running tests on parallel workers.
- We are using this plugin to group tests by file and run on different workers.
- Parallel test worker processes operate on separate postgres databases(and separate graphql-engines connected to these databases). Thus tests on one worker will not affect the tests on the other worker.
- With two workers, this decreases execution times by half, as the tests on event triggers usually takes a long time, but does not consume much CPU.
hasura-bot pushed a commit that referenced this pull request Jun 10, 2025
<!-- The PR description should answer 2 important questions: -->

### What

<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->

<!-- Consider: do we need to add a changelog entry? -->

<!-- Does this PR introduce new validation that might break old builds?
-->

<!-- Consider: do we need to put new checks behind a flag? -->

### How

<!-- How is it trying to accomplish it (what are the implementation
steps)? -->

V3_GIT_ORIGIN_REV_ID: dca8dbcf0cc50217086f5960decca6fa7ac49eac
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

c/build-system Related to the build-release system c/server Related to server s/ok-to-merge Status: This pull request can be merged to master

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants