randomness #2: real dkg from randomnet #12127

zjma · 2024-02-21T01:38:29Z

Description

Test Plan

…m_randomnet

trunk-io · 2024-02-21T01:38:32Z

⏱️ 14h 7m total CI duration on this PR

Job	Cumulative Duration	Recent Runs
rust-unit-coverage	4h 17m	🟩
rust-smoke-coverage	3h 17m	🟩
rust-unit-tests	1h 7m	⬜ 🟩 🟩
rust-smoke-tests	58m	🟩 🟩
windows-build	55m	🟩 🟩 🟩
execution-performance / single-node-performance	42m	🟩 🟩
forge-e2e-test / forge	40m	🟩 🟥 🟩
rust-images / rust-all	30m	🟩 🟩
forge-compat-test / forge	26m	🟩 🟩
rust-lints	16m	⬜ 🟩 🟩
run-tests-main-branch	15m	⬜ 🟥 🟥
cli-e2e-tests / run-cli-tests	15m	🟥 🟩
check	9m	⬜ 🟩 🟩
check-dynamic-deps	7m	🟩 🟩 🟩
general-lints	5m	⬜ 🟩 🟩
indexer-grpc-e2e-tests / test-indexer-grpc-docker-compose	4m	🟩 🟩
node-api-compatibility-tests / node-api-compatibility-tests	2m	🟩 🟩
semgrep/ci	1m	🟩 🟩 🟩
file_change_determinator	28s	🟩 🟩 🟩
file_change_determinator	26s	🟩 🟩 🟩
execution-performance / file_change_determinator	22s	🟩 🟩
file_change_determinator	21s	🟩 🟩
upload-to-codecov	13s	🟩
permission-check	10s	🟩 🟩 🟩
determine-docker-build-metadata	10s	🟩 🟩
permission-check	10s	🟩 🟩 🟩
permission-check	7s	🟩 🟩 🟩
permission-check	7s	🟩 🟩 🟩
permission-check	6s	🟩 🟩

_{settings ⋅ feedback ⋅ docs ⋅ learn more about trunk.io}

codecov · 2024-02-21T05:58:35Z

Codecov Report

Attention: 326 lines in your changes are missing coverage. Please review.

Comparison is base (67f372a) 70.2% compared to head (dbe537c) 69.9%.
Report is 15 commits behind head on main.

❗ Current head dbe537c differs from pull request most recent head 8677892. Consider uploading reports for the commit 8677892 to get more accurate results

Files	Patch %	Lines
types/src/dkg/real_dkg/mod.rs	0.0%	181 Missing ⚠️
types/src/dkg/real_dkg/rounding/mod.rs	79.3%	38 Missing ⚠️
aptos-move/aptos-vm/src/aptos_vm.rs	0.0%	36 Missing ⚠️
types/src/dkg/mod.rs	3.1%	31 Missing ⚠️
types/src/randomness.rs	0.0%	23 Missing ⚠️
types/src/on_chain_config/aptos_features.rs	9.0%	10 Missing ⚠️
types/src/on_chain_config/consensus_config.rs	0.0%	3 Missing ⚠️
types/src/transaction/mod.rs	50.0%	3 Missing ⚠️
dkg/src/dkg_manager/mod.rs	75.0%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff            @@
##             main   #12127     +/-   ##
=========================================
- Coverage    70.2%    69.9%   -0.4%     
=========================================
  Files        2231     2234      +3     
  Lines      419874   420434    +560     
=========================================
- Hits       295085   294009   -1076     
- Misses     124789   126425   +1636

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

danielxiangzl · 2024-02-21T20:56:22Z

types/src/dkg/real_dkg/mod.rs

+    let dkg_rounding =
+        DKGRounding::new(&validator_stakes, SECRECY_THRESHOLD, RECONSTRUCT_THRESHOLD);
+
+    println!(


We should move some parts to other places so we can properly add logs / alert.

danielxiangzl · 2024-02-21T21:02:07Z

types/src/dkg/real_dkg/mod.rs

+        Ok((sk, pk))
+    }
+
+    fn reconstruct_secret_from_shares(


This should only be used for testing?

github-actions · 2024-02-21T22:33:41Z

✅ Forge suite `compat` success on `aptos-node-v1.9.5` ==> `86778926bde121d2186ee4742b073293cbf97043`

Compatibility test results for aptos-node-v1.9.5 ==> 86778926bde121d2186ee4742b073293cbf97043 (PR)
1. Check liveness of validators at old version: aptos-node-v1.9.5
compatibility::simple-validator-upgrade::liveness-check : committed: 5379 txn/s, latency: 5125 ms, (p50: 4800 ms, p90: 8400 ms, p99: 14900 ms), latency samples: 225920
2. Upgrading first Validator to new version: 86778926bde121d2186ee4742b073293cbf97043
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 613 txn/s, submitted: 668 txn/s, expired: 55 txn/s, latency: 35424 ms, (p50: 40800 ms, p90: 58000 ms, p99: 58900 ms), latency samples: 51544
3. Upgrading rest of first batch to new version: 86778926bde121d2186ee4742b073293cbf97043
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 228 txn/s, submitted: 494 txn/s, expired: 266 txn/s, latency: 40896 ms, (p50: 46400 ms, p90: 56000 ms, p99: 61800 ms), latency samples: 20990
4. upgrading second batch to new version: 86778926bde121d2186ee4742b073293cbf97043
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 2614 txn/s, latency: 11429 ms, (p50: 13200 ms, p90: 17100 ms, p99: 17700 ms), latency samples: 115020
5. check swarm health
Compatibility test for aptos-node-v1.9.5 ==> 86778926bde121d2186ee4742b073293cbf97043 passed
Test Ok

github-actions · 2024-02-21T22:49:31Z

✅ Forge suite `realistic_env_max_load` success on `86778926bde121d2186ee4742b073293cbf97043`

two traffics test: inner traffic : committed: 7765 txn/s, latency: 5040 ms, (p50: 4800 ms, p90: 6300 ms, p99: 10500 ms), latency samples: 3362480
two traffics test : committed: 100 txn/s, latency: 2179 ms, (p50: 2100 ms, p90: 2400 ms, p99: 5300 ms), latency samples: 1740
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.222, avg: 0.201", "QsPosToProposal: max: 0.377, avg: 0.346", "ConsensusProposalToOrdered: max: 0.542, avg: 0.514", "ConsensusOrderedToCommit: max: 0.464, avg: 0.441", "ConsensusProposalToCommit: max: 0.979, avg: 0.956"]
Max round gap was 1 [limit 4] at version 1595642. Max no progress secs was 4.311876 [limit 15] at version 1595642.
Test Ok

zekun000 · 2024-02-23T00:10:09Z

types/src/dkg/real_dkg/mod.rs

+}
+
+#[derive(Debug)]
+pub struct RealDKG {}


let's not call it RealDKG, how about wDKG or something?

will rename later!

zjma added 5 commits February 19, 2024 14:57

types update from randomnet

a664783

update

0299d76

lint

e1a7dcb

Merge remote-tracking branch 'origin/main' into zjma/types_update_fro…

c503297

…m_randomnet

real dkg and rounding

dbe537c

zjma marked this pull request as ready for review February 21, 2024 01:38

zjma requested review from JoshLind, davidiw, georgemitenkov, vgao1996, wrwg and zekun000 as code owners February 21, 2024 01:38

zjma requested review from danielxiangzl and removed request for JoshLind, davidiw, georgemitenkov, vgao1996 and wrwg February 21, 2024 01:38

zjma added the CICD:run-e2e-tests when this label is present github actions will run all land-blocking e2e tests from the PR label Feb 21, 2024

This comment has been minimized.

Sign in to view

Base automatically changed from zjma/types_update_from_randomnet to main February 21, 2024 20:41

danielxiangzl reviewed Feb 21, 2024

View reviewed changes

danielxiangzl approved these changes Feb 21, 2024

View reviewed changes

rounding

8677892

This comment has been minimized.

Sign in to view

zekun000 approved these changes Feb 23, 2024

View reviewed changes

zjma merged commit e8b9b28 into main Feb 23, 2024

zjma deleted the zjma/real_dkg_from_randomnet branch February 23, 2024 00:31

banool mentioned this pull request Feb 26, 2024

Use indexer API via API Gateway throughout codebase #12231

Closed

randomness #2: real dkg from randomnet #12127

randomness #2: real dkg from randomnet #12127

Uh oh!

Conversation

zjma commented Feb 21, 2024

Description

Test Plan

Uh oh!

trunk-io bot commented Feb 21, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

codecov bot commented Feb 21, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

danielxiangzl Feb 21, 2024

Choose a reason for hiding this comment

Uh oh!

danielxiangzl Feb 21, 2024

Choose a reason for hiding this comment

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

github-actions bot commented Feb 21, 2024

✅ Forge suite compat success on aptos-node-v1.9.5 ==> 86778926bde121d2186ee4742b073293cbf97043

Uh oh!

This comment has been minimized.

github-actions bot commented Feb 21, 2024

✅ Forge suite realistic_env_max_load success on 86778926bde121d2186ee4742b073293cbf97043

Uh oh!

zekun000 Feb 23, 2024

Choose a reason for hiding this comment

Uh oh!

zjma Feb 23, 2024

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

trunk-io bot commented Feb 21, 2024 •

edited

Loading

codecov bot commented Feb 21, 2024 •

edited

Loading

✅ Forge suite `compat` success on `aptos-node-v1.9.5` ==> `86778926bde121d2186ee4742b073293cbf97043`

✅ Forge suite `realistic_env_max_load` success on `86778926bde121d2186ee4742b073293cbf97043`