fix: deduplicate slots to delete archived DatacolumnSidecars #8357
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
There was a problem hiding this comment.
Summary of Changes
Hello @twoeths, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!
This pull request addresses a critical bug in the archiveBlocks.ts utility responsible for pruning old DataColumnSidecars. The fix ensures that the deletion process correctly identifies and removes unique data entries by preventing duplicate slot identifiers from being processed, thereby maintaining data integrity and efficient storage management.
Highlights
- DataColumnSidecar Deletion Fix: Implemented a deduplication mechanism using a Set to ensure only unique slot identifiers are processed when deleting archived DataColumnSidecars in archiveBlocks.ts, resolving an issue where duplicate slots were causing incorrect deletion behavior.
Using Gemini Code Assist
The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.
Invoking Gemini
You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.
| Feature | Command | Description |
|---|---|---|
| Code Review | /gemini review |
Performs a code review for the current pull request in its current state. |
| Pull Request Summary | /gemini summary |
Provides a summary of the current pull request in its current state. |
| Comment | @gemini-code-assist | Responds in comments when explicitly tagged, both in issue comments and review comments. |
| Help | /gemini help |
Displays a list of available commands. |
Customization
To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.
Limitations & Feedback
Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.
You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.
Footnotes
-
Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩
Contributor
There was a problem hiding this comment.
Code Review
This pull request correctly addresses a bug where duplicate slots could be passed for deletion when archiving DataColumnSidecars. The fix involves using a Set to deduplicate the slots, which is the right approach. The changes also include improved logging, which adds valuable context for debugging. My review includes a suggestion to make the deduplication and sorting logic more concise, which would improve the code's readability and maintainability.
packages/beacon-node/src/chain/archiveStore/utils/archiveBlocks.ts
Outdated
Show resolved
Hide resolved
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## unstable #8357 +/- ##
============================================
- Coverage 54.07% 54.07% -0.01%
============================================
Files 849 849
Lines 64147 64148 +1
Branches 4852 4853 +1
============================================
- Hits 34689 34688 -1
- Misses 29381 29383 +2
Partials 77 77 🚀 New features to boost your workflow:
|
Contributor
|
| Benchmark suite | Current: a9f2667 | Previous: 2f71bae | Ratio |
|---|---|---|---|
| getSlashingsAndExits - default max | 307.60 us/op | 72.866 us/op | 4.22 |
Full benchmark results
| Benchmark suite | Current: a9f2667 | Previous: 2f71bae | Ratio |
|---|---|---|---|
| getPubkeys - index2pubkey - req 1000 vs - 250000 vc | 1.7262 ms/op | 977.31 us/op | 1.77 |
| getPubkeys - validatorsArr - req 1000 vs - 250000 vc | 45.745 us/op | 36.272 us/op | 1.26 |
| BLS verify - blst | 1.1940 ms/op | 966.03 us/op | 1.24 |
| BLS verifyMultipleSignatures 3 - blst | 1.7371 ms/op | 1.3040 ms/op | 1.33 |
| BLS verifyMultipleSignatures 8 - blst | 2.5927 ms/op | 2.1128 ms/op | 1.23 |
| BLS verifyMultipleSignatures 32 - blst | 8.3882 ms/op | 5.6775 ms/op | 1.48 |
| BLS verifyMultipleSignatures 64 - blst | 14.183 ms/op | 10.696 ms/op | 1.33 |
| BLS verifyMultipleSignatures 128 - blst | 23.304 ms/op | 17.892 ms/op | 1.30 |
| BLS deserializing 10000 signatures | 918.42 ms/op | 693.14 ms/op | 1.33 |
| BLS deserializing 100000 signatures | 9.5739 s/op | 6.9414 s/op | 1.38 |
| BLS verifyMultipleSignatures - same message - 3 - blst | 1.3587 ms/op | 896.35 us/op | 1.52 |
| BLS verifyMultipleSignatures - same message - 8 - blst | 1.5424 ms/op | 1.0415 ms/op | 1.48 |
| BLS verifyMultipleSignatures - same message - 32 - blst | 2.8798 ms/op | 1.7067 ms/op | 1.69 |
| BLS verifyMultipleSignatures - same message - 64 - blst | 3.7833 ms/op | 2.6540 ms/op | 1.43 |
| BLS verifyMultipleSignatures - same message - 128 - blst | 5.9582 ms/op | 4.3915 ms/op | 1.36 |
| BLS aggregatePubkeys 32 - blst | 29.160 us/op | 19.466 us/op | 1.50 |
| BLS aggregatePubkeys 128 - blst | 95.490 us/op | 69.276 us/op | 1.38 |
| notSeenSlots=1 numMissedVotes=1 numBadVotes=10 | 126.89 ms/op | 60.021 ms/op | 2.11 |
| notSeenSlots=1 numMissedVotes=0 numBadVotes=4 | 93.490 ms/op | 49.572 ms/op | 1.89 |
| notSeenSlots=2 numMissedVotes=1 numBadVotes=10 | 61.236 ms/op | 35.476 ms/op | 1.73 |
| getSlashingsAndExits - default max | 307.60 us/op | 72.866 us/op | 4.22 |
| getSlashingsAndExits - 2k | 393.75 us/op | 353.10 us/op | 1.12 |
| proposeBlockBody type=full, size=empty | 6.7479 ms/op | 5.6563 ms/op | 1.19 |
| isKnown best case - 1 super set check | 240.00 ns/op | 208.00 ns/op | 1.15 |
| isKnown normal case - 2 super set checks | 226.00 ns/op | 203.00 ns/op | 1.11 |
| isKnown worse case - 16 super set checks | 222.00 ns/op | 204.00 ns/op | 1.09 |
| InMemoryCheckpointStateCache - add get delete | 3.8200 us/op | 2.4010 us/op | 1.59 |
| validate api signedAggregateAndProof - struct | 1.7455 ms/op | 1.7543 ms/op | 0.99 |
| validate gossip signedAggregateAndProof - struct | 1.6668 ms/op | 2.2705 ms/op | 0.73 |
| batch validate gossip attestation - vc 640000 - chunk 32 | 132.87 us/op | 114.96 us/op | 1.16 |
| batch validate gossip attestation - vc 640000 - chunk 64 | 114.80 us/op | 101.09 us/op | 1.14 |
| batch validate gossip attestation - vc 640000 - chunk 128 | 112.95 us/op | 95.030 us/op | 1.19 |
| batch validate gossip attestation - vc 640000 - chunk 256 | 113.22 us/op | 97.805 us/op | 1.16 |
| pickEth1Vote - no votes | 1.0852 ms/op | 921.36 us/op | 1.18 |
| pickEth1Vote - max votes | 9.5222 ms/op | 5.4383 ms/op | 1.75 |
| pickEth1Vote - Eth1Data hashTreeRoot value x2048 | 15.658 ms/op | 11.614 ms/op | 1.35 |
| pickEth1Vote - Eth1Data hashTreeRoot tree x2048 | 24.411 ms/op | 21.379 ms/op | 1.14 |
| pickEth1Vote - Eth1Data fastSerialize value x2048 | 451.39 us/op | 436.66 us/op | 1.03 |
| pickEth1Vote - Eth1Data fastSerialize tree x2048 | 3.0374 ms/op | 2.8667 ms/op | 1.06 |
| bytes32 toHexString | 413.00 ns/op | 356.00 ns/op | 1.16 |
| bytes32 Buffer.toString(hex) | 277.00 ns/op | 237.00 ns/op | 1.17 |
| bytes32 Buffer.toString(hex) from Uint8Array | 359.00 ns/op | 322.00 ns/op | 1.11 |
| bytes32 Buffer.toString(hex) + 0x | 273.00 ns/op | 238.00 ns/op | 1.15 |
| Object access 1 prop | 0.13400 ns/op | 0.11200 ns/op | 1.20 |
| Map access 1 prop | 0.13700 ns/op | 0.12100 ns/op | 1.13 |
| Object get x1000 | 6.3810 ns/op | 6.4030 ns/op | 1.00 |
| Map get x1000 | 6.8450 ns/op | 6.2760 ns/op | 1.09 |
| Object set x1000 | 34.319 ns/op | 28.594 ns/op | 1.20 |
| Map set x1000 | 22.962 ns/op | 18.941 ns/op | 1.21 |
| Return object 10000 times | 0.31830 ns/op | 0.28190 ns/op | 1.13 |
| Throw Error 10000 times | 5.0328 us/op | 4.3935 us/op | 1.15 |
| toHex | 154.03 ns/op | 130.13 ns/op | 1.18 |
| Buffer.from | 131.64 ns/op | 123.96 ns/op | 1.06 |
| shared Buffer | 89.914 ns/op | 89.842 ns/op | 1.00 |
| fastMsgIdFn sha256 / 200 bytes | 2.4910 us/op | 2.1600 us/op | 1.15 |
| fastMsgIdFn h32 xxhash / 200 bytes | 233.00 ns/op | 205.00 ns/op | 1.14 |
| fastMsgIdFn h64 xxhash / 200 bytes | 308.00 ns/op | 264.00 ns/op | 1.17 |
| fastMsgIdFn sha256 / 1000 bytes | 7.6000 us/op | 7.3310 us/op | 1.04 |
| fastMsgIdFn h32 xxhash / 1000 bytes | 401.00 ns/op | 330.00 ns/op | 1.22 |
| fastMsgIdFn h64 xxhash / 1000 bytes | 372.00 ns/op | 335.00 ns/op | 1.11 |
| fastMsgIdFn sha256 / 10000 bytes | 78.038 us/op | 64.693 us/op | 1.21 |
| fastMsgIdFn h32 xxhash / 10000 bytes | 1.9300 us/op | 1.8280 us/op | 1.06 |
| fastMsgIdFn h64 xxhash / 10000 bytes | 1.4680 us/op | 1.2040 us/op | 1.22 |
| send data - 1000 256B messages | 21.566 ms/op | 15.410 ms/op | 1.40 |
| send data - 1000 512B messages | 25.390 ms/op | 19.591 ms/op | 1.30 |
| send data - 1000 1024B messages | 33.390 ms/op | 27.665 ms/op | 1.21 |
| send data - 1000 1200B messages | 27.952 ms/op | 23.208 ms/op | 1.20 |
| send data - 1000 2048B messages | 28.665 ms/op | 23.463 ms/op | 1.22 |
| send data - 1000 4096B messages | 31.875 ms/op | 25.874 ms/op | 1.23 |
| send data - 1000 16384B messages | 51.661 ms/op | 43.173 ms/op | 1.20 |
| send data - 1000 65536B messages | 142.89 ms/op | 116.07 ms/op | 1.23 |
| enrSubnets - fastDeserialize 64 bits | 955.00 ns/op | 884.00 ns/op | 1.08 |
| enrSubnets - ssz BitVector 64 bits | 361.00 ns/op | 344.00 ns/op | 1.05 |
| enrSubnets - fastDeserialize 4 bits | 147.00 ns/op | 128.00 ns/op | 1.15 |
| enrSubnets - ssz BitVector 4 bits | 361.00 ns/op | 340.00 ns/op | 1.06 |
| prioritizePeers score -10:0 att 32-0.1 sync 2-0 | 254.81 us/op | 239.66 us/op | 1.06 |
| prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 | 290.94 us/op | 265.27 us/op | 1.10 |
| prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 | 543.73 us/op | 370.36 us/op | 1.47 |
| prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 | 779.97 us/op | 698.76 us/op | 1.12 |
| prioritizePeers score 0:0 att 64-1 sync 4-1 | 969.41 us/op | 833.72 us/op | 1.16 |
| array of 16000 items push then shift | 1.7086 us/op | 1.6248 us/op | 1.05 |
| LinkedList of 16000 items push then shift | 8.6560 ns/op | 6.9120 ns/op | 1.25 |
| array of 16000 items push then pop | 86.376 ns/op | 75.794 ns/op | 1.14 |
| LinkedList of 16000 items push then pop | 7.9690 ns/op | 7.1350 ns/op | 1.12 |
| array of 24000 items push then shift | 2.4975 us/op | 2.3948 us/op | 1.04 |
| LinkedList of 24000 items push then shift | 10.388 ns/op | 7.1960 ns/op | 1.44 |
| array of 24000 items push then pop | 119.31 ns/op | 103.29 ns/op | 1.16 |
| LinkedList of 24000 items push then pop | 10.127 ns/op | 7.0050 ns/op | 1.45 |
| intersect bitArray bitLen 8 | 6.7830 ns/op | 6.4700 ns/op | 1.05 |
| intersect array and set length 8 | 42.460 ns/op | 38.521 ns/op | 1.10 |
| intersect bitArray bitLen 128 | 32.792 ns/op | 29.877 ns/op | 1.10 |
| intersect array and set length 128 | 699.14 ns/op | 620.79 ns/op | 1.13 |
| bitArray.getTrueBitIndexes() bitLen 128 | 1.3250 us/op | 1.0070 us/op | 1.32 |
| bitArray.getTrueBitIndexes() bitLen 248 | 2.1170 us/op | 1.7530 us/op | 1.21 |
| bitArray.getTrueBitIndexes() bitLen 512 | 4.2110 us/op | 3.6150 us/op | 1.16 |
| Buffer.concat 32 items | 864.00 ns/op | 605.00 ns/op | 1.43 |
| Uint8Array.set 32 items | 1.9730 us/op | 872.00 ns/op | 2.26 |
| Buffer.copy | 3.3810 us/op | 2.0490 us/op | 1.65 |
| Uint8Array.set - with subarray | 3.2150 us/op | 1.6050 us/op | 2.00 |
| Uint8Array.set - without subarray | 2.4040 us/op | 941.00 ns/op | 2.55 |
| getUint32 - dataview | 218.00 ns/op | 196.00 ns/op | 1.11 |
| getUint32 - manual | 132.00 ns/op | 120.00 ns/op | 1.10 |
| Set add up to 64 items then delete first | 2.8303 us/op | 2.2854 us/op | 1.24 |
| OrderedSet add up to 64 items then delete first | 4.7819 us/op | 3.5864 us/op | 1.33 |
| Set add up to 64 items then delete last | 3.6720 us/op | 2.3249 us/op | 1.58 |
| OrderedSet add up to 64 items then delete last | 5.2464 us/op | 4.0544 us/op | 1.29 |
| Set add up to 64 items then delete middle | 2.4969 us/op | 2.6552 us/op | 0.94 |
| OrderedSet add up to 64 items then delete middle | 6.7994 us/op | 5.4663 us/op | 1.24 |
| Set add up to 128 items then delete first | 6.6545 us/op | 5.2948 us/op | 1.26 |
| OrderedSet add up to 128 items then delete first | 10.969 us/op | 7.7888 us/op | 1.41 |
| Set add up to 128 items then delete last | 6.6574 us/op | 5.3870 us/op | 1.24 |
| OrderedSet add up to 128 items then delete last | 9.6433 us/op | 8.4400 us/op | 1.14 |
| Set add up to 128 items then delete middle | 6.4267 us/op | 5.1003 us/op | 1.26 |
| OrderedSet add up to 128 items then delete middle | 17.883 us/op | 14.378 us/op | 1.24 |
| Set add up to 256 items then delete first | 13.504 us/op | 11.264 us/op | 1.20 |
| OrderedSet add up to 256 items then delete first | 21.743 us/op | 15.893 us/op | 1.37 |
| Set add up to 256 items then delete last | 14.180 us/op | 10.405 us/op | 1.36 |
| OrderedSet add up to 256 items then delete last | 20.267 us/op | 16.196 us/op | 1.25 |
| Set add up to 256 items then delete middle | 12.807 us/op | 11.122 us/op | 1.15 |
| OrderedSet add up to 256 items then delete middle | 46.712 us/op | 43.646 us/op | 1.07 |
| transfer serialized Status (84 B) | 2.4400 us/op | 2.1210 us/op | 1.15 |
| copy serialized Status (84 B) | 1.2680 us/op | 1.1040 us/op | 1.15 |
| transfer serialized SignedVoluntaryExit (112 B) | 2.3520 us/op | 2.1970 us/op | 1.07 |
| copy serialized SignedVoluntaryExit (112 B) | 1.2810 us/op | 1.1320 us/op | 1.13 |
| transfer serialized ProposerSlashing (416 B) | 2.3960 us/op | 2.2450 us/op | 1.07 |
| copy serialized ProposerSlashing (416 B) | 1.4490 us/op | 1.4860 us/op | 0.98 |
| transfer serialized Attestation (485 B) | 2.4600 us/op | 2.5230 us/op | 0.98 |
| copy serialized Attestation (485 B) | 1.8530 us/op | 1.2430 us/op | 1.49 |
| transfer serialized AttesterSlashing (33232 B) | 3.1610 us/op | 2.3620 us/op | 1.34 |
| copy serialized AttesterSlashing (33232 B) | 5.1600 us/op | 3.7170 us/op | 1.39 |
| transfer serialized Small SignedBeaconBlock (128000 B) | 4.0120 us/op | 3.3490 us/op | 1.20 |
| copy serialized Small SignedBeaconBlock (128000 B) | 14.461 us/op | 10.107 us/op | 1.43 |
| transfer serialized Avg SignedBeaconBlock (200000 B) | 4.2850 us/op | 3.9140 us/op | 1.09 |
| copy serialized Avg SignedBeaconBlock (200000 B) | 21.032 us/op | 13.305 us/op | 1.58 |
| transfer serialized BlobsSidecar (524380 B) | 5.8600 us/op | 3.3280 us/op | 1.76 |
| copy serialized BlobsSidecar (524380 B) | 68.524 us/op | 61.754 us/op | 1.11 |
| transfer serialized Big SignedBeaconBlock (1000000 B) | 4.8710 us/op | 3.3740 us/op | 1.44 |
| copy serialized Big SignedBeaconBlock (1000000 B) | 172.52 us/op | 118.95 us/op | 1.45 |
| pass gossip attestations to forkchoice per slot | 3.0134 ms/op | 2.7629 ms/op | 1.09 |
| forkChoice updateHead vc 100000 bc 64 eq 0 | 463.25 us/op | 447.62 us/op | 1.03 |
| forkChoice updateHead vc 600000 bc 64 eq 0 | 3.3315 ms/op | 2.9439 ms/op | 1.13 |
| forkChoice updateHead vc 1000000 bc 64 eq 0 | 5.8811 ms/op | 4.8902 ms/op | 1.20 |
| forkChoice updateHead vc 600000 bc 320 eq 0 | 2.9183 ms/op | 2.8399 ms/op | 1.03 |
| forkChoice updateHead vc 600000 bc 1200 eq 0 | 2.9687 ms/op | 2.8695 ms/op | 1.03 |
| forkChoice updateHead vc 600000 bc 7200 eq 0 | 3.1247 ms/op | 3.1057 ms/op | 1.01 |
| forkChoice updateHead vc 600000 bc 64 eq 1000 | 10.600 ms/op | 10.482 ms/op | 1.01 |
| forkChoice updateHead vc 600000 bc 64 eq 10000 | 10.464 ms/op | 10.406 ms/op | 1.01 |
| forkChoice updateHead vc 600000 bc 64 eq 300000 | 13.472 ms/op | 13.579 ms/op | 0.99 |
| computeDeltas 500000 validators 300 proto nodes | 3.9544 ms/op | 3.9672 ms/op | 1.00 |
| computeDeltas 500000 validators 1200 proto nodes | 3.8926 ms/op | 3.8804 ms/op | 1.00 |
| computeDeltas 500000 validators 7200 proto nodes | 3.9738 ms/op | 3.9433 ms/op | 1.01 |
| computeDeltas 750000 validators 300 proto nodes | 6.2101 ms/op | 5.7793 ms/op | 1.07 |
| computeDeltas 750000 validators 1200 proto nodes | 6.0897 ms/op | 5.9633 ms/op | 1.02 |
| computeDeltas 750000 validators 7200 proto nodes | 6.1223 ms/op | 6.0310 ms/op | 1.02 |
| computeDeltas 1400000 validators 300 proto nodes | 11.381 ms/op | 11.450 ms/op | 0.99 |
| computeDeltas 1400000 validators 1200 proto nodes | 11.210 ms/op | 11.426 ms/op | 0.98 |
| computeDeltas 1400000 validators 7200 proto nodes | 11.616 ms/op | 11.273 ms/op | 1.03 |
| computeDeltas 2100000 validators 300 proto nodes | 18.589 ms/op | 17.075 ms/op | 1.09 |
| computeDeltas 2100000 validators 1200 proto nodes | 19.936 ms/op | 17.272 ms/op | 1.15 |
| computeDeltas 2100000 validators 7200 proto nodes | 21.471 ms/op | 17.521 ms/op | 1.23 |
| altair processAttestation - 250000 vs - 7PWei normalcase | 3.6824 ms/op | 2.7368 ms/op | 1.35 |
| altair processAttestation - 250000 vs - 7PWei worstcase | 4.7010 ms/op | 3.8768 ms/op | 1.21 |
| altair processAttestation - setStatus - 1/6 committees join | 173.10 us/op | 146.27 us/op | 1.18 |
| altair processAttestation - setStatus - 1/3 committees join | 304.27 us/op | 264.06 us/op | 1.15 |
| altair processAttestation - setStatus - 1/2 committees join | 423.56 us/op | 356.42 us/op | 1.19 |
| altair processAttestation - setStatus - 2/3 committees join | 588.68 us/op | 471.32 us/op | 1.25 |
| altair processAttestation - setStatus - 4/5 committees join | 786.15 us/op | 713.21 us/op | 1.10 |
| altair processAttestation - setStatus - 100% committees join | 898.68 us/op | 917.35 us/op | 0.98 |
| altair processBlock - 250000 vs - 7PWei normalcase | 7.9928 ms/op | 5.2301 ms/op | 1.53 |
| altair processBlock - 250000 vs - 7PWei normalcase hashState | 54.101 ms/op | 33.161 ms/op | 1.63 |
| altair processBlock - 250000 vs - 7PWei worstcase | 55.650 ms/op | 43.013 ms/op | 1.29 |
| altair processBlock - 250000 vs - 7PWei worstcase hashState | 112.33 ms/op | 89.451 ms/op | 1.26 |
| phase0 processBlock - 250000 vs - 7PWei normalcase | 2.4768 ms/op | 2.8182 ms/op | 0.88 |
| phase0 processBlock - 250000 vs - 7PWei worstcase | 33.061 ms/op | 28.500 ms/op | 1.16 |
| altair processEth1Data - 250000 vs - 7PWei normalcase | 438.25 us/op | 390.37 us/op | 1.12 |
| getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 | 10.606 us/op | 11.440 us/op | 0.93 |
| getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 | 51.759 us/op | 56.251 us/op | 0.92 |
| getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42 | 14.124 us/op | 12.827 us/op | 1.10 |
| getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18 | 8.8430 us/op | 7.9240 us/op | 1.12 |
| getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020 | 169.33 us/op | 220.89 us/op | 0.77 |
| getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777 | 2.0277 ms/op | 1.9165 ms/op | 1.06 |
| getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 | 3.1186 ms/op | 2.5983 ms/op | 1.20 |
| getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 | 2.9473 ms/op | 2.4515 ms/op | 1.20 |
| getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 | 7.2836 ms/op | 7.1642 ms/op | 1.02 |
| getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 | 3.1004 ms/op | 2.7712 ms/op | 1.12 |
| getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 | 7.1344 ms/op | 6.3064 ms/op | 1.13 |
| Tree 40 250000 create | 895.13 ms/op | 609.61 ms/op | 1.47 |
| Tree 40 250000 get(125000) | 164.58 ns/op | 146.31 ns/op | 1.12 |
| Tree 40 250000 set(125000) | 2.4931 us/op | 1.5588 us/op | 1.60 |
| Tree 40 250000 toArray() | 25.549 ms/op | 21.634 ms/op | 1.18 |
| Tree 40 250000 iterate all - toArray() + loop | 27.404 ms/op | 21.936 ms/op | 1.25 |
| Tree 40 250000 iterate all - get(i) | 69.003 ms/op | 58.881 ms/op | 1.17 |
| Array 250000 create | 5.3094 ms/op | 2.6674 ms/op | 1.99 |
| Array 250000 clone - spread | 3.9646 ms/op | 1.5023 ms/op | 2.64 |
| Array 250000 get(125000) | 0.46400 ns/op | 0.41900 ns/op | 1.11 |
| Array 250000 set(125000) | 0.48700 ns/op | 0.43700 ns/op | 1.11 |
| Array 250000 iterate all - loop | 94.883 us/op | 85.960 us/op | 1.10 |
| phase0 afterProcessEpoch - 250000 vs - 7PWei | 47.061 ms/op | 44.335 ms/op | 1.06 |
| Array.fill - length 1000000 | 5.4693 ms/op | 3.8592 ms/op | 1.42 |
| Array push - length 1000000 | 34.150 ms/op | 27.480 ms/op | 1.24 |
| Array.get | 0.31187 ns/op | 0.28986 ns/op | 1.08 |
| Uint8Array.get | 0.48092 ns/op | 0.46646 ns/op | 1.03 |
| phase0 beforeProcessEpoch - 250000 vs - 7PWei | 27.356 ms/op | 19.866 ms/op | 1.38 |
| altair processEpoch - mainnet_e81889 | 310.63 ms/op | 314.90 ms/op | 0.99 |
| mainnet_e81889 - altair beforeProcessEpoch | 20.577 ms/op | 18.561 ms/op | 1.11 |
| mainnet_e81889 - altair processJustificationAndFinalization | 6.6190 us/op | 7.0570 us/op | 0.94 |
| mainnet_e81889 - altair processInactivityUpdates | 6.4943 ms/op | 5.7061 ms/op | 1.14 |
| mainnet_e81889 - altair processRewardsAndPenalties | 45.107 ms/op | 44.908 ms/op | 1.00 |
| mainnet_e81889 - altair processRegistryUpdates | 826.00 ns/op | 796.00 ns/op | 1.04 |
| mainnet_e81889 - altair processSlashings | 190.00 ns/op | 201.00 ns/op | 0.95 |
| mainnet_e81889 - altair processEth1DataReset | 185.00 ns/op | 190.00 ns/op | 0.97 |
| mainnet_e81889 - altair processEffectiveBalanceUpdates | 1.3843 ms/op | 1.2241 ms/op | 1.13 |
| mainnet_e81889 - altair processSlashingsReset | 1.4210 us/op | 1.3540 us/op | 1.05 |
| mainnet_e81889 - altair processRandaoMixesReset | 1.7990 us/op | 1.2560 us/op | 1.43 |
| mainnet_e81889 - altair processHistoricalRootsUpdate | 227.00 ns/op | 201.00 ns/op | 1.13 |
| mainnet_e81889 - altair processParticipationFlagUpdates | 573.00 ns/op | 568.00 ns/op | 1.01 |
| mainnet_e81889 - altair processSyncCommitteeUpdates | 151.00 ns/op | 149.00 ns/op | 1.01 |
| mainnet_e81889 - altair afterProcessEpoch | 45.931 ms/op | 47.121 ms/op | 0.97 |
| capella processEpoch - mainnet_e217614 | 1.1121 s/op | 999.29 ms/op | 1.11 |
| mainnet_e217614 - capella beforeProcessEpoch | 69.163 ms/op | 64.305 ms/op | 1.08 |
| mainnet_e217614 - capella processJustificationAndFinalization | 6.2410 us/op | 6.1970 us/op | 1.01 |
| mainnet_e217614 - capella processInactivityUpdates | 15.583 ms/op | 15.671 ms/op | 0.99 |
| mainnet_e217614 - capella processRewardsAndPenalties | 201.71 ms/op | 219.64 ms/op | 0.92 |
| mainnet_e217614 - capella processRegistryUpdates | 8.2500 us/op | 8.1080 us/op | 1.02 |
| mainnet_e217614 - capella processSlashings | 204.00 ns/op | 188.00 ns/op | 1.09 |
| mainnet_e217614 - capella processEth1DataReset | 181.00 ns/op | 188.00 ns/op | 0.96 |
| mainnet_e217614 - capella processEffectiveBalanceUpdates | 4.2426 ms/op | 4.2674 ms/op | 0.99 |
| mainnet_e217614 - capella processSlashingsReset | 1.1010 us/op | 1.0320 us/op | 1.07 |
| mainnet_e217614 - capella processRandaoMixesReset | 1.4420 us/op | 1.3410 us/op | 1.08 |
| mainnet_e217614 - capella processHistoricalRootsUpdate | 198.00 ns/op | 177.00 ns/op | 1.12 |
| mainnet_e217614 - capella processParticipationFlagUpdates | 595.00 ns/op | 593.00 ns/op | 1.00 |
| mainnet_e217614 - capella afterProcessEpoch | 119.97 ms/op | 135.62 ms/op | 0.88 |
| phase0 processEpoch - mainnet_e58758 | 290.87 ms/op | 331.31 ms/op | 0.88 |
| mainnet_e58758 - phase0 beforeProcessEpoch | 79.966 ms/op | 83.492 ms/op | 0.96 |
| mainnet_e58758 - phase0 processJustificationAndFinalization | 7.0170 us/op | 6.4760 us/op | 1.08 |
| mainnet_e58758 - phase0 processRewardsAndPenalties | 47.013 ms/op | 39.623 ms/op | 1.19 |
| mainnet_e58758 - phase0 processRegistryUpdates | 3.4130 us/op | 3.5840 us/op | 0.95 |
| mainnet_e58758 - phase0 processSlashings | 195.00 ns/op | 200.00 ns/op | 0.97 |
| mainnet_e58758 - phase0 processEth1DataReset | 178.00 ns/op | 182.00 ns/op | 0.98 |
| mainnet_e58758 - phase0 processEffectiveBalanceUpdates | 1.1610 ms/op | 1.2533 ms/op | 0.93 |
| mainnet_e58758 - phase0 processSlashingsReset | 972.00 ns/op | 1.0440 us/op | 0.93 |
| mainnet_e58758 - phase0 processRandaoMixesReset | 1.5000 us/op | 1.3240 us/op | 1.13 |
| mainnet_e58758 - phase0 processHistoricalRootsUpdate | 179.00 ns/op | 196.00 ns/op | 0.91 |
| mainnet_e58758 - phase0 processParticipationRecordUpdates | 925.00 ns/op | 988.00 ns/op | 0.94 |
| mainnet_e58758 - phase0 afterProcessEpoch | 38.083 ms/op | 36.471 ms/op | 1.04 |
| phase0 processEffectiveBalanceUpdates - 250000 normalcase | 2.5629 ms/op | 1.4048 ms/op | 1.82 |
| phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 | 2.0903 ms/op | 2.5997 ms/op | 0.80 |
| altair processInactivityUpdates - 250000 normalcase | 22.031 ms/op | 17.740 ms/op | 1.24 |
| altair processInactivityUpdates - 250000 worstcase | 25.134 ms/op | 19.916 ms/op | 1.26 |
| phase0 processRegistryUpdates - 250000 normalcase | 9.5850 us/op | 7.3080 us/op | 1.31 |
| phase0 processRegistryUpdates - 250000 badcase_full_deposits | 335.68 us/op | 308.35 us/op | 1.09 |
| phase0 processRegistryUpdates - 250000 worstcase 0.5 | 112.05 ms/op | 104.79 ms/op | 1.07 |
| altair processRewardsAndPenalties - 250000 normalcase | 32.216 ms/op | 31.108 ms/op | 1.04 |
| altair processRewardsAndPenalties - 250000 worstcase | 31.860 ms/op | 28.623 ms/op | 1.11 |
| phase0 getAttestationDeltas - 250000 normalcase | 6.1624 ms/op | 8.6633 ms/op | 0.71 |
| phase0 getAttestationDeltas - 250000 worstcase | 6.5187 ms/op | 6.9070 ms/op | 0.94 |
| phase0 processSlashings - 250000 worstcase | 114.83 us/op | 110.13 us/op | 1.04 |
| altair processSyncCommitteeUpdates - 250000 | 12.178 ms/op | 11.429 ms/op | 1.07 |
| BeaconState.hashTreeRoot - No change | 237.00 ns/op | 229.00 ns/op | 1.03 |
| BeaconState.hashTreeRoot - 1 full validator | 93.375 us/op | 87.314 us/op | 1.07 |
| BeaconState.hashTreeRoot - 32 full validator | 1.2021 ms/op | 814.36 us/op | 1.48 |
| BeaconState.hashTreeRoot - 512 full validator | 12.367 ms/op | 10.546 ms/op | 1.17 |
| BeaconState.hashTreeRoot - 1 validator.effectiveBalance | 112.22 us/op | 90.472 us/op | 1.24 |
| BeaconState.hashTreeRoot - 32 validator.effectiveBalance | 1.8869 ms/op | 1.4503 ms/op | 1.30 |
| BeaconState.hashTreeRoot - 512 validator.effectiveBalance | 26.092 ms/op | 24.874 ms/op | 1.05 |
| BeaconState.hashTreeRoot - 1 balances | 71.537 us/op | 84.255 us/op | 0.85 |
| BeaconState.hashTreeRoot - 32 balances | 751.79 us/op | 700.66 us/op | 1.07 |
| BeaconState.hashTreeRoot - 512 balances | 8.3489 ms/op | 8.3431 ms/op | 1.00 |
| BeaconState.hashTreeRoot - 250000 balances | 163.49 ms/op | 182.32 ms/op | 0.90 |
| aggregationBits - 2048 els - zipIndexesInBitList | 23.217 us/op | 26.820 us/op | 0.87 |
| byteArrayEquals 32 | 54.782 ns/op | 54.587 ns/op | 1.00 |
| Buffer.compare 32 | 17.304 ns/op | 17.672 ns/op | 0.98 |
| byteArrayEquals 1024 | 1.6114 us/op | 1.6165 us/op | 1.00 |
| Buffer.compare 1024 | 24.952 ns/op | 25.134 ns/op | 0.99 |
| byteArrayEquals 16384 | 25.580 us/op | 25.885 us/op | 0.99 |
| Buffer.compare 16384 | 185.27 ns/op | 178.93 ns/op | 1.04 |
| byteArrayEquals 123687377 | 208.51 ms/op | 195.42 ms/op | 1.07 |
| Buffer.compare 123687377 | 12.621 ms/op | 9.6036 ms/op | 1.31 |
| byteArrayEquals 32 - diff last byte | 56.138 ns/op | 55.000 ns/op | 1.02 |
| Buffer.compare 32 - diff last byte | 18.206 ns/op | 17.856 ns/op | 1.02 |
| byteArrayEquals 1024 - diff last byte | 1.6848 us/op | 1.6524 us/op | 1.02 |
| Buffer.compare 1024 - diff last byte | 26.190 ns/op | 27.686 ns/op | 0.95 |
| byteArrayEquals 16384 - diff last byte | 26.780 us/op | 26.575 us/op | 1.01 |
| Buffer.compare 16384 - diff last byte | 217.81 ns/op | 202.96 ns/op | 1.07 |
| byteArrayEquals 123687377 - diff last byte | 196.19 ms/op | 199.74 ms/op | 0.98 |
| Buffer.compare 123687377 - diff last byte | 7.8250 ms/op | 8.1339 ms/op | 0.96 |
| byteArrayEquals 32 - random bytes | 5.0900 ns/op | 5.2700 ns/op | 0.97 |
| Buffer.compare 32 - random bytes | 16.998 ns/op | 17.847 ns/op | 0.95 |
| byteArrayEquals 1024 - random bytes | 5.1280 ns/op | 5.2910 ns/op | 0.97 |
| Buffer.compare 1024 - random bytes | 17.134 ns/op | 18.200 ns/op | 0.94 |
| byteArrayEquals 16384 - random bytes | 5.1350 ns/op | 5.3740 ns/op | 0.96 |
| Buffer.compare 16384 - random bytes | 17.050 ns/op | 18.013 ns/op | 0.95 |
| byteArrayEquals 123687377 - random bytes | 6.9000 ns/op | 6.6400 ns/op | 1.04 |
| Buffer.compare 123687377 - random bytes | 21.200 ns/op | 20.140 ns/op | 1.05 |
| regular array get 100000 times | 45.687 us/op | 36.109 us/op | 1.27 |
| wrappedArray get 100000 times | 34.327 us/op | 36.313 us/op | 0.95 |
| arrayWithProxy get 100000 times | 12.507 ms/op | 23.047 ms/op | 0.54 |
| ssz.Root.equals | 47.841 ns/op | 49.052 ns/op | 0.98 |
| byteArrayEquals | 45.839 ns/op | 47.959 ns/op | 0.96 |
| Buffer.compare | 10.367 ns/op | 11.163 ns/op | 0.93 |
| processSlot - 1 slots | 10.979 us/op | 11.273 us/op | 0.97 |
| processSlot - 32 slots | 1.9907 ms/op | 3.5116 ms/op | 0.57 |
| getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei | 3.3410 ms/op | 3.5165 ms/op | 0.95 |
| getCommitteeAssignments - req 1 vs - 250000 vc | 2.1229 ms/op | 2.1773 ms/op | 0.98 |
| getCommitteeAssignments - req 100 vs - 250000 vc | 4.2156 ms/op | 4.2517 ms/op | 0.99 |
| getCommitteeAssignments - req 1000 vs - 250000 vc | 4.4137 ms/op | 4.5598 ms/op | 0.97 |
| findModifiedValidators - 10000 modified validators | 791.20 ms/op | 737.01 ms/op | 1.07 |
| findModifiedValidators - 1000 modified validators | 821.55 ms/op | 692.28 ms/op | 1.19 |
| findModifiedValidators - 100 modified validators | 347.87 ms/op | 276.94 ms/op | 1.26 |
| findModifiedValidators - 10 modified validators | 255.04 ms/op | 191.14 ms/op | 1.33 |
| findModifiedValidators - 1 modified validators | 207.99 ms/op | 189.96 ms/op | 1.09 |
| findModifiedValidators - no difference | 238.07 ms/op | 143.11 ms/op | 1.66 |
| compare ViewDUs | 7.3750 s/op | 6.4049 s/op | 1.15 |
| compare each validator Uint8Array | 1.8079 s/op | 1.2383 s/op | 1.46 |
| compare ViewDU to Uint8Array | 1.1946 s/op | 1.1344 s/op | 1.05 |
| migrate state 1000000 validators, 24 modified, 0 new | 908.65 ms/op | 842.38 ms/op | 1.08 |
| migrate state 1000000 validators, 1700 modified, 1000 new | 1.2319 s/op | 1.1692 s/op | 1.05 |
| migrate state 1000000 validators, 3400 modified, 2000 new | 1.4076 s/op | 1.3319 s/op | 1.06 |
| migrate state 1500000 validators, 24 modified, 0 new | 976.25 ms/op | 905.07 ms/op | 1.08 |
| migrate state 1500000 validators, 1700 modified, 1000 new | 1.2259 s/op | 1.2174 s/op | 1.01 |
| migrate state 1500000 validators, 3400 modified, 2000 new | 1.4158 s/op | 1.3670 s/op | 1.04 |
| RootCache.getBlockRootAtSlot - 250000 vs - 7PWei | 6.0100 ns/op | 4.3200 ns/op | 1.39 |
| state getBlockRootAtSlot - 250000 vs - 7PWei | 395.01 ns/op | 426.65 ns/op | 0.93 |
| naive computeProposerIndex 100000 validators | 58.558 ms/op | 56.856 ms/op | 1.03 |
| computeProposerIndex 100000 validators | 1.5002 ms/op | 1.5564 ms/op | 0.96 |
| naiveGetNextSyncCommitteeIndices 1000 validators | 8.0234 s/op | 8.3868 s/op | 0.96 |
| getNextSyncCommitteeIndices 1000 validators | 112.18 ms/op | 112.36 ms/op | 1.00 |
| naiveGetNextSyncCommitteeIndices 10000 validators | 8.5949 s/op | 7.7771 s/op | 1.11 |
| getNextSyncCommitteeIndices 10000 validators | 111.50 ms/op | 113.10 ms/op | 0.99 |
| naiveGetNextSyncCommitteeIndices 100000 validators | 8.3604 s/op | 7.2142 s/op | 1.16 |
| getNextSyncCommitteeIndices 100000 validators | 112.85 ms/op | 114.01 ms/op | 0.99 |
| naive computeShuffledIndex 100000 validators | 25.714 s/op | 23.028 s/op | 1.12 |
| cached computeShuffledIndex 100000 validators | 575.00 ms/op | 536.62 ms/op | 1.07 |
| naive computeShuffledIndex 2000000 validators | 632.51 s/op | 470.70 s/op | 1.34 |
| cached computeShuffledIndex 2000000 validators | 66.447 s/op | 28.731 s/op | 2.31 |
| computeProposers - vc 250000 | 651.93 us/op | 619.36 us/op | 1.05 |
| computeEpochShuffling - vc 250000 | 43.385 ms/op | 41.651 ms/op | 1.04 |
| getNextSyncCommittee - vc 250000 | 11.877 ms/op | 10.328 ms/op | 1.15 |
| computeSigningRoot for AttestationData | 34.462 us/op | 20.769 us/op | 1.66 |
| hash AttestationData serialized data then Buffer.toString(base64) | 1.7627 us/op | 1.5242 us/op | 1.16 |
| toHexString serialized data | 1.5185 us/op | 1.0975 us/op | 1.38 |
| Buffer.toString(base64) | 179.05 ns/op | 156.42 ns/op | 1.14 |
| nodejs block root to RootHex using toHex | 150.57 ns/op | 148.39 ns/op | 1.01 |
| nodejs block root to RootHex using toRootHex | 98.418 ns/op | 86.410 ns/op | 1.14 |
| nodejs fromhex(blob) | 128.66 ms/op | 106.70 ms/op | 1.21 |
| nodejs fromHexInto(blob) | 104.20 ms/op | 93.006 ms/op | 1.12 |
| browser block root to RootHex using the deprecated toHexString | 237.26 ns/op | 205.58 ns/op | 1.15 |
| browser block root to RootHex using toHex | 191.02 ns/op | 167.44 ns/op | 1.14 |
| browser block root to RootHex using toRootHex | 166.39 ns/op | 157.05 ns/op | 1.06 |
| browser fromHexInto(blob) | 850.61 us/op | 801.51 us/op | 1.06 |
| browser fromHex(blob) | 818.85 ms/op | 767.53 ms/op | 1.07 |
by benchmarkbot/action
matthewkeil
previously approved these changes
Sep 9, 2025
Member
matthewkeil
left a comment
There was a problem hiding this comment.
Just a small nit on the logging. Up to you if you want to use your or the suggestion though. Both are fine
packages/beacon-node/src/chain/archiveStore/utils/archiveBlocks.ts
Outdated
Show resolved
Hide resolved
nflaig
previously approved these changes
Sep 9, 2025
Member
nflaig
left a comment
There was a problem hiding this comment.
I noticed this as well when debugging last time, thought #8345 might already deal with it but I think we need to deduplicate as you did in this PR cc @nazarhussain
nflaig
reviewed
Sep 9, 2025
packages/beacon-node/src/chain/archiveStore/utils/archiveBlocks.ts
Outdated
Show resolved
Hide resolved
nazarhussain
reviewed
Sep 9, 2025
| ).map((p) => p.prefix); | ||
|
|
||
| const prefixedKeys = await db.dataColumnSidecarArchive.keys({ | ||
| lt: db.dataColumnSidecarArchive.getMaxKeyRaw(computeStartSlotAtEpoch(dataColumnSidecarsMinEpoch)), |
Contributor
There was a problem hiding this comment.
The syntax is changed in #8345, may be we include this logic in that PR.
Contributor
Author
There was a problem hiding this comment.
@nazarhussain feel free to include this in your PR if you're close
nflaig
approved these changes
Sep 9, 2025
wemeetagain
approved these changes
Sep 9, 2025
Member
|
🎉 This PR is included in v1.35.0 🎉 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
archiveBlocks.ts: duplicate slots when delete DataColumnSidecarsDescription