Scalable-friendly dct-inl.h #4029
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mo271
previously approved these changes
Dec 23, 2024
Contributor
Author
|
Lets split this into smaller parts. And add DCT bench. |
Contributor
Author
|
@mo271 ready for re-review |
Contributor
Author
|
(no changes, just now tested to be performance neutral) |
mo271
approved these changes
Jan 8, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Seems to be performance / binary size neutral on non-scalable arch.
BEFORE
Benchmark Time CPU Iterations items_per_second
DCT2x2/AVX2 8.65 ns 8.64 ns 163279224 462.812M/s
DCT2x2/SSE4 9.51 ns 9.51 ns 145840086 420.697M/s
DCT2x2/SSE2 10.2 ns 10.2 ns 137965038 393.798M/s
IDCT2x2/AVX2 12.4 ns 12.4 ns 110683707 322.803M/s
IDCT2x2/SSE4 12.5 ns 12.5 ns 111257180 320.209M/s
IDCT2x2/SSE2 12.5 ns 12.5 ns 110645291 320.963M/s
DCT4x4/AVX2 21.1 ns 21.1 ns 66738583 757.589M/s
DCT4x4/SSE4 17.7 ns 17.7 ns 77164856 904.717M/s
DCT4x4/SSE2 17.6 ns 17.6 ns 76568602 907.967M/s
IDCT4x4/AVX2 15.8 ns 15.8 ns 85941537 1.01386G/s
IDCT4x4/SSE4 13.8 ns 13.8 ns 102441823 1.16064G/s
IDCT4x4/SSE2 13.8 ns 13.8 ns 102838695 1.15965G/s
DCT8x8/AVX2 33.6 ns 33.6 ns 41406294 1.90603G/s
DCT8x8/SSE4 72.1 ns 72.1 ns 19135442 888.269M/s
DCT8x8/SSE2 71.4 ns 71.4 ns 19692444 896.711M/s
IDCT8x8/AVX2 30.3 ns 30.2 ns 46697467 2.1168G/s
IDCT8x8/SSE4 57.2 ns 57.2 ns 24798191 1.11977G/s
IDCT8x8/SSE2 57.1 ns 57.0 ns 24581133 1.12239G/s
DCT16x16/AVX2 251 ns 251 ns 5503114 1.02073G/s
DCT16x16/SSE4 429 ns 429 ns 3216253 597.384M/s
DCT16x16/SSE2 429 ns 429 ns 3317040 597.318M/s
IDCT16x16/AVX2 185 ns 185 ns 7540520 1.38262G/s
IDCT16x16/SSE4 286 ns 285 ns 4878036 897.19M/s
IDCT16x16/SSE2 287 ns 287 ns 4884563 891.697M/s
DCT32x32/AVX2 1597 ns 1594 ns 879096 642.273M/s
DCT32x32/SSE4 2706 ns 2703 ns 520515 378.867M/s
DCT32x32/SSE2 2712 ns 2708 ns 534026 378.101M/s
IDCT32x32/AVX2 1329 ns 1327 ns 1006754 771.389M/s
IDCT32x32/SSE4 2189 ns 2186 ns 620004 468.361M/s
IDCT32x32/SSE2 2190 ns 2188 ns 644304 468.112M/s
DCT16x8/AVX2 92.7 ns 92.6 ns 15146240 1.38204G/s
DCT16x8/SSE4 176 ns 176 ns 7975487 728.783M/s
DCT16x8/SSE2 175 ns 175 ns 7963358 732.994M/s
IDCT16x8/AVX2 85.4 ns 85.3 ns 16543903 1.50125G/s
IDCT16x8/SSE4 125 ns 125 ns 10957725 1.02589G/s
IDCT16x8/SSE2 127 ns 127 ns 11240331 1.01176G/s
DCT8x16/AVX2 101 ns 101 ns 14086122 1.26753G/s
DCT8x16/SSE4 191 ns 191 ns 7308541 671.081M/s
DCT8x16/SSE2 192 ns 192 ns 7444663 667.769M/s
IDCT8x16/AVX2 81.2 ns 81.1 ns 17373958 1.57826G/s
IDCT8x16/SSE4 145 ns 144 ns 9800821 886.19M/s
IDCT8x16/SSE2 144 ns 144 ns 9538732 891.066M/s
DCT32x8/AVX2 247 ns 247 ns 5660226 1.03652G/s
DCT32x8/SSE4 463 ns 463 ns 3030028 553.096M/s
DCT32x8/SSE2 469 ns 468 ns 3042642 546.599M/s
IDCT32x8/AVX2 239 ns 239 ns 5804120 1.07145G/s
IDCT32x8/SSE4 388 ns 387 ns 3622023 661.343M/s
IDCT32x8/SSE2 387 ns 386 ns 3658984 662.731M/s
DCT8x32/AVX2 236 ns 236 ns 5880316 1.08591G/s
DCT8x32/SSE4 479 ns 478 ns 2932801 535.116M/s
DCT8x32/SSE2 478 ns 477 ns 2930687 536.394M/s
IDCT8x32/AVX2 207 ns 207 ns 6705385 1.23679G/s
IDCT8x32/SSE4 355 ns 354 ns 3988644 722.562M/s
IDCT8x32/SSE2 355 ns 354 ns 3979307 722.511M/s
DCT32x16/AVX2 686 ns 685 ns 2006662 747.441M/s
DCT32x16/SSE4 1193 ns 1191 ns 1198642 429.824M/s
DCT32x16/SSE2 1195 ns 1193 ns 1172367 429.058M/s
IDCT32x16/AVX2 586 ns 585 ns 2409013 874.825M/s
IDCT32x16/SSE4 953 ns 951 ns 1486993 538.154M/s
IDCT32x16/SSE2 962 ns 960 ns 1463629 533.128M/s
DCT16x32/AVX2 766 ns 765 ns 1846340 669.26M/s
DCT16x32/SSE4 1311 ns 1309 ns 1093630 391.011M/s
DCT16x32/SSE2 1315 ns 1313 ns 1044802 389.971M/s
IDCT16x32/AVX2 619 ns 618 ns 2215023 828.191M/s
IDCT16x32/SSE4 933 ns 932 ns 1531055 549.078M/s
IDCT16x32/SSE2 933 ns 932 ns 1499626 549.107M/s
DCT4x8/AVX2 45.9 ns 45.9 ns 30144845 697.83M/s
DCT4x8/SSE4 40.3 ns 40.3 ns 34831895 794.898M/s
DCT4x8/SSE2 40.7 ns 40.7 ns 34353124 786.386M/s
IDCT4x8/AVX2 46.8 ns 46.7 ns 29317221 685.318M/s
IDCT4x8/SSE4 33.6 ns 33.5 ns 42013440 954.565M/s
IDCT4x8/SSE2 33.9 ns 33.9 ns 40418266 944.58M/s
DCT8x4/AVX2 35.7 ns 35.7 ns 39501023 897.193M/s
DCT8x4/SSE4 36.5 ns 36.4 ns 38230430 878.234M/s
DCT8x4/SSE2 36.0 ns 35.9 ns 38879900 890.399M/s
IDCT8x4/AVX2 35.1 ns 35.1 ns 40499391 912.351M/s
IDCT8x4/SSE4 27.4 ns 27.4 ns 51002009 1.16797G/s
IDCT8x4/SSE2 27.8 ns 27.7 ns 51330769 1.15331G/s
DCT64x64/AVX2 10342 ns 10327 ns 134243 396.634M/s
DCT64x64/SSE4 17605 ns 17550 ns 81165 233.394M/s
DCT64x64/SSE2 17531 ns 17510 ns 77906 233.927M/s
IDCT64x64/AVX2 8169 ns 8158 ns 179346 502.061M/s
IDCT64x64/SSE4 13014 ns 12997 ns 104667 315.146M/s
IDCT64x64/SSE2 13246 ns 13237 ns 104642 309.437M/s
DCT64x32/AVX2 3980 ns 3975 ns 346311 515.2M/s
DCT64x32/SSE4 6832 ns 6825 ns 209739 300.06M/s
DCT64x32/SSE2 6742 ns 6735 ns 206565 304.091M/s
IDCT64x32/AVX2 3285 ns 3281 ns 430401 624.129M/s
IDCT64x32/SSE4 5443 ns 5437 ns 257685 376.648M/s
IDCT64x32/SSE2 5471 ns 5464 ns 260721 374.789M/s
DCT32x64/AVX2 4554 ns 4549 ns 303611 450.209M/s
DCT32x64/SSE4 7410 ns 7401 ns 187870 276.715M/s
DCT32x64/SSE2 7321 ns 7311 ns 185215 280.108M/s
IDCT32x64/AVX2 3438 ns 3434 ns 402263 596.413M/s
IDCT32x64/SSE4 5806 ns 5799 ns 243568 353.189M/s
IDCT32x64/SSE2 5535 ns 5529 ns 247702 370.398M/s
DCT128x128/AVX2 52581 ns 52523 ns 26635 311.939M/s
DCT128x128/SSE4 89495 ns 89396 ns 15754 183.274M/s
DCT128x128/SSE2 88192 ns 88077 ns 15837 186.019M/s
IDCT128x128/AVX2 42846 ns 42792 ns 32862 382.879M/s
IDCT128x128/SSE4 75131 ns 75026 ns 18468 218.378M/s
IDCT128x128/SSE2 72298 ns 72196 ns 19370 226.938M/s
DCT128x64/AVX2 24448 ns 24423 ns 57557 335.428M/s
DCT128x64/SSE4 40396 ns 40354 ns 34940 203.002M/s
DCT128x64/SSE2 40119 ns 40073 ns 34998 204.426M/s
IDCT128x64/AVX2 19297 ns 19278 ns 70987 424.941M/s
IDCT128x64/SSE4 31921 ns 31879 ns 44226 256.969M/s
IDCT128x64/SSE2 30969 ns 30933 ns 46388 264.829M/s
DCT64x128/AVX2 27010 ns 26975 ns 52034 303.694M/s
DCT64x128/SSE4 43920 ns 43863 ns 32057 186.762M/s
DCT64x128/SSE2 44010 ns 43945 ns 31631 186.415M/s
IDCT64x128/AVX2 21371 ns 21337 ns 67204 383.926M/s
IDCT64x128/SSE4 34765 ns 34725 ns 40394 235.908M/s
IDCT64x128/SSE2 33576 ns 33541 ns 42208 244.237M/s
DCT256x256/AVX2 267878 ns 267619 ns 5102 244.886M/s
DCT256x256/SSE4 449082 ns 448548 ns 3033 146.107M/s
DCT256x256/SSE2 452817 ns 452296 ns 3159 144.896M/s
IDCT256x256/AVX2 205160 ns 204869 ns 6955 319.892M/s
IDCT256x256/SSE4 372660 ns 372168 ns 3737 176.093M/s
IDCT256x256/SSE2 352328 ns 351874 ns 3984 186.249M/s
DCT256x128/AVX2 121072 ns 120896 ns 11589 271.043M/s
DCT256x128/SSE4 204078 ns 203837 ns 6845 160.756M/s
DCT256x128/SSE2 204963 ns 204731 ns 6875 160.054M/s
IDCT256x128/AVX2 94859 ns 94758 ns 14648 345.806M/s
IDCT256x128/SSE4 169668 ns 169467 ns 8214 193.359M/s
IDCT256x128/SSE2 160265 ns 160054 ns 8728 204.731M/s
DCT128x256/AVX2 134585 ns 134397 ns 10375 243.816M/s
DCT128x256/SSE4 224767 ns 224507 ns 6204 145.956M/s
DCT128x256/SSE2 221648 ns 221403 ns 6439 148.002M/s
IDCT128x256/AVX2 105524 ns 105386 ns 12815 310.934M/s
IDCT128x256/SSE4 191297 ns 191030 ns 7498 171.533M/s
IDCT128x256/SSE2 181951 ns 181694 ns 7655 180.347M/s
AFTER
Benchmark Time CPU Iterations items_per_second
DCT2x2/AVX2 8.59 ns 8.59 ns 164251242 465.86M/s
DCT2x2/SSE4 9.47 ns 9.46 ns 147903995 422.814M/s
DCT2x2/SSE2 9.93 ns 9.91 ns 139839728 403.444M/s
IDCT2x2/AVX2 12.5 ns 12.5 ns 115275226 320.673M/s
IDCT2x2/SSE4 12.5 ns 12.5 ns 112985592 319.365M/s
IDCT2x2/SSE2 12.7 ns 12.7 ns 109858810 315.43M/s
DCT4x4/AVX2 21.0 ns 21.0 ns 66728622 761.413M/s
DCT4x4/SSE4 17.7 ns 17.6 ns 78292218 907.336M/s
DCT4x4/SSE2 17.8 ns 17.8 ns 79748007 900.258M/s
IDCT4x4/AVX2 15.8 ns 15.8 ns 87666511 1.01383G/s
IDCT4x4/SSE4 13.6 ns 13.6 ns 102289756 1.17789G/s
IDCT4x4/SSE2 13.7 ns 13.6 ns 104189460 1.17378G/s
DCT8x8/AVX2 33.6 ns 33.6 ns 41482517 1.90477G/s
DCT8x8/SSE4 71.5 ns 71.4 ns 19261556 896.424M/s
DCT8x8/SSE2 70.8 ns 70.7 ns 19387606 905.19M/s
IDCT8x8/AVX2 30.4 ns 30.4 ns 46254210 2.106G/s
IDCT8x8/SSE4 57.5 ns 57.4 ns 24326654 1.11414G/s
IDCT8x8/SSE2 57.3 ns 57.2 ns 24300551 1.1186G/s
DCT16x16/AVX2 258 ns 258 ns 5478289 992.233M/s
DCT16x16/SSE4 425 ns 424 ns 3370510 603.364M/s
DCT16x16/SSE2 417 ns 417 ns 3375754 614.263M/s
IDCT16x16/AVX2 193 ns 193 ns 7327712 1.32679G/s
IDCT16x16/SSE4 297 ns 297 ns 4744350 862.386M/s
IDCT16x16/SSE2 299 ns 299 ns 4677072 857.346M/s
DCT32x32/AVX2 1642 ns 1640 ns 883433 624.507M/s
DCT32x32/SSE4 2741 ns 2738 ns 511976 374.024M/s
DCT32x32/SSE2 2714 ns 2711 ns 514434 377.708M/s
IDCT32x32/AVX2 1402 ns 1400 ns 960774 731.278M/s
IDCT32x32/SSE4 2194 ns 2192 ns 634129 467.172M/s
IDCT32x32/SSE2 2226 ns 2223 ns 641868 460.695M/s
DCT16x8/AVX2 92.8 ns 92.7 ns 15358011 1.38074G/s
DCT16x8/SSE4 172 ns 172 ns 8123236 743.05M/s
DCT16x8/SSE2 173 ns 172 ns 8021614 742.314M/s
IDCT16x8/AVX2 83.9 ns 83.8 ns 16347067 1.52753G/s
IDCT16x8/SSE4 124 ns 124 ns 11278587 1.02956G/s
IDCT16x8/SSE2 126 ns 126 ns 11206953 1.01466G/s
DCT8x16/AVX2 101 ns 100 ns 10000000 1.27404G/s
DCT8x16/SSE4 186 ns 185 ns 7495390 690.633M/s
DCT8x16/SSE2 190 ns 190 ns 7423312 675.278M/s
IDCT8x16/AVX2 79.1 ns 79.0 ns 17329479 1.6206G/s
IDCT8x16/SSE4 146 ns 146 ns 9938416 877.18M/s
IDCT8x16/SSE2 145 ns 145 ns 9744149 881.774M/s
DCT32x8/AVX2 246 ns 245 ns 5557763 1.04338G/s
DCT32x8/SSE4 476 ns 475 ns 2991046 538.436M/s
DCT32x8/SSE2 480 ns 480 ns 2952998 533.472M/s
IDCT32x8/AVX2 235 ns 235 ns 6096236 1.09117G/s
IDCT32x8/SSE4 384 ns 384 ns 3703177 666.94M/s
IDCT32x8/SSE2 382 ns 381 ns 3654424 671.74M/s
DCT8x32/AVX2 242 ns 242 ns 5812323 1.05886G/s
DCT8x32/SSE4 482 ns 482 ns 2880256 531.612M/s
DCT8x32/SSE2 487 ns 486 ns 2873019 526.426M/s
IDCT8x32/AVX2 201 ns 201 ns 7014734 1.27428G/s
IDCT8x32/SSE4 356 ns 356 ns 3948104 719.809M/s
IDCT8x32/SSE2 355 ns 355 ns 3902815 721.641M/s
DCT32x16/AVX2 704 ns 703 ns 2035064 728.061M/s
DCT32x16/SSE4 1194 ns 1193 ns 1165196 429.207M/s
DCT32x16/SSE2 1182 ns 1180 ns 1214082 433.954M/s
IDCT32x16/AVX2 614 ns 614 ns 2310798 834.554M/s
IDCT32x16/SSE4 948 ns 947 ns 1487380 540.767M/s
IDCT32x16/SSE2 949 ns 948 ns 1457505 540.24M/s
DCT16x32/AVX2 770 ns 769 ns 1848258 665.664M/s
DCT16x32/SSE4 1323 ns 1322 ns 1066226 387.422M/s
DCT16x32/SSE2 1306 ns 1304 ns 1072984 392.638M/s
IDCT16x32/AVX2 638 ns 637 ns 2226685 804.049M/s
IDCT16x32/SSE4 947 ns 946 ns 1482502 541.237M/s
IDCT16x32/SSE2 945 ns 943 ns 1495840 542.661M/s
DCT4x8/AVX2 43.1 ns 42.9 ns 32280067 745.228M/s
DCT4x8/SSE4 38.4 ns 38.4 ns 37200407 833.745M/s
DCT4x8/SSE2 37.5 ns 37.4 ns 37378109 855.112M/s
IDCT4x8/AVX2 48.3 ns 48.2 ns 28861137 663.582M/s
IDCT4x8/SSE4 30.6 ns 30.6 ns 46810715 1.04569G/s
IDCT4x8/SSE2 30.9 ns 30.9 ns 46389300 1.03605G/s
DCT8x4/AVX2 34.4 ns 34.4 ns 40875532 930.599M/s
DCT8x4/SSE4 34.1 ns 34.0 ns 41862208 940.548M/s
DCT8x4/SSE2 33.9 ns 33.9 ns 41440493 944.866M/s
IDCT8x4/AVX2 34.5 ns 34.4 ns 39615865 929.994M/s
IDCT8x4/SSE4 26.9 ns 26.8 ns 53139049 1.19197G/s
IDCT8x4/SSE2 26.5 ns 26.5 ns 53300362 1.20864G/s
DCT64x64/AVX2 10133 ns 10121 ns 139679 404.685M/s
DCT64x64/SSE4 17209 ns 17188 ns 81806 238.31M/s
DCT64x64/SSE2 16930 ns 16907 ns 82569 242.267M/s
IDCT64x64/AVX2 7681 ns 7673 ns 181321 533.852M/s
IDCT64x64/SSE4 12863 ns 12848 ns 107891 318.815M/s
IDCT64x64/SSE2 12683 ns 12665 ns 108947 323.398M/s
DCT64x32/AVX2 3917 ns 3912 ns 359583 523.482M/s
DCT64x32/SSE4 6826 ns 6817 ns 205904 300.405M/s
DCT64x32/SSE2 6817 ns 6808 ns 204138 300.834M/s
IDCT64x32/AVX2 3234 ns 3230 ns 429329 634.14M/s
IDCT64x32/SSE4 5352 ns 5345 ns 264895 383.153M/s
IDCT64x32/SSE2 5446 ns 5440 ns 262870 376.495M/s
DCT32x64/AVX2 4338 ns 4334 ns 323597 472.591M/s
DCT32x64/SSE4 7464 ns 7458 ns 187096 274.623M/s
DCT32x64/SSE2 7411 ns 7401 ns 188938 276.711M/s
IDCT32x64/AVX2 3492 ns 3488 ns 398068 587.225M/s
IDCT32x64/SSE4 5670 ns 5662 ns 250191 361.705M/s
IDCT32x64/SSE2 5917 ns 5911 ns 240235 346.489M/s
DCT128x128/AVX2 53510 ns 53436 ns 26756 306.608M/s
DCT128x128/SSE4 87871 ns 87769 ns 15865 186.672M/s
DCT128x128/SSE2 88744 ns 88629 ns 15774 184.86M/s
IDCT128x128/AVX2 43245 ns 43193 ns 32040 379.321M/s
IDCT128x128/SSE4 70807 ns 70726 ns 20100 231.655M/s
IDCT128x128/SSE2 74590 ns 74532 ns 19325 219.824M/s
DCT128x64/AVX2 24195 ns 24176 ns 60239 338.852M/s
DCT128x64/SSE4 39186 ns 39156 ns 34789 209.214M/s
DCT128x64/SSE2 39932 ns 39903 ns 34096 205.297M/s
IDCT128x64/AVX2 18724 ns 18707 ns 73141 437.906M/s
IDCT128x64/SSE4 30555 ns 30529 ns 47533 268.339M/s
IDCT128x64/SSE2 31206 ns 31180 ns 45692 262.73M/s
DCT64x128/AVX2 26499 ns 26480 ns 53068 309.364M/s
DCT64x128/SSE4 43422 ns 43387 ns 32347 188.812M/s
DCT64x128/SSE2 43081 ns 43041 ns 31744 190.331M/s
IDCT64x128/AVX2 21479 ns 21456 ns 67583 381.806M/s
IDCT64x128/SSE4 34247 ns 34211 ns 40726 239.452M/s
IDCT64x128/SSE2 35605 ns 35572 ns 39696 230.296M/s
DCT256x256/AVX2 267127 ns 266938 ns 5273 245.51M/s
DCT256x256/SSE4 451592 ns 451095 ns 3139 145.282M/s
DCT256x256/SSE2 448363 ns 447850 ns 3142 146.335M/s
IDCT256x256/AVX2 223140 ns 222869 ns 6201 294.056M/s
IDCT256x256/SSE4 362540 ns 362103 ns 3829 180.987M/s
IDCT256x256/SSE2 386805 ns 386252 ns 3673 169.671M/s
DCT256x128/AVX2 121739 ns 121537 ns 11634 269.614M/s
DCT256x128/SSE4 203322 ns 203121 ns 6893 161.323M/s
DCT256x128/SSE2 204131 ns 203891 ns 6996 160.713M/s
IDCT256x128/AVX2 96137 ns 95990 ns 14062 341.368M/s
IDCT256x128/SSE4 159289 ns 159073 ns 8844 205.994M/s
IDCT256x128/SSE2 169258 ns 169054 ns 8186 193.831M/s
DCT128x256/AVX2 130960 ns 130774 ns 10873 250.57M/s
DCT128x256/SSE4 218805 ns 218523 ns 6434 149.953M/s
DCT128x256/SSE2 224963 ns 224670 ns 5954 145.85M/s
IDCT128x256/AVX2 108395 ns 108225 ns 13065 302.777M/s
IDCT128x256/SSE4 180396 ns 180167 ns 7852 181.875M/s
IDCT128x256/SSE2 192930 ns 192678 ns 7254 170.066M/s