[
https://issues.apache.org/jira/browse/GEODE-9340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17550664#comment-17550664
]
Geode Integration commented on GEODE-9340:
------------------------------------------
Seen on support/1.14 in [benchmark-with-security-manager
#69|https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-support-1-14-main/jobs/benchmark-with-security-manager/builds/69].
> Benchmark instability in PartitionedPutLongBenchmark
> ----------------------------------------------------
>
> Key: GEODE-9340
> URL: https://issues.apache.org/jira/browse/GEODE-9340
> Project: Geode
> Issue Type: Bug
> Components: benchmarks
> Affects Versions: 1.15.0
> Reporter: Sarah Abbey
> Priority: Major
> Labels: pull-request-available
>
> PartitionedPutLongBenchmark failed in CI
> (https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-develop-main/jobs/benchmark-base/builds/6):
> {code:java}
> This is ITERATION 1 of benchmarking against baseline.
> P2pPartitionedGetBenchmark avg ops/sec
> Baseline: 825011.38 Test: 835847.67 Difference: +1.3%
> avg latency
> Baseline: 871392.31 Test: 859444.66 Difference: -1.4%
> P2pPartitionedPutBenchmark avg ops/sec
> Baseline: 123838.43 Test: 122686.30 Difference: -0.9%
> avg latency
> Baseline: 6015719.73 Test: 6119472.19 Difference: +1.7%
> P2pPartitionedPutBytesBenchmark avg ops/sec
> Baseline: 174887.77 Test: 171040.93 Difference: -2.2%
> avg latency
> Baseline: 4145337.60 Test: 4236159.60 Difference: +2.2%
> PartitionedFunctionExecutionBenchmark avg ops/sec
> Baseline: 248635.36 Test: 261498.94 Difference: +5.2%
> avg latency
> Baseline: 867122.63 Test: 824550.34 Difference: -4.9%
> PartitionedFunctionExecutionWithArgumentsBenchmark avg ops/sec
> Baseline: 280071.19 Test: 275305.31 Difference: -1.7%
> avg latency
> Baseline: 1026643.12 Test: 1044307.43 Difference: +1.7%
> PartitionedFunctionExecutionWithFiltersBenchmark avg ops/sec
> Baseline: 301416.23 Test: 304317.30 Difference: +1.0%
> avg latency
> Baseline: 1908390.88 Test: 1890040.46 Difference: -1.0%
> PartitionedGetBenchmark avg ops/sec
> Baseline: 790800.52 Test: 784514.74 Difference: -0.8%
> avg latency
> Baseline: 908357.58 Test: 915790.96 Difference: +0.8%
> PartitionedGetLongBenchmark avg ops/sec
> Baseline: 1020821.32 Test: 996529.93 Difference: -2.4%
> avg latency
> Baseline: 703761.09 Test: 720744.36 Difference: +2.4%
> PartitionedGetStringBenchmark avg ops/sec
> Baseline: 1028992.93 Test: 1010447.47 Difference: -1.8%
> avg latency
> Baseline: 698009.55 Test: 710765.29 Difference: +1.8%
> PartitionedIndexedQueryBenchmark avg ops/sec
> Baseline: 30868.78 Test: 31478.90 Difference: +2.0%
> avg latency
> Baseline: 18670093.21 Test: 18278083.16 Difference: -2.1%
> PartitionedNonIndexedQueryBenchmark avg ops/sec
> Baseline: 99.45 Test: 101.97 Difference: +2.5%
> avg latency
> Baseline: 723415530.75 Test: 705653061.86 Difference: -2.5%
> PartitionedPutAllBenchmark avg ops/sec
> Baseline: 7921.61 Test: 7816.66 Difference: -1.3%
> avg latency
> Baseline: 18172638.37 Test: 18416169.28 Difference: +1.3%
> PartitionedPutAllLongBenchmark avg ops/sec
> Baseline: 1379.53 Test: 1169.16 Difference: -15.2%
> avg latency
> Baseline: 105140260.44 Test: 123722914.94 Difference: +17.7%
> PartitionedPutBenchmark avg ops/sec
> Baseline: 474986.11 Test: 467924.19 Difference: -1.5%
> avg latency
> Baseline: 1514276.31 Test: 1536263.99 Difference: +1.5%
> PartitionedPutBytesBenchmark avg ops/sec
> Baseline: 457550.69 Test: 456011.33 Difference: -0.3%
> avg latency
> Baseline: 1570713.84 Test: 1575841.02 Difference: +0.3%
> PartitionedPutLongBenchmark avg ops/sec
> Baseline: 418221.79 Test: 389221.70 Difference: -6.9%
> avg latency
> Baseline: 1717869.66 Test: 1849602.96 Difference: +7.7%
> PartitionedPutStringBenchmark avg ops/sec
> Baseline: 410007.93 Test: 390442.31 Difference: -4.8%
> avg latency
> Baseline: 1754915.95 Test: 1841243.74 Difference: +4.9%
> ReplicatedFunctionExecutionBenchmark avg ops/sec
> Baseline: 478780.89 Test: 466855.62 Difference: -2.5%
> avg latency
> Baseline: 449616.85 Test: 461217.05 Difference: +2.6%
> ReplicatedFunctionExecutionWithArgumentsBenchmark avg ops/sec
> Baseline: 504979.76 Test: 501025.09 Difference: -0.8%
> avg latency
> Baseline: 2279046.16 Test: 2296973.89 Difference: +0.8%
> ReplicatedFunctionExecutionWithFiltersBenchmark avg ops/sec
> Baseline: 523232.66 Test: 517943.28 Difference: -1.0%
> avg latency
> Baseline: 1374146.10 Test: 1387969.18 Difference: +1.0%
> ReplicatedGetBenchmark avg ops/sec
> Baseline: 799001.77 Test: 788077.23 Difference: -1.4%
> avg latency
> Baseline: 899092.08 Test: 911932.02 Difference: +1.4%
> ReplicatedGetLongBenchmark avg ops/sec
> Baseline: 1038120.92 Test: 1025267.47 Difference: -1.2%
> avg latency
> Baseline: 691970.73 Test: 700641.58 Difference: +1.3%
> ReplicatedIndexedQueryBenchmark avg ops/sec
> Baseline: 32088.81 Test: 31641.88 Difference: -1.4%
> avg latency
> Baseline: 17960402.47 Test: 18222972.89 Difference: +1.5%
> ReplicatedNonIndexedQueryBenchmark avg ops/sec
> Baseline: 49.31 Test: 48.15 Difference: -2.4%
> avg latency
> Baseline: 1456350672.39 Test: 1490270040.10 Difference: +2.3%
> ReplicatedPutAllBenchmark avg ops/sec
> Baseline: 821.20 Test: 793.66 Difference: -3.4%
> avg latency
> Baseline: 175498155.76 Test: 181712150.69 Difference: +3.5%
> ReplicatedPutAllLongBenchmark avg ops/sec
> Baseline: 855.75 Test: 812.41 Difference: -5.1%
> avg latency
> Baseline: 168343905.55 Test: 177510321.30 Difference: +5.4%
> ReplicatedPutBenchmark avg ops/sec
> Baseline: 485135.10 Test: 481879.66 Difference: -0.7%
> avg latency
> Baseline: 1482039.07 Test: 1491885.72 Difference: +0.7%
> ReplicatedPutLongBenchmark avg ops/sec
> Baseline: 454212.92 Test: 443628.38 Difference: -2.3%
> avg latency
> Baseline: 1583988.26 Test: 1620588.33 Difference: +2.3%
> This is ITERATION 2 of benchmarking against baseline.
> PartitionedPutAllLongBenchmark avg ops/sec
> Baseline: 1359.77 Test: 1263.03 Difference: -7.1%
> avg latency
> Baseline: 106472640.40 Test: 114522255.46 Difference: +7.6%
> PartitionedPutLongBenchmark avg ops/sec
> Baseline: 426518.85 Test: 400573.57 Difference: -6.1%
> avg latency
> Baseline: 1686679.27 Test: 1793189.27 Difference: +6.3%
> ReplicatedPutAllLongBenchmark avg ops/sec
> Baseline: 847.77 Test: 802.20 Difference: -5.4%
> avg latency
> Baseline: 170110197.18 Test: 179656637.86 Difference: +5.6%
> This is ITERATION 3 of benchmarking against baseline.
> PartitionedPutAllLongBenchmark avg ops/sec
> Baseline: 1308.38 Test: 1371.30 Difference: +4.8%
> avg latency
> Baseline: 110742328.38 Test: 105657848.59 Difference: -4.6%
> PartitionedPutLongBenchmark avg ops/sec
> Baseline: 422675.38 Test: 393034.80 Difference: -7.0%
> avg latency
> Baseline: 1701984.68 Test: 1830333.62 Difference: +7.5%
> ReplicatedPutAllLongBenchmark avg ops/sec
> Baseline: 882.11 Test: 868.74 Difference: -1.5%
> avg latency
> Baseline: 163447771.46 Test: 166032265.56 Difference: +1.6%
> This is ITERATION 4 of benchmarking against baseline.
> PartitionedPutLongBenchmark avg ops/sec
> Baseline: 421497.23 Test: 394141.73 Difference: -6.5%
> avg latency
> Baseline: 1705094.48 Test: 1823187.55 Difference: +6.9%
> This is ITERATION 5 of benchmarking against baseline.
> PartitionedPutLongBenchmark avg ops/sec
> Baseline: 410979.61 Test: 387125.91 Difference: -5.8%
> avg latency
> Baseline: 1749420.68 Test: 1854964.43 Difference: +6.0%
> {code}
> Potentially, there is some instability in this benchmark since it passed in
> the runs immediately before and after.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
