michaeljmarshall commented on PR #16145: URL: https://github.com/apache/lucene/pull/16145#issuecomment-5904377113
I just pushed a commit with significantly updated logic. It might have belonged in a separate PR, but I decided to keep it here since the git history and the comments provide the relevant motivation for the change. I'll update the PR description accordingly. Thanks for the reference @jimczi. I think the `NoReuseHint` could be used to skip the backoff logic and always issue the madvise call. But even with the backoff logic, my numbers suggest this change (as of 4721c753de7222cb334cdd94edf26465489aaddf) should perform the same or better for most use cases. One of the notable changes I made in the latest commit is the removal of the volatile write to the `AtomicInteger` from the hot read path once we reach the max back off. The primary benefit of skipping this update is independence of threads, which should reduce contention and reduce unnecessary cache line invalidation. I ran benchmarks and posted the results here https://github.com/michaeljmarshall/lucene/commit/e7138cc4b2d4c6441d877d6a41ba835fb0d2a5d4. The benchmark tooling was based on @neoremind's tooling in https://github.com/apache/lucene/pull/16279 and https://github.com/neoremind/lucene/commit/13c55fe2e34efecdfbcbcfd13d85ea2ccdaadc33. I used claude to build the integration tooling to get the benchmark up and running. The complete results are here https://github.com/michaeljmarshall/lucene/blob/e7138cc4b2d4c6441d877d6a41ba835fb0d2a5d4/lucene/benchmark-jmh/scripts/prefetch-backoff/results.md. The results page has all of my per-iteration results. What follows are the aggregation of these results. Note that the error bars are large on some, but most show a clear story. For methodology, see https://github.com/michaeljmarshall/lucene/blob/e7138cc4b2d4c6441d877d6a41ba835fb0d2a5d4/lucene/benchmark-jmh/scripts/prefetch-backoff/README.md. tl;dr: The results here are from a GCP `n2-custom-16-32768` (Cascade Lake, 8 cores / 16 threads, 31 GiB) with two local SSDs in RAID-0, ext4, Ubuntu 22.04. # Results Generated by compare.py and iterations.py from the JSON in results/ and results-hot2/. ## main vs branch, first pass (results/) ### hot / randomread (ops/ms, higher is better) | benchmark | main | branch | delta | |---|---:|---:|---:| | mmap_T01 [readSize=16384,readsPerOp=16] | 44.7 ± 1.9 | 44.9 ± 0.7 | +0.3% | | mmap_T04 [readSize=16384,readsPerOp=16] | 180.3 ± 1.4 | 179.0 ± 1.4 | -0.7% | | mmap_T08 [readSize=16384,readsPerOp=16] | 327.4 ± 13.3 | 332.6 ± 3.0 | +1.6% | | mmap_T16 [readSize=16384,readsPerOp=16] | 336.1 ± 3.2 | 346.6 ± 1.0 | +3.1% | | mmapBatchedPrefetch_T01 [readSize=16384,readsPerOp=16] | 44.1 ± 0.7 | 42.5 ± 2.5 | -3.7% | | mmapBatchedPrefetch_T04 [readSize=16384,readsPerOp=16] | 177.4 ± 1.4 | 177.1 ± 4.0 | -0.2% | | mmapBatchedPrefetch_T08 [readSize=16384,readsPerOp=16] | 328.2 ± 2.1 | 329.7 ± 9.7 | +0.4% | | mmapBatchedPrefetch_T16 [readSize=16384,readsPerOp=16] | 342.6 ± 2.6 | 341.6 ± 8.6 | -0.3% | | mmapRandom_T01 [readSize=16384,readsPerOp=16] | 44.8 ± 1.4 | 44.8 ± 1.6 | +0.1% | | mmapRandom_T04 [readSize=16384,readsPerOp=16] | 181.1 ± 1.3 | 178.7 ± 1.7 | -1.4% | | mmapRandom_T08 [readSize=16384,readsPerOp=16] | 336.6 ± 2.5 | 331.6 ± 1.2 | -1.5% | | mmapRandom_T16 [readSize=16384,readsPerOp=16] | 349.0 ± 2.6 | 345.6 ± 2.0 | -1.0% | | mmapRandomBatchedPrefetch_T01 [readSize=16384,readsPerOp=16] | 44.4 ± 1.7 | 44.6 ± 0.9 | +0.5% | | mmapRandomBatchedPrefetch_T04 [readSize=16384,readsPerOp=16] | 177.8 ± 1.4 | 166.4 ± 40.2 | -6.4% | | mmapRandomBatchedPrefetch_T08 [readSize=16384,readsPerOp=16] | 329.5 ± 3.2 | 288.8 ± 53.0 | -12.4% | | mmapRandomBatchedPrefetch_T16 [readSize=16384,readsPerOp=16] | 341.7 ± 3.4 | 339.3 ± 9.8 | -0.7% | ### hot / storedfields (ops/ms, higher is better) | benchmark | main | branch | delta | |---|---:|---:|---:| | prefetchOnly_T01 [topK=100] | 42.2 ± 0.4 | 38.7 ± 0.3 | -8.1% | | prefetchOnly_T01 [topK=10] | 409.2 ± 35.4 | 375.5 ± 2.3 | -8.2% | | prefetchOnly_T08 [topK=100] | 210.5 ± 12.7 | 309.9 ± 0.5 | +47.2% | | prefetchOnly_T08 [topK=10] | 2,007.4 ± 250.3 | 3,042.8 ± 132.1 | +51.6% | | retrieveNoPrefetch_T01 [topK=100] | 2.8 ± 0.0 | 2.8 ± 0.1 | +0.1% | | retrieveNoPrefetch_T01 [topK=10] | 28.0 ± 0.5 | 28.1 ± 0.5 | +0.4% | | retrieveNoPrefetch_T08 [topK=100] | 17.4 ± 0.1 | 17.6 ± 0.1 | +1.2% | | retrieveNoPrefetch_T08 [topK=10] | 173.7 ± 0.8 | 174.7 ± 1.0 | +0.5% | | retrievePrefetchThenRead_T01 [topK=100] | 2.6 ± 0.0 | 2.6 ± 0.0 | -0.9% | | retrievePrefetchThenRead_T01 [topK=10] | 26.2 ± 0.7 | 26.0 ± 0.6 | -0.7% | | retrievePrefetchThenRead_T08 [topK=100] | 16.6 ± 0.5 | 16.8 ± 0.1 | +1.1% | | retrievePrefetchThenRead_T08 [topK=10] | 164.2 ± 1.5 | 166.5 ± 1.1 | +1.4% | ### pressure / randomread (ops/ms, higher is better) | benchmark | main | branch | delta | |---|---:|---:|---:| | mmap_T01 [readSize=16384,readsPerOp=16] | 0.8 ± 0.1 | 0.7 ± 0.0 | -6.7% | | mmap_T04 [readSize=16384,readsPerOp=16] | 3.1 ± 0.2 | 3.1 ± 0.2 | -1.7% | | mmap_T08 [readSize=16384,readsPerOp=16] | 3.8 ± 0.1 | 3.8 ± 0.1 | -0.5% | | mmap_T16 [readSize=16384,readsPerOp=16] | 3.8 ± 0.1 | 3.8 ± 0.1 | -0.2% | | mmapBatchedPrefetch_T01 [readSize=16384,readsPerOp=16] | 1.0 ± 0.1 | 4.0 ± 0.3 | +304.2% | | mmapBatchedPrefetch_T04 [readSize=16384,readsPerOp=16] | 3.3 ± 0.2 | 13.4 ± 0.8 | +310.4% | | mmapBatchedPrefetch_T08 [readSize=16384,readsPerOp=16] | 3.8 ± 0.1 | 19.5 ± 0.4 | +409.2% | | mmapBatchedPrefetch_T16 [readSize=16384,readsPerOp=16] | 3.8 ± 0.2 | 19.2 ± 1.1 | +399.7% | | mmapRandom_T01 [readSize=16384,readsPerOp=16] | 0.4 ± 0.1 | 0.4 ± 0.0 | +3.0% | | mmapRandom_T04 [readSize=16384,readsPerOp=16] | 1.6 ± 0.1 | 1.6 ± 0.1 | -1.1% | | mmapRandom_T08 [readSize=16384,readsPerOp=16] | 2.9 ± 0.1 | 2.8 ± 0.1 | -5.7% | | mmapRandom_T16 [readSize=16384,readsPerOp=16] | 5.1 ± 0.4 | 5.0 ± 0.3 | -2.5% | | mmapRandomBatchedPrefetch_T01 [readSize=16384,readsPerOp=16] | 0.4 ± 0.0 | 3.8 ± 0.1 | +779.8% | | mmapRandomBatchedPrefetch_T04 [readSize=16384,readsPerOp=16] | 1.6 ± 0.3 | 13.6 ± 0.3 | +743.7% | | mmapRandomBatchedPrefetch_T08 [readSize=16384,readsPerOp=16] | 3.2 ± 0.5 | 18.6 ± 4.8 | +475.0% | | mmapRandomBatchedPrefetch_T16 [readSize=16384,readsPerOp=16] | 6.3 ± 1.0 | 19.6 ± 0.2 | +211.8% | ### pressure / storedfields (ops/ms, higher is better) | benchmark | main | branch | delta | |---|---:|---:|---:| | prefetchOnly_T01 [topK=100] | 40.0 ± 0.3 | 0.8 ± 0.0 | -98.1% | | prefetchOnly_T01 [topK=10] | 367.9 ± 96.0 | 7.6 ± 0.2 | -97.9% | | prefetchOnly_T08 [topK=100] | 200.9 ± 19.2 | 0.8 ± 0.1 | -99.6% | | prefetchOnly_T08 [topK=10] | 2,179.8 ± 145.3 | 7.5 ± 0.2 | -99.7% | | retrieveNoPrefetch_T01 [topK=100] | 0.1 ± 0.0 | 0.1 ± 0.0 | +12.9% | | retrieveNoPrefetch_T01 [topK=10] | 0.5 ± 0.1 | 0.6 ± 0.0 | +19.6% | | retrieveNoPrefetch_T08 [topK=100] | 0.4 ± 0.2 | 0.4 ± 0.1 | +0.2% | | retrieveNoPrefetch_T08 [topK=10] | 4.0 ± 1.8 | 4.3 ± 0.8 | +8.0% | | retrievePrefetchThenRead_T01 [topK=100] | 0.1 ± 0.0 | 0.4 ± 0.1 | +256.7% | | retrievePrefetchThenRead_T01 [topK=10] | 1.0 ± 1.0 | 3.0 ± 0.7 | +190.1% | | retrievePrefetchThenRead_T08 [topK=100] | 0.6 ± 0.4 | 0.6 ± 0.1 | +7.3% | | retrievePrefetchThenRead_T08 [topK=10] | 5.1 ± 1.9 | 6.1 ± 2.5 | +19.8% | ### cold / randomread (ops/ms, higher is better) | benchmark | main | branch | delta | |---|---:|---:|---:| | mmap_T01 [readSize=16384,readsPerOp=16] | 14.8 ± 51.1 | 18.1 ± 56.1 | +21.8% | | mmap_T04 [readSize=16384,readsPerOp=16] | 169.2 ± 2.6 | 176.6 ± 2.7 | +4.4% | | mmap_T08 [readSize=16384,readsPerOp=16] | 311.4 ± 17.2 | 324.0 ± 2.0 | +4.0% | | mmap_T16 [readSize=16384,readsPerOp=16] | 331.1 ± 2.0 | 339.5 ± 2.8 | +2.6% | | mmapBatchedPrefetch_T01 [readSize=16384,readsPerOp=16] | 19.2 ± 54.0 | 23.7 ± 48.8 | +23.4% | | mmapBatchedPrefetch_T04 [readSize=16384,readsPerOp=16] | 170.9 ± 4.9 | 172.9 ± 0.9 | +1.2% | | mmapBatchedPrefetch_T08 [readSize=16384,readsPerOp=16] | 311.2 ± 14.2 | 319.3 ± 2.0 | +2.6% | | mmapBatchedPrefetch_T16 [readSize=16384,readsPerOp=16] | 322.1 ± 3.5 | 336.3 ± 2.1 | +4.4% | | mmapRandom_T01 [readSize=16384,readsPerOp=16] | 0.1 ± 0.0 | 0.1 ± 0.0 | -3.7% | | mmapRandom_T04 [readSize=16384,readsPerOp=16] | 0.5 ± 0.3 | 0.6 ± 0.4 | +17.3% | | mmapRandom_T08 [readSize=16384,readsPerOp=16] | 14.4 ± 66.4 | 45.1 ± 188.0 | +212.3% | | mmapRandom_T16 [readSize=16384,readsPerOp=16] | 205.3 ± 431.5 | 233.7 ± 453.5 | +13.8% | | mmapRandomBatchedPrefetch_T01 [readSize=16384,readsPerOp=16] | 0.6 ± 0.6 | 24.2 ± 48.2 | +3712.2% | | mmapRandomBatchedPrefetch_T04 [readSize=16384,readsPerOp=16] | 90.9 ± 209.7 | 171.6 ± 2.6 | +88.7% | | mmapRandomBatchedPrefetch_T08 [readSize=16384,readsPerOp=16] | 281.4 ± 116.4 | 315.9 ± 3.3 | +12.2% | | mmapRandomBatchedPrefetch_T16 [readSize=16384,readsPerOp=16] | 307.2 ± 11.1 | 331.3 ± 4.4 | +7.8% | ### cold / storedfields (ops/ms, higher is better) | benchmark | main | branch | delta | |---|---:|---:|---:| | prefetchOnly_T01 [topK=100] | 16.6 ± 3.8 | 0.9 ± 0.0 | -94.6% | | prefetchOnly_T01 [topK=10] | 151.1 ± 14.0 | 9.1 ± 0.2 | -94.0% | | prefetchOnly_T08 [topK=100] | 77.2 ± 5.1 | 28.7 ± 2.0 | -62.8% | | prefetchOnly_T08 [topK=10] | 770.5 ± 156.9 | 296.8 ± 140.3 | -61.5% | | retrieveNoPrefetch_T01 [topK=100] | 0.0 ± 0.0 | 0.0 ± 0.0 | -2.2% | | retrieveNoPrefetch_T01 [topK=10] | 0.2 ± 0.0 | 0.2 ± 0.0 | -0.4% | | retrieveNoPrefetch_T08 [topK=100] | 0.3 ± 0.3 | 0.3 ± 0.3 | +0.2% | | retrieveNoPrefetch_T08 [topK=10] | 2.9 ± 2.8 | 2.9 ± 2.9 | +1.3% | | retrievePrefetchThenRead_T01 [topK=100] | 0.1 ± 0.0 | 2.2 ± 1.1 | +2974.6% | | retrievePrefetchThenRead_T01 [topK=10] | 0.9 ± 0.1 | 20.3 ± 17.6 | +2223.2% | | retrievePrefetchThenRead_T08 [topK=100] | 10.2 ± 19.9 | 16.5 ± 0.7 | +60.5% | | retrievePrefetchThenRead_T08 [topK=10] | 98.0 ± 196.4 | 164.2 ± 7.3 | +67.5% | ## main vs branch, hot repeat (results-hot2/) ### hot / randomread (ops/ms, higher is better) | benchmark | main | branch | delta | |---|---:|---:|---:| | mmap_T01 [readSize=16384,readsPerOp=16] | 45.2 ± 1.2 | 45.1 ± 0.8 | -0.3% | | mmap_T04 [readSize=16384,readsPerOp=16] | 181.9 ± 2.2 | 181.0 ± 1.3 | -0.5% | | mmap_T08 [readSize=16384,readsPerOp=16] | 337.6 ± 2.6 | 336.0 ± 2.1 | -0.5% | | mmap_T16 [readSize=16384,readsPerOp=16] | 349.4 ± 3.0 | 348.6 ± 1.6 | -0.2% | | mmapBatchedPrefetch_T01 [readSize=16384,readsPerOp=16] | 45.2 ± 0.7 | 44.8 ± 1.1 | -1.0% | | mmapBatchedPrefetch_T04 [readSize=16384,readsPerOp=16] | 179.3 ± 2.3 | 179.7 ± 1.7 | +0.2% | | mmapBatchedPrefetch_T08 [readSize=16384,readsPerOp=16] | 331.7 ± 2.9 | 333.2 ± 1.9 | +0.5% | | mmapBatchedPrefetch_T16 [readSize=16384,readsPerOp=16] | 344.7 ± 1.5 | 347.9 ± 5.1 | +0.9% | | mmapRandom_T01 [readSize=16384,readsPerOp=16] | 45.4 ± 1.6 | 45.6 ± 1.2 | +0.3% | | mmapRandom_T04 [readSize=16384,readsPerOp=16] | 182.4 ± 1.3 | 182.3 ± 1.5 | -0.0% | | mmapRandom_T08 [readSize=16384,readsPerOp=16] | 337.4 ± 1.9 | 337.4 ± 2.6 | +0.0% | | mmapRandom_T16 [readSize=16384,readsPerOp=16] | 348.8 ± 2.0 | 350.5 ± 2.0 | +0.5% | | mmapRandomBatchedPrefetch_T01 [readSize=16384,readsPerOp=16] | 44.3 ± 1.3 | 44.5 ± 0.6 | +0.3% | | mmapRandomBatchedPrefetch_T04 [readSize=16384,readsPerOp=16] | 177.5 ± 1.9 | 179.3 ± 3.0 | +1.0% | | mmapRandomBatchedPrefetch_T08 [readSize=16384,readsPerOp=16] | 331.8 ± 2.5 | 331.7 ± 3.0 | -0.0% | | mmapRandomBatchedPrefetch_T16 [readSize=16384,readsPerOp=16] | 345.7 ± 2.4 | 345.9 ± 1.5 | +0.1% | ### hot / storedfields (ops/ms, higher is better) | benchmark | main | branch | delta | |---|---:|---:|---:| | prefetchOnly_T01 [topK=100] | 42.5 ± 0.1 | 38.8 ± 0.1 | -8.7% | | prefetchOnly_T01 [topK=10] | 399.7 ± 1.3 | 380.0 ± 16.6 | -4.9% | | prefetchOnly_T08 [topK=100] | 221.2 ± 15.4 | 309.2 ± 0.3 | +39.8% | | prefetchOnly_T08 [topK=10] | 2,066.2 ± 86.1 | 3,045.3 ± 116.7 | +47.4% | | retrieveNoPrefetch_T01 [topK=100] | 2.8 ± 0.0 | 2.8 ± 0.0 | -0.8% | | retrieveNoPrefetch_T01 [topK=10] | 28.2 ± 0.4 | 27.6 ± 0.3 | -2.1% | | retrieveNoPrefetch_T08 [topK=100] | 17.5 ± 0.1 | 17.2 ± 0.2 | -1.9% | | retrieveNoPrefetch_T08 [topK=10] | 175.3 ± 0.7 | 172.5 ± 2.5 | -1.6% | | retrievePrefetchThenRead_T01 [topK=100] | 2.6 ± 0.0 | 2.6 ± 0.1 | -1.2% | | retrievePrefetchThenRead_T01 [topK=10] | 26.0 ± 0.4 | 25.8 ± 0.5 | -1.0% | | retrievePrefetchThenRead_T08 [topK=100] | 16.8 ± 0.2 | 16.7 ± 0.2 | -0.7% | | retrievePrefetchThenRead_T08 [topK=10] | 167.5 ± 3.3 | 165.5 ± 1.6 | -1.2% | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
