shauryachats opened a new pull request, #14992: URL: https://github.com/apache/pinot/pull/14992
This PR adds performance benchmarking logic to identify and measure the improvement of different strategies for hash map selection, to make a data-driven choice on the hash map used to power `GroupIdGenerator`, which has been described in the issue https://github.com/apache/pinot/issues/14685. The results of this benchmark are: ``` Benchmark (_cardinality) Mode Cnt Score Error Units BenchmarkObjectOpenHashMap.object2IntOpenHashMap 500000 avgt 20 111.262 ± 5.448 ms/op BenchmarkObjectOpenHashMap.object2IntOpenHashMap 1000000 avgt 20 299.255 ± 9.814 ms/op BenchmarkObjectOpenHashMap.object2IntOpenHashMap 5000000 avgt 20 1859.503 ± 58.990 ms/op BenchmarkObjectOpenHashMap.object2IntOpenHashMap 20000000 avgt 20 8236.525 ± 170.751 ms/op BenchmarkObjectOpenHashMap.object2IntReservedOpenHashMap 500000 avgt 20 79.908 ± 4.715 ms/op BenchmarkObjectOpenHashMap.object2IntReservedOpenHashMap 1000000 avgt 20 180.827 ± 19.987 ms/op BenchmarkObjectOpenHashMap.object2IntReservedOpenHashMap 5000000 avgt 20 1051.368 ± 49.204 ms/op BenchmarkObjectOpenHashMap.object2IntReservedOpenHashMap 20000000 avgt 20 3340.668 ± 106.874 ms/op BenchmarkObjectOpenHashMap.vanillaHashMap 500000 avgt 20 109.589 ± 2.836 ms/op BenchmarkObjectOpenHashMap.vanillaHashMap 1000000 avgt 20 265.262 ± 5.215 ms/op BenchmarkObjectOpenHashMap.vanillaHashMap 5000000 avgt 20 1556.399 ± 49.787 ms/op BenchmarkObjectOpenHashMap.vanillaHashMap 20000000 avgt 20 6757.234 ± 314.138 ms/op BenchmarkObjectOpenHashMap.vanillaReservedHashMap 500000 avgt 20 98.798 ± 4.344 ms/op BenchmarkObjectOpenHashMap.vanillaReservedHashMap 1000000 avgt 20 228.480 ± 6.570 ms/op BenchmarkObjectOpenHashMap.vanillaReservedHashMap 5000000 avgt 20 1067.580 ± 48.764 ms/op BenchmarkObjectOpenHashMap.vanillaReservedHashMap 20000000 avgt 20 4725.897 ± 284.449 ms/op ``` which yields that the reserved hashmap performs ~2.5x better than the current unreserved hashmap, which led to the following PR: https://github.com/apache/pinot/pull/14981. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
