[
https://issues.apache.org/jira/browse/HADOOP-15598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16539750#comment-16539750
]
Prasanth Jayachandran commented on HADOOP-15598:
------------------------------------------------
Results from JMH microbenchmark. As we can see for bytebuffers of size >128
there is no significant benefit as most of the time is spent on the chunk
looping. But for smaller size, the synchronization has significant impact.
{noformat}
Benchmark (size) Mode Cnt
Score Error Units
ComputeChecksumBenchmark.benchmarkChecksumWithoutPatch 8 avgt 10
66.970 ± 1.247 ns/op
ComputeChecksumBenchmark.benchmarkChecksumWithoutPatch 16 avgt 10
91.524 ± 2.141 ns/op
ComputeChecksumBenchmark.benchmarkChecksumWithoutPatch 32 avgt 10
127.155 ± 8.037 ns/op
ComputeChecksumBenchmark.benchmarkChecksumWithoutPatch 64 avgt 10
233.941 ± 14.327 ns/op
ComputeChecksumBenchmark.benchmarkChecksumWithoutPatch 128 avgt 10
382.898 ± 4.316 ns/op
ComputeChecksumBenchmark.benchmarkChecksumWithoutPatch 1024 avgt 10
2707.026 ± 53.266 ns/op
ComputeChecksumBenchmark.benchmarkChecksumWithoutPatch 4096 avgt 10
11411.330 ± 181.900 ns/op
Benchmark (size) Mode Cnt
Score Error Units
ComputeChecksumBenchmark.benchmarkChecksumWithPatch 8 avgt 10
27.948 ± 0.507 ns/op
ComputeChecksumBenchmark.benchmarkChecksumWithPatch 16 avgt 10
49.336 ± 1.212 ns/op
ComputeChecksumBenchmark.benchmarkChecksumWithPatch 32 avgt 10
96.344 ± 1.360 ns/op
ComputeChecksumBenchmark.benchmarkChecksumWithPatch 64 avgt 10
182.513 ± 2.553 ns/op
ComputeChecksumBenchmark.benchmarkChecksumWithPatch 128 avgt 10
356.205 ± 4.282 ns/op
ComputeChecksumBenchmark.benchmarkChecksumWithPatch 1024 avgt 10
2825.526 ± 31.177 ns/op
ComputeChecksumBenchmark.benchmarkChecksumWithPatch 4096 avgt 10
12078.055 ± 262.343 ns/op
{noformat}
> DataChecksum calculate checksum is contented on hashtable synchronization
> -------------------------------------------------------------------------
>
> Key: HADOOP-15598
> URL: https://issues.apache.org/jira/browse/HADOOP-15598
> Project: Hadoop Common
> Issue Type: Bug
> Components: common
> Affects Versions: 3.2.0
> Reporter: Prasanth Jayachandran
> Priority: Major
> Attachments: HADOOP-15598.1.patch, Screen Shot 2018-07-11 at 1.45.06
> AM.png
>
>
> When profiling a multi-threaded hive streaming ingest, observed lock
> contention on java.util.Properties getProperty() to check if os is "sparc".
> java.util.Properties internally uses HashTable. HashTable.get() is
> synchronized method. In the test application, on a 30s profile with 64
> threads ~40% CPU time is spent on getProperty() contention. See attached
> snapshot.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
