[
https://issues.apache.org/jira/browse/HDFS-17364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17840969#comment-17840969
]
ASF GitHub Bot commented on HDFS-17364:
---------------------------------------
hadoop-yetus commented on PR #6514:
URL: https://github.com/apache/hadoop/pull/6514#issuecomment-2078145607
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 01s | | No case conflicting files
found. |
| +0 :ok: | spotbugs | 0m 00s | | spotbugs executables are not
available. |
| +0 :ok: | codespell | 0m 00s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 01s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 00s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 00s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 107m 55s | | trunk passed |
| +1 :green_heart: | compile | 6m 03s | | trunk passed |
| +1 :green_heart: | checkstyle | 5m 16s | | trunk passed |
| +1 :green_heart: | mvnsite | 6m 32s | | trunk passed |
| +1 :green_heart: | javadoc | 5m 41s | | trunk passed |
| +1 :green_heart: | shadedclient | 172m 45s | | branch has no errors
when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 44s | | the patch passed |
| +1 :green_heart: | compile | 3m 00s | | the patch passed |
| +1 :green_heart: | javac | 3m 00s | | the patch passed |
| +1 :green_heart: | blanks | 0m 00s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 2m 21s | | the patch passed |
| +1 :green_heart: | mvnsite | 3m 25s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 50s | | the patch passed |
| +1 :green_heart: | shadedclient | 185m 14s | | patch has no errors
when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 6m 11s | | The patch does not
generate ASF License warnings. |
| | | 494m 11s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| GITHUB PR | https://github.com/apache/hadoop/pull/6514 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | MINGW64_NT-10.0-17763 6320a162bf26 3.4.10-87d57229.x86_64
2024-02-14 20:17 UTC x86_64 Msys |
| Build tool | maven |
| Personality | /c/hadoop/dev-support/bin/hadoop.sh |
| git revision | trunk / 36954d34bac56942b8827587a13fed7893b51efa |
| Default Java | Azul Systems, Inc.-1.8.0_332-b09 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/job/PR-6514/1/testReport/
|
| modules | C: hadoop-hdfs-project/hadoop-hdfs-client U:
hadoop-hdfs-project/hadoop-hdfs-client |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/job/PR-6514/1/console
|
| versions | git=2.44.0.windows.1 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
> Use WeakReferencedElasticByteBufferPool in DFSStripedInputStream
> ----------------------------------------------------------------
>
> Key: HDFS-17364
> URL: https://issues.apache.org/jira/browse/HDFS-17364
> Project: Hadoop HDFS
> Issue Type: Improvement
> Reporter: Bryan Beaudreault
> Priority: Major
> Labels: pull-request-available
>
> DFSStripedInputStream uses ElasticByteBufferPool to allocate byte buffers for
> the "curStripeBuf". This is used for non-positional (stateful) reads and is
> allocated with a size of numDataBlocks * cellSize. For RS-6-3-1024k, that
> means each DFSStripedInputStream could allocate a 6mb buffer. When the IS is
> finished, the buffer is put back in the pool. Over time and with spikes of
> concurrent reads, the pool grows and most of the buffers sit there unused.
>
> WeakReferencedElasticByteBufferPool was introduced HADOOP-18105 and mitigates
> this issue because the excess buffers can be GC'd once they are no longer
> needed. We should use this same pool in DFSStripedInputStream
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
