[
https://issues.apache.org/jira/browse/HADOOP-18896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18102034#comment-18102034
]
ASF GitHub Bot commented on HADOOP-18896:
-----------------------------------------
anshuksi282-ksolves commented on PR #8622:
URL: https://github.com/apache/hadoop/pull/8622#issuecomment-5190311928
Hi @pan3793,
Thank you for the detailed and insightful review! I have pushed a new commit
addressing all your findings:
1. Removed the dead if (bufSize < 0) and if (checksumBufSize < 0) branches,
as Math.multiplyExact handles the overflow by throwing an ArithmeticException.
2. Reverted the checksum buffer initialization back to the simple new
byte[getChecksumSize() * BUFFER_NUM_CHUNKS] one-liner, since getChecksumSize()
will never be large enough to overflow.
3. Updated resetChecksumBufSize() to use Math.multiplyExact for consistency
with the constructor and to guard against any latent overflow issues.
4. Regarding the exception type, I've left it as IllegalArgumentException
for now as you noted it's just informational, but please let me know if you'd
prefer me to switch it to HadoopIllegalArgumentException.
Please let me know if everything looks good now!
> NegativeArraySizeException thrown in FSOutputSummer.java given large
> file.bytes-per-checksum
> --------------------------------------------------------------------------------------------
>
> Key: HADOOP-18896
> URL: https://issues.apache.org/jira/browse/HADOOP-18896
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 3.3.6
> Reporter: rstest
> Priority: Critical
> Labels: pull-request-available
>
> Buffer size of FSOutputSummer equals to `file.bytes-per-checksum` times
> `BUFFER_NUM_CHUNKS`. A large `file.bytes-per-checksum` causes buffer size to
> overflow and crash with NegativeArraySizeException.
> To reproduce:
> 1. set `file.bytes-per-checksum` to 238609295
> 2. `mvn surefire:test
> -Dtest=org.apache.hadoop.hdfs.TestDecommissionWithStriped#testFileSmallerThanOneStripe`
> We created a PR that provides a fix which checks the buffer size is positive
> after multiplying `file.bytes-per-checksum` with `BUFFER_NUM_CHUNKS`
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
