[
https://issues.apache.org/jira/browse/MAPREDUCE-7524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18068842#comment-18068842
]
ASF GitHub Bot commented on MAPREDUCE-7524:
-------------------------------------------
github-actions[bot] closed pull request #8134: MAPREDUCE-7524 Addressed
indeterminism in TestCombineFileInputFormat
URL: https://github.com/apache/hadoop/pull/8134
> testSplitPlacementForCompressedFiles is non-deterministic from time to time
> ---------------------------------------------------------------------------
>
> Key: MAPREDUCE-7524
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7524
> Project: Hadoop Map/Reduce
> Issue Type: Test
> Reporter: Paco Chan
> Priority: Trivial
> Labels: pull-request-available
>
> {code:java}
> org.apache.hadoop.mapreduce.lib.input.TestCombineFileInputFormat.testSplitPlacementForCompressedFiles{code}
> makes a number of assertions that are not guaranteed by internal
> implementation.
>
> For example, the test makes assumptions on the order in which racks are
> processed. Splits in the HDFS can appear in a different order depending on
> the internal state / implementation / metadata which is not guaranteed by the
> interface.
>
> For example, there are assertions such as split.equals(splits.get(0)), which
> assumes a specific ordering of InputSplit objects that may vary depending on
> internal implementation.
>
> This can be addressed by eliminating assertions based on the internal
> implementation / ways that the racks are split.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
