[
https://issues.apache.org/jira/browse/MAPREDUCE-7475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17842290#comment-17842290
]
ASF GitHub Bot commented on MAPREDUCE-7475:
-------------------------------------------
kaiyaok2 opened a new pull request, #6785:
URL: https://github.com/apache/hadoop/pull/6785
### Description of PR
As described in https://issues.apache.org/jira/browse/MAPREDUCE-7475: 2
tests are not idempotent and fails upon repeated execution within the same JVM
instance due to self-induced state pollution. Specifically, these tests try to
make the directory `TEST_ROOT_DIR` and write to it. The tests do not clean up
(remove) the directory after execution. Therefore, in the second execution,
`TEST_ROOT_DIR` would already exist and the exception `Could not create test
dir` would be thrown. The tests shall be fixed as unit tests shall be
idempotent.
Below are the 2 non-idempotent tests:
* `org.apache.hadoop.mapred.TestOldCombinerGrouping.testCombiner`
* `org.apache.hadoop.mapreduce.TestNewCombinerGrouping.testCombiner`
##Sample Failure Message for
`org.apache.hadoop.mapreduce.TestNewCombinerGrouping.testCombiner` (in the 2nd
run of the test):
```
java.lang.RuntimeException: Could not create test dir:
/home/kaiyaok2/NIOExperiments/github.com/apache/hadoop/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/build/test/data/e4ee268d-2946-43a2-93b5-f2b4ac647279
at
org.apache.hadoop.mapreduce.TestNewCombinerGrouping.testCombiner(TestNewCombinerGrouping.java:109)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
```
### How was this patch tested?
After the patch, rerunning the tests in the same JVM does not produce any
exceptions.
### Code changes:
Check if the test directory already exists (and delete it if so) before
calling `mkdirs()`.
> 2 tests are non-idempotent (passes in the first run but fails in repeated
> runs in the same JVM)
> -----------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-7475
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7475
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Environment: Ubuntu 22.04, Java 17
> Reporter: Kaiyao Ke
> Priority: Major
> Original Estimate: 10m
> Remaining Estimate: 10m
>
> 2 tests are not idempotent and fails upon repeated execution within the same
> JVM instance due to self-induced state pollution. Specifically, these tests
> try to make the directory TEST_ROOT_DIR and write to it. The tests do not
> clean up (remove) the directory after execution. Therefore, in the second
> execution, TEST_ROOT_DIR would already exist and the exception `Could not
> create test dir` would be thrown. Below are the 2 non-idempotent tests:
> * org.apache.hadoop.mapred.TestOldCombinerGrouping.testCombiner
> * org.apache.hadoop.mapreduce.TestNewCombinerGrouping.testCombiner
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
