symious commented on code in PR #5603:
URL: https://github.com/apache/hadoop/pull/5603#discussion_r1185646826
##########
hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/DistCpUtils.java:
##########
@@ -596,6 +596,8 @@ public static CopyMapper.ChecksumComparison
checksumsAreEqual(
} else if (sourceChecksum.equals(targetChecksum)) {
return CopyMapper.ChecksumComparison.TRUE;
}
+ LOG.info("Checksum not equal. Source checksum: {}, target checksum: {}",
+ sourceChecksum, targetChecksum);
Review Comment:
In our failed jobs, the mismatch source and target path was printed by
https://github.com/apache/hadoop/blob/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/DistCpUtils.java#L633.
Thanks for pointing that out. The paths have also been added to DistCpUtils.
##########
hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyCommitter.java:
##########
@@ -569,6 +572,7 @@ private void testCommitWithChecksumMismatch(boolean skipCrc)
fs, new Path(sourceBase + srcFilename), null,
fs, new Path(targetBase + srcFilename),
sourceCurrStatus.getLen()));
+ assertThat(log.getOutput(), containsString("Checksum not equal"));
Review Comment:
Copied the hamcrest version from
"org.apache.hadoop.conf.TestReconfiguration".
But changed to the Assertions.assertThat one.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
