[
https://issues.apache.org/jira/browse/HDFS-17972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18111456#comment-18111456
]
ASF GitHub Bot commented on HDFS-17972:
---------------------------------------
pan3793 commented on code in PR #8712:
URL: https://github.com/apache/hadoop/pull/8712#discussion_r3932026368
##########
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientRetries.java:
##########
@@ -1279,12 +1279,15 @@ public void
testDFSClientConfigurationLocateFollowingBlock()
@Timeout(value = 120)
public void testLeaseRenewAndDFSOutputStreamDeadLock() throws Exception {
CountDownLatch testLatch = new CountDownLatch(1);
+ DFSClientFaultInjector oldInjector = DFSClientFaultInjector.get();
DFSClientFaultInjector.set(new DFSClientFaultInjector() {
public void delayWhenRenewLeaseTimeout() {
try {
- testLatch.await();
+ // Bounded, so a renewer thread cannot be parked here forever if this
+ // test dies without running its finally block.
Review Comment:
Wording nit. The `finally` block always runs; the problem is that
`out1.close()` never returns, so it is reached too late. Suggest:
```suggestion
// Bounded, so a stalled out1.close() cannot leave the renewer
// holding the LeaseRenewer monitor forever.
```
> TestDFSClientRetries#testLeaseRenewAndDFSOutputStreamDeadLock hangs the
> surefire fork instead of failing
> --------------------------------------------------------------------------------------------------------
>
> Key: HDFS-17972
> URL: https://issues.apache.org/jira/browse/HDFS-17972
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs, test
> Reporter: Jose Luis López
> Priority: Major
> Labels: pull-request-available
>
> h2. Problem
> This problem has no production impact. It is limited to the test
> infraestructure.
> {{testLeaseRenewAndDFSOutputStreamDeadLock}} blocks indefinitely instead of
> failing at its 120s deadline. It converts a bounded 120-second test failure
> into a build-killing hang that destroys the test report for an entire module
> and leaves no machine-readable trace. Yetus emits no per-class failure table
> for the affected run, so the contributor loses the result for every test in
> hadoop-hdfs, not just this one.
> Observed in precommit as {{testLeaseRenewAndDFSOutputStreamDeadLock}} and
> {{testLeaseRenewSocketTimeout}} failing together, always as timeouts, never
> as assertions, and both passing when re-run individually. Worst case seen:
> PR-8634 build 3 ran ~24 hours and died with exit code 137.
> This problem hangs the test process for very long time instead of producing a
> failure in the timeout period of the test.
> The impact is that all tests from the hdfs module fail because of the hang,
> instead of failing just one test. The fix would turn every future occurrence
> from failing all tests back into a single failed test.
> h2. Root cause
> Two defects combine:
> # The timeout cannot interrupt a blocked test.
> # The test leaks a global fault injector.
> This conditions occur in heavy load of the testing infraestructure.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
