[
https://issues.apache.org/jira/browse/HDFS-17893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18066418#comment-18066418
]
Dave Marion commented on HDFS-17893:
------------------------------------
It appears that HDFS-17080 added the calls to DFSStripedInputStream.close in
StripeReader.readStripe to resolve an issue with EC connection leaks.
HDFS-17811 added the retry read feature to DFSStripedInputStream which is
failing when an IOException occurs and the DFSStripedInputStream is closed. I'm
not sure the StripeReader should be closing the DFSStripedInputStream. Removing
the calls to close added in HDFS-17080, but not sure how that will affect the
original issue of connection leaks. Seems like the StripeReader should only
close the BlockReader's that it creates.
> NPE in DFSStripedInputStream.readWithStrategy
> ----------------------------------------------
>
> Key: HDFS-17893
> URL: https://issues.apache.org/jira/browse/HDFS-17893
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: ec
> Affects Versions: 3.5.0
> Reporter: Dave Marion
> Priority: Major
>
> [DFSStripedInputStream.readWithStrategy|https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSStripedInputStream.java#L391]
> checks to see if it is closed, then enters a retry loop where it ends up
> calling *readOneStripe* (line 421). *readOneStripe* calls *readStripe* on the
> {*}StatefulStripeReader{*}, which defaults to its parent class
> {*}StripeReader{*}. *StripeReader.readStripe* (line 356) on an exception
> calls close on the *DFSStripedInputStream* which ends up setting *decoder* to
> null.
> *DFSStripedInputStream.readWithStrategy* catches the IOException and retries
> calling *readOneStripe* again, which throws a NullPointerException in the
> call to *resetCurStripeBuffer* because the *decoder* is null when
> *useDirectBuffer* is called.
>
> With the retries that DFSStripedInputStream is doing in readWithStrategy,
> it's likely that either the StripedReader should not be closing the
> DFSStripedInputStream, or the closed check needs to happen again at the top
> of the retry loop.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
