[
https://issues.apache.org/jira/browse/HBASE-28955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17896133#comment-17896133
]
Duo Zhang commented on HBASE-28955:
-----------------------------------
The problem is for complilation. There is only one protected constructor for
DFSOutputStream, and the parameters are different between hadoop2 and hadoop3.
There is no way for us to write a code which could compile for both hadoop3 and
hadoop2...
A possible way is to separate the DummyDFSOutputStream to a separated module,
and when compiling with hadoop2 we do not link it, or maybe we coud discuss
again whether we still want to support hadoop2 for newer 2.x releases?
Thanks.
> Improve lease renew for FanOutOneBlockAsyncDFSOutput
> ----------------------------------------------------
>
> Key: HBASE-28955
> URL: https://issues.apache.org/jira/browse/HBASE-28955
> Project: HBase
> Issue Type: Bug
> Components: Filesystem Integration, io
> Reporter: Duo Zhang
> Assignee: Duo Zhang
> Priority: Major
> Labels: pull-request-available
> Fix For: 3.0.0-beta-2
>
>
> When working with hadoop 3.4.x, we saw this in the stdout file
> {noformat}
> Exception in thread "LeaseRenewer:zhangduo@home"
> java.lang.NullPointerException: Cannot invoke
> "org.apache.hadoop.hdfs.DFSOutputStream.getNamespace()" because
> "outputStream" is null
> at org.apache.hadoop.hdfs.DFSClient.getNamespaces(DFSClient.java:596)
> at org.apache.hadoop.hdfs.DFSClient.renewLease(DFSClient.java:618)
> at
> org.apache.hadoop.hdfs.client.impl.LeaseRenewer.renew(LeaseRenewer.java:425)
> at
> org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:445)
> at
> org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$800(LeaseRenewer.java:77)
> at
> org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:336)
> at java.base/java.lang.Thread.run(Thread.java:840)
> {noformat}
> This is because in newer DFSClient implementation, we need to pass namespace
> when renewer lease so we can not just pass null as DFSOutputStream when
> calling DFSClient.beginFileLease. We should find a way to deal with it.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
