[
https://issues.apache.org/jira/browse/HADOOP-14872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Zhuge updated HADOOP-14872:
--------------------------------
Description:
Discovered in IMPALA-5909.
Opening an encrypted HDFS file returns a chain of wrapped input streams:
{noformat}
HdfsDataInputStream
CryptoInputStream
DFSInputStream
{noformat}
If an application such as Impala or HBase calls HdfsDataInputStream#unbuffer,
FSDataInputStream#unbuffer will be called:
{code:java}
try {
((CanUnbuffer)in).unbuffer();
} catch (ClassCastException e) {
throw new UnsupportedOperationException("this stream does not " +
"support unbuffering.");
}
{code}
If the {{in}} class does not implement CanUnbuffer, UOE will be thrown. If the
application is not careful, it can see tons of UOEs.
In comparison opening an non-encrypted HDFS file return this chain:
{noformat}
HdfsDataInputStream
DFSInputStream
{noformat}
DFSInputStream implements CanUnbuffer.
It is good for CryptoInputStream to implement CanUnbuffer for 2 reasons:
* Release buffer, cache, or any other resource when instructed
* Avoid the UOE described above. Applications may not handle the UOE very well.
was:
Discovered in IMPALA-5909.
CryptoInputStream extending FSDataInputStream should implement unbuffer method
* Release buffer and cache when instructed
* Avoid calling super unbuffer method that throws UOE. Applications may not
handle the UOE very well.
> CryptoInputStream should implement unbuffer
> -------------------------------------------
>
> Key: HADOOP-14872
> URL: https://issues.apache.org/jira/browse/HADOOP-14872
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs
> Affects Versions: 2.6.4
> Reporter: John Zhuge
> Assignee: John Zhuge
>
> Discovered in IMPALA-5909.
> Opening an encrypted HDFS file returns a chain of wrapped input streams:
> {noformat}
> HdfsDataInputStream
> CryptoInputStream
> DFSInputStream
> {noformat}
> If an application such as Impala or HBase calls HdfsDataInputStream#unbuffer,
> FSDataInputStream#unbuffer will be called:
> {code:java}
> try {
> ((CanUnbuffer)in).unbuffer();
> } catch (ClassCastException e) {
> throw new UnsupportedOperationException("this stream does not " +
> "support unbuffering.");
> }
> {code}
> If the {{in}} class does not implement CanUnbuffer, UOE will be thrown. If
> the application is not careful, it can see tons of UOEs.
> In comparison opening an non-encrypted HDFS file return this chain:
> {noformat}
> HdfsDataInputStream
> DFSInputStream
> {noformat}
> DFSInputStream implements CanUnbuffer.
> It is good for CryptoInputStream to implement CanUnbuffer for 2 reasons:
> * Release buffer, cache, or any other resource when instructed
> * Avoid the UOE described above. Applications may not handle the UOE very
> well.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
