[
https://issues.apache.org/jira/browse/HADOOP-10047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gopal V updated HADOOP-10047:
-----------------------------
Description:
With the Zero-Copy reads in HDFS (HDFS-5260), it becomes important to perform
all I/O operations without copying data into byte[] buffers or other buffers
which wrap over them.
This is a proposal for adding a DirectDecompressor interface to the
io.compress, to indicate codecs which want to surface the direct buffer layer
upwards.
The implementation should work with direct heap/mmap buffers and cannot assume
.array() availability.
was:
With the Zero-Copy reads in HDFS (HDFS-5260), it becomes important to perform
all I/O operations without copying data into byte[] buffers or other buffers
which wrap over them.
This is a proposal for adding new DirectCompressor and DirectDecompressor
interfaces to the io.compress, to indicate codecs which want to surface the
direct buffer layer upwards.
The implementation may or may not copy the buffers passed in, but should work
with direct heap/mmap buffers and cannot assume .array() availability.
> Add a directbuffer Decompressor API to hadoop
> ---------------------------------------------
>
> Key: HADOOP-10047
> URL: https://issues.apache.org/jira/browse/HADOOP-10047
> Project: Hadoop Common
> Issue Type: New Feature
> Components: io
> Reporter: Gopal V
> Assignee: Gopal V
> Labels: compression
> Attachments: DirectCompressor.html, DirectDecompressor.html,
> HADOOP-10047-redo-WIP.patch, HADOOP-10047-WIP.patch,
> HADOOP-10047-with-tests.patch
>
>
> With the Zero-Copy reads in HDFS (HDFS-5260), it becomes important to perform
> all I/O operations without copying data into byte[] buffers or other buffers
> which wrap over them.
> This is a proposal for adding a DirectDecompressor interface to the
> io.compress, to indicate codecs which want to surface the direct buffer layer
> upwards.
> The implementation should work with direct heap/mmap buffers and cannot
> assume .array() availability.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
