SaadASTheDev opened a new pull request, #8541:
URL: https://github.com/apache/hbase/pull/8541
## Why
HBase's `ByteBuffDecompressor` interface enables block decompression
directly from/to
`ByteBuffer` objects, avoiding intermediate byte array copies. GZIP had no
implementation
of this interface — all GZIP decompression went through a byte array
round-trip regardless
of whether the underlying buffers were on-heap or off-heap.
## What
Adds `GzipByteBuffDecompressor`, a `ByteBuffDecompressor` for GZIP that
handles two cases:
- **Off-heap (direct ByteBuffers):** delegates to Hadoop's native
`ZlibDirectDecompressor`
with `GZIP_FORMAT`. Requires native zlib to be loaded; `canDecompress()`
returns false
if it isn't.
- **On-heap (heap ByteBuffers):** uses Java's `Inflater` in raw DEFLATE
(`nowrap`) mode,
skipping the 10-byte GZIP header manually and verifying the CRC32 and
ISIZE trailer
fields after inflation. The native `ZlibDirectDecompressor` requires a
stable native
memory address, which heap `ByteBuffer`s do not provide, making this
fallback necessary.
Also adds:
- `GzipHFileDecompressionContext` — carries per-context configuration (e.g.
whether
ByteBuff decompression is allowed), wired in via `reinit()`
- Input-length validation — rejects inputs shorter than the minimum valid
GZIP member
(header + trailer = 18 bytes)
## Testing
- Off-heap → off-heap decompression
- On-heap → on-heap decompression
- CRC32 mismatch detection
- ISIZE mismatch detection
- Input too short
- Output buffer too small
- `canDecompress()` guard logic
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
