rdblue opened a new pull request, #18544:
URL: https://github.com/apache/iceberg/pull/18544
This adds a `MutableBitmap` interface to expose a `set` method for modifying
a bitmap and a mutable implementation of Mumbling.
The mutable bitmap implementation is based on a read-only bitmap. The
mutable class keeps a map of "overlay" `Container` objects that track the
contents of containers that have been modified, either as a sparse ordered list
of local positions or as a dense 32-byte bitmap. When serializing the bitmap,
unchanged regions are copied from the original base bitmap and overlay
containers are written in between.
This also uses and updates `ByteBuffers` methods. When using the unsigned
little-endian methods, I found that they depend on the byte order of the
buffer. This assumption works in the Variant package where these were first
added, but not in general.
Test plan:
- Validates pass-through to original bitmap
- Validate overlay works for set bits
- set bits override overlay
- set bits show up in isSet
- setting bits in reverse order works if the byte representation is
negative (i.e. 254)
- Serialization
- changing an existing container
- adding containers past the original bitmap end
- updating an empty bitmap
- sparse containers
- dense containers
- Correct cardinality with all cases
- Sparse overlay container
- Dense overlay container
- Conversion from sparse to dense
- Empty mutable bitmap
- Random test
This also includes but does not run a size test similar to the original Rust
implementation's that simulates binary pages of bitmaps to check compression
and correctness.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
