vbhanuchander-lang commented on PR #17071:
URL: https://github.com/apache/iceberg/pull/17071#issuecomment-5274570961
@eye-gu this was closed by the stale bot, not on merit — your fix is correct
and your test is good.
I independently verified the whole thing before finding your PR, and I want
to record that here so
it is easier for a committer to act on:
- `lval >> (Integer.SIZE - 1)` leaves exponent bits in the low 32 bits of
the mask. For `1.0d` the
mask is `0x800000007fe00000` instead of `0x8000000000000000`, so the low
bytes get corrupted.
- Ordering is still correct on the high bytes, so it only breaks when two
values agree on their top
31 bits. There it inverts: all 2016 pairs from `1.0d + i * 2^-30` encode
backwards, and 48 of the
496 pairs among 32 consecutive float bit patterns above `1.0f` — the
smallest being `1.0f` and
`Math.nextUp(1.0f)`.
- Why CI never caught it: `testFloatOrdering`/`testDoubleOrdering` draw from
`nextFloat()`/`nextDouble()`. I replayed the exact `Random(42)` sequence —
**none of the 100 000
pairs in either test share bits 63..33**, so neither test ever reaches the
corrupted low bytes.
- No compatibility concern: the encoding is never persisted.
`SparkZOrderFileRewriteRunner` adds it
as the temporary `ICEZVALUE` column, sorts on it, and drops it.
I opened #17628 before I spotted this PR — that was my mistake, and I have
credited you in its
commit message and borrowed your boundary pairs with attribution. **I would
rather yours merged
than mine.** The bot's message says you can revive it at any time by
commenting; if you do, I will
close #17628 immediately and you are welcome to any of my extra tests.
Also worth flagging for whoever picks this up: #17130 is an auto-generated
PR for the same issue
that is still open and contains the production fix with no test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
