wombatu-kun commented on issue #17027: URL: https://github.com/apache/iceberg/issues/17027#issuecomment-4849648963
Traced the root cause on current `main`. `truncate`'s result type is the source type unchanged (`Truncate.getResultType`), and `TruncateUtil.truncateDecimal` truncates toward negative infinity with no bounds check, so the most-negative representable value can floor to one integer digit beyond the source precision: `decimal(7,3)` value `-9999.999` with width 10 becomes `-10000.000` (precision 8). The partition field is still typed `decimal(7,3)`, so `DecimalUtil.toReusedFixLengthBytes` rejects the value at write time and the append aborts. This was previously worked in #12969 (a follow-up to #12915), which was closed as stale without converging. The blocker is that any fix is a spec change: the spec currently pins truncate's result type to "Source type" and the decimal arithmetic to `v - (((v % W) + W) % W)`. Three directions were floated there, each a spec change: - Widen the result precision to `max(minTruncatedPrecision, P)` (#12969's approach). @RussellSpitzer noted this needs a spec change and that a fixed "+1 precision" is not always enough for large widths. - Clamp the transform output to the lowest/highest in-range value. @aykut-bozkurt preferred this because it also avoids incorrectly skipping files during filter pushdown. - Reject the offending partition spec at table-creation time (@RussellSpitzer / @aykut-bozkurt). It is also not decimal-specific. `int`/`long` truncate overflow the same way near their range limits (@nandorKollar's `Integer.MIN_VALUE` example on #12969), tracked separately in #13105, which was likewise closed as stale with the same "probably needs a spec change" conclusion. Given three prior stale threads on the same underlying problem, it would help to settle the direction before writing code: widen vs clamp vs reject, and decimal-only vs all numeric types. Since each option changes the spec, this likely needs a short dev@ discussion or a maintainer decision on #12969. Happy to implement once there is a direction. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
