badalprasadsingh opened a new issue, #1222: URL: https://github.com/apache/iceberg-go/issues/1222
### Apache Iceberg version main (development) ### Please describe the bug π ### Description In Iceberg-Go, `iceberg.TruncateTransform.Transformer`for `StringType` slices the input by raw UTF-8 byte length instead of Unicode code points. The [**Iceberg spec**](https://iceberg.apache.org/spec/#truncate-transform-details) defines string truncate as truncation to *N Unicode characters*. For any non-ASCII input (CJK, emoji, accented Latin), Iceberg-Go: - produces output that may not be valid UTF-8 because the slice cuts mid-codepoint, and - routes rows to a different partition directory Example: `truncate(5, "δΈηζ¨ε₯½δΈη")` ### Proposed fix Walk the input by Unicode code points using `unicode/utf8`. Keep binary truncation byte-based. Something similar to what Iceberg-Java does in [**Truncate Transform**](https://github.com/apache/iceberg/blob/ccfec2b771c31ad7b9bd893fb3dc9630c3cf982f/api/src/main/java/org/apache/iceberg/transforms/Truncate.java). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
