rdblue commented on PR #11775: URL: https://github.com/apache/iceberg/pull/11775#issuecomment-3156720335
Following up on the bug described in [this comment](https://github.com/apache/iceberg/pull/11775#discussion_r2074715304) here instead of in that thread because the thread discusses solutions to handling nanos, not just the overflow problem. Here's a test that reproduces the problem: ```java @Test public void testStringToTimestampNanosLiteral() { Schema schema = new Schema( Types.NestedField.required(1, "id", Types.LongType.get()), Types.NestedField.optional(2, "ts", Types.TimestampNanoType.withoutZone())); PartitionSpec spec = PartitionSpec.builderFor(schema).identity("ts").build(); Expression expr = Expressions.equal("ts", "2022-07-26T12:13:14.123456789"); Expression projected = Projections.inclusive(spec).project(expr); Binder.bind(schema.asStruct(), projected); } ``` What's happening is the projection will first bind the original predicate because it needs a bound ID reference rather than a name reference. That ID is used to find the partition fields that can project the predicate. The binding process produces the expected `TimestampNanoLiteral(1658837594123456789L)`. The bug is in the `Identity` transform's projection code, which needs to produce a new predicate that is unbound and uses a reference for the partition field's name (the partition name does not have to match). When it [constructs the new unbound predicate](https://github.com/apache/iceberg/blob/e1e0a74/api/src/main/java/org/apache/iceberg/transforms/Identity.java#L152-L153), it passes the underlying value rather than the unchanged literal. Updating that line to pass the literal instead of the value fixes the problem because it doesn't lose the context that the value was already a nanosecond timestamp value. I'll open a PR to fix this. Hopefully it can make it into 1.10. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
