kevinjqliu commented on code in PR #1569:
URL: https://github.com/apache/iceberg-rust/pull/1569#discussion_r2532671357
##########
crates/integrations/datafusion/src/physical_plan/expr_to_predicate.rs:
##########
@@ -214,18 +218,46 @@ fn scalar_value_to_datum(value: &ScalarValue) ->
Option<Datum> {
ScalarValue::LargeUtf8(Some(v)) => Some(Datum::string(v.clone())),
ScalarValue::Date32(Some(v)) => Some(Datum::date(*v)),
ScalarValue::Date64(Some(v)) => Some(Datum::date((*v / MILLIS_PER_DAY)
as i32)),
+ ScalarValue::TimestampSecond(Some(v), tz) => {
+ interpret_timestamptz_micros(v.checked_mul(MICROS_PER_SECOND)?,
tz.as_deref())
+ }
+ ScalarValue::TimestampMillisecond(Some(v), tz) => {
+
interpret_timestamptz_micros(v.checked_mul(MICROS_PER_MILLISECOND)?,
tz.as_deref())
+ }
+ ScalarValue::TimestampMicrosecond(Some(v), tz) => {
+ interpret_timestamptz_micros(*v, tz.as_deref())
+ }
+ ScalarValue::TimestampNanosecond(Some(v), Some(_)) =>
Some(Datum::timestamptz_nanos(*v)),
+ ScalarValue::TimestampNanosecond(Some(v), None) =>
Some(Datum::timestamp_nanos(*v)),
Review Comment:
Iceberg treats timestamp with and without timestamp as 2 distinct primitive
types.
https://iceberg.apache.org/spec/#primitive-types
without timezone should map to Iceberg `timestamp`
with timezone should map to Iceberg `timestamptz`
furthermore Iceberg timestamptz is represented in UTC. So when converting
scalar timestamp value to Iceberg timestamptz, we would have to normalize to
UTC first
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
