c-thiel commented on PR #491: URL: https://github.com/apache/iceberg-rust/pull/491#issuecomment-2270712857
> I have a few `ToDo`s in the code. Most of them are for tests in other modules that would fail with proper partition binding. > > If someone has insights on the following, it would also be very helpful: Should we compare transform.deduped_name to find redundant partitionings or the transformation itself? If we switch to deduped_name, we would not support year & month & day partitions on the same column, as it is always "time". > > deduped_name would probably be more correct for bucket transform, to disallow different bucket sizes on the same column. Java seems to use deduped_name: https://github.com/apache/iceberg/blob/4dbc7f578eee7ceb9def35ebfa1a4cc236fb598f/api/src/main/java/org/apache/iceberg/PartitionSpec.java#L420 > > Does java disalow year & month partitions on the same column? If so our tests are broken > > https://github.com/apache/iceberg-rust/blob/de450c80bbe5d7d7dcb919ded7015ca447d8edb1/crates/iceberg/src/expr/visitors/inclusive_projection.rs#L329-L356 I checked spark and it also fails: pyspark.errors.exceptions.captured.IllegalArgumentException: Cannot add redundant partition: 1000: ts_year: year(2) conflicts with 1001: ts_month: month(2) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
