Fokko opened a new pull request, #763: URL: https://github.com/apache/iceberg-rust/pull/763
This PR removes `SchemalessPartitionSpec` and `UnboundPartitionSpecField`. We could also combine `BoundPartitionSpec` and `UnboundPartitionSpec` if we like, but this is already quite a big change. From the spec: > The field-id property was added for each partition field in v2. > In v1, the reference implementation assigned field ids sequentially > in each spec starting at 1,000. See Partition Evolution for more details. > In v1, partition field IDs were not tracked, but were assigned sequentially > starting at 1000 in the reference implementation. This assignment caused > problems when reading metadata tables based on manifest files from multiple > specs because partition fields with the same ID may contain different data types. > For compatibility with old versions, the following rules are recommended for partition evolution in v1 tables: > - Do not reorder partition fields > - Do not drop partition fields; instead replace the field's transform with the void transform > - Only add partition fields at the end of the previous partition spec I think for simplicity, we should assign the field-IDs starting from 1000, and this will greatly simplify the objects that we need. Next to that, I also believe that users shouldn't have to worry about the field-IDs and that it should be kept internal to Iceberg-Rust. For the evolution of the partition spec, we should have something similar as Java and [PyIceberg](https://py.iceberg.apache.org/api/#partition-evolution), in particular for V1 tables, we have to take the rules above into account, otherwise, there is a serious issue of data-loss, or bricking a table. If we agree on this, I'm happy to implement that API. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
