anoopj commented on code in PR #18108:
URL: https://github.com/apache/iceberg/pull/18108#discussion_r4009333330
##########
core/src/main/java/org/apache/iceberg/TrackedFileAdapters.java:
##########
@@ -549,6 +565,22 @@ public ManifestFile copy() {
}
}
+ // Manifests store partitions in the union of all specs' partition types,
but each file's
+ // partition must be read using its own spec's field order.
+ private static StructLike projectPartition(StructLike partition,
Types.StructType specType) {
+ if (partition == null) {
+ return PartitionData.EMPTY;
+ }
+
+ Types.StructType unionType = ((PartitionData)
partition).getPartitionType();
+ if (unionType.equals(specType)) {
+ return partition;
+ }
+
+ return new PartitionData(specType)
+ .copyFor(StructProjection.create(unionType, specType).wrap(partition));
Review Comment:
Note to reviewers: We are doing three object allocations here, which isn't
great. We can cache the first two (ie the `PartitionData` and
`StructProjection`), but that will require changing the factory into a stateful
class we can hang the cache on.
Is there a better option?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
