szehon-ho commented on code in PR #10020:
URL: https://github.com/apache/iceberg/pull/10020#discussion_r1540218841
##########
core/src/main/java/org/apache/iceberg/PositionDeletesTable.java:
##########
@@ -278,28 +344,38 @@ public void close() throws IOException {
@Override
public CloseableIterator<ScanTask> iterator() {
+ // Partition filter by base table filter
Expression partitionFilter =
Projections.inclusive(spec,
isCaseSensitive()).project(baseTableFilter);
- // Filter partitions
+ // Read manifests (use original table's partition ids to
de-serialize partition values)
CloseableIterable<ManifestEntry<DeleteFile>> deleteFileEntries =
- ManifestFiles.readDeleteManifest(manifest, table().io(),
transformedSpecs)
+ ManifestFiles.readDeleteManifest(manifest, table().io(),
table().specs())
Review Comment:
Here is a bit tricky. We need to read manifests using the original
partition field ids (as that's how they are stored in the manifest file), so go
back to use the original table's partition spec which preserves the original
field ids.
But now we can no longer use the schema of that spec to bind the
user-provided partition filter, because the filter needs to bind the reasisgned
ones of the metadata table schema. So here we split out the row-filter from
ManifestReader and evaluate it separately outside. We do that below using
transformedSpecs (which have the reassigned partition field ids).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
