rraulinio opened a new pull request, #1191: URL: https://github.com/apache/iceberg-go/pull/1191
## Summary Fix manifest-level pruning when a partition source column has been dropped from the current schema. Manifest partition summaries are positional and correspond to the full partition field list in the manifest's partition spec. Before this change, `newManifestEvaluator` used `PartitionSpec.PartitionType(schema)`, which drops partition fields whose source column is missing from the provided schema. That can shift later partition fields to the wrong position. For example, if the manifest was written with partition fields `[q_part, p_part]`, and the current schema has dropped `q`, then `p_part` could be evaluated against `q_part`'s summary slot. This can incorrectly prune a manifest and return incomplete results. This change keeps all partition fields when building the manifest evaluator schema. Dropped-source fields are represented as `UnknownType` placeholders so later fields remain positionally aligned. ## Tests Added `TestManifestEvaluatorWithDroppedPartitionSource`, covering: - partition spec: `identity(q), identity(p)` - current schema: `q` dropped, `p` retained - row filter: `p = 2`, projected to `p_part` - manifest summaries: `q_part` at slot 0, `p_part` at slot 1 Before the fix, the manifest can be incorrectly pruned. After the fix, it remains eligible. ## Validation - `go test ./table -run TestManifestEvaluatorWithDroppedPartitionSource -count=1` - `go test ./table` - `go test ./...` Addressing https://github.com/apache/iceberg-go/issues/1190. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
