zeroshade commented on PR #1191: URL: https://github.com/apache/iceberg-go/pull/1191#issuecomment-4694968108
Thanks for the fix — the positional-alignment bug checks out, and this faithfully mirrors Java's `PartitionSpec.partitionType()` (the `UnknownType` placeholder for dropped sources, introduced in apache/iceberg#11868). One follow-up worth tracking (non-blocking for this PR): we now have two divergent ways to build a partition struct from a `(spec, schema)` pair: - `PartitionSpec.PartitionType()` — drops fields whose source column is missing, compacting positions - the new `manifestPartitionFields()` — keeps every field, placeholding dropped sources with `UnknownType` Scoping the fix to a local helper here is the right call: changing `PartitionType()` itself would ripple through its other ~10 callers (partition-path generation, codecs, snapshot producers, fanout writers), so keeping the blast radius small is correct. That said, the divergence is a latent footgun — anyone who later reaches for `PartitionType()` in a context that indexes a positional structure (e.g. a manifest `FieldSummary` slice) will silently reintroduce exactly this bug. Two small suggestions: 1. Add a short doc note on `PartitionType()` stating that it compacts dropped-source fields and therefore must not be used to index positional partition summaries. 2. File a follow-up to converge `PartitionType()` onto Java's `UnknownType` behavior so there's a single source of truth (separate PR, given the blast radius). For reference, the data-file partition path (`buildPartitionEvaluator` → `GetPartitionRecord`) is unaffected, since `DataFile.Partition()` is keyed by field ID rather than by position. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
