Gabriel39 opened a new pull request, #68393:
URL: https://github.com/apache/doris/pull/68393
### What problem does this PR solve?
Iceberg partition evolution can reuse a partition name for a different field
ID. For example, dropping an identity field, adding it under an alias, then
renaming it back while adding another transform leaves two historical field IDs
with the same name. Queries against `$partitions`, `$files`, and other metadata
tables then fail with `Invalid schema: multiple fields for name
partition.record_key`.
Vendor Iceberg 1.11.0's `Partitioning` using the existing connector override
pattern and change only unified partition field naming. Keep the newest spec's
preferred name and suffix conflicting older fields with their field IDs,
reserving existing names to prevent secondary collisions. Preserve all
historical fields, IDs, types, and ordering; do not modify table metadata. The
upstream implementation otherwise remains unchanged, with a scoped Checkstyle
exemption matching the existing vendored Iceberg classes.
### Release note
Fix Iceberg metadata table queries after partition evolution reuses a name
for different partition field IDs. Conflicting historical fields receive unique
names in the metadata table's partition struct.
### Check List (For Author)
- Test
- [x] Unit Test
- New coverage for conflicting names, suffix collisions, ordinary
evolution, v1 dropped fields, metadata task serialization, and partition
predicates.
- Reproduced the failure before the fix with the current Iceberg
dependency.
- FE Checkstyle passed; 273 tests passed across
`IcebergPartitionNameConflictTest`, `IcebergPartitionUtilsTest`,
`IcebergScanPlanProviderTest`, and `IcebergConnectorMetadataSysTableTest`.
- Behavior changed:
- [x] Yes. Previously failing metadata queries can expose both
historical and current partition fields.
- Does this need documentation?
- [x] No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
