drexler-sky commented on code in PR #18885:
URL: https://github.com/apache/iceberg/pull/18885#discussion_r4580816457
##########
core/src/main/java/org/apache/iceberg/SchemaUpdate.java:
##########
@@ -51,6 +51,16 @@
class SchemaUpdate implements UpdateSchema {
private static final Logger LOG =
LoggerFactory.getLogger(SchemaUpdate.class);
private static final int TABLE_ROOT_ID = -1;
+ private static final Set<String> COLUMN_PROPERTY_PREFIXES =
Review Comment:
Done, thanks. I moved the set to `TableProperties.COLUMN_PROPERTY_PREFIXES`.
Spark 4.2 `SparkCatalog` now uses it too, so `CREATE TABLE LIKE` also handles
the compression codec/level prefixes that its old copy was missing.
##########
core/src/main/java/org/apache/iceberg/SchemaUpdate.java:
##########
@@ -506,22 +506,33 @@ private TableMetadata
applyChangesToMetadata(TableMetadata metadata) {
// Transform the metrics if they exist
if (base != null && base.properties() != null) {
- Schema newSchema = newMetadata.schema();
- List<String> deletedColumns =
-
deletes.stream().map(schema::findColumnName).collect(Collectors.toList());
- Map<String, String> renamedColumns =
- updates.keySet().stream()
- .filter(id -> !addedNameToId.containsValue(id)) // remove added
columns
- .filter(id ->
!schema.findColumnName(id).equals(newSchema.findColumnName(id)))
- .collect(Collectors.toMap(schema::findColumnName,
newSchema::findColumnName));
+ Map<Integer, String> newIdToName = newMetadata.schema().idToName();
+ List<String> deletedColumns = Lists.newArrayList();
+ Map<String, String> renamedColumns = Maps.newHashMap();
+ // compare all fields so that nested fields of a deleted or renamed
struct are included
+ schema
+ .idToName()
Review Comment:
Done, thanks for the suggestion. I added
`PropertyUtil.applySchemaChanges(properties, oldSchema, newSchema)`. It only
looks at properties with a column property prefix: for each one, it finds the
field ID in the old schema and its name in the new schema, then renames or
drops the key. Tables without column properties skip the work, and renamed or
dropped parent structs are handled because the key already names the leaf
column.
`SchemaUpdate` and Spark 4.2 `SparkCatalog` both use the new method. Since
nothing calls the old `applySchemaChanges(properties, deletedColumns,
renamedColumns)` anymore, I deprecated it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]