rawman opened a new pull request, #12551: URL: https://github.com/apache/iceberg/pull/12551
Iceberg didn't support exporting schema to Glue catalog in the past, which I believe was introduced in https://github.com/apache/iceberg/pull/3048. It means it is perfectly fine to have an Iceberg table in Glue that doesn't have defined StorageDescriptor in https://docs.aws.amazon.com/cli/latest/reference/glue/get-table.html response. Unfortunately the current Iceberg version requires this information to be present, otherwise the table metadata conversion calls fail silently with `NullPointerException` ``` java.lang.NullPointerException: Cannot invoke "software.amazon.awssdk.services.glue.model.StorageDescriptor.columns()" because the return value of "software.amazon.awssdk.services.glue.model.Table.storageDescriptor()" is null at org.apache.iceberg.aws.glue.IcebergToGlueConverter.setTableInputInformation(IcebergToGlueConverter.java:268) at org.apache.iceberg.aws.glue.GlueTableOperations.lambda$persistGlueTable$1(GlueTableOperations.java:321) at software.amazon.awssdk.utils.builder.SdkBuilder.applyMutation(SdkBuilder.java:61) at org.apache.iceberg.aws.glue.GlueTableOperations.persistGlueTable(GlueTableOperations.java:319) at org.apache.iceberg.aws.glue.GlueTableOperations.doCommit(GlueTableOperations.java:160) at org.apache.iceberg.BaseMetastoreTableOperations.commit(BaseMetastoreTableOperations.java:125) at org.apache.iceberg.SnapshotProducer.lambda$commit$4(SnapshotProducer.java:461) at org.apache.iceberg.util.Tasks$Builder.runTaskWithRetry(Tasks.java:413) at org.apache.iceberg.util.Tasks$Builder.runSingleThreaded(Tasks.java:219) at org.apache.iceberg.util.Tasks$Builder.run(Tasks.java:203) at org.apache.iceberg.util.Tasks$Builder.run(Tasks.java:196) ``` This preventing saving the schema after updating to a newer version of the library. This MR fix this issue by checking if StorageDescriptor exists first. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
