mxm commented on code in PR #16044:
URL: https://github.com/apache/iceberg/pull/16044#discussion_r3118659502
##########
flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/TableSerializerCache.java:
##########
@@ -120,7 +122,11 @@ private class SerializerInfo {
}
private void update() {
- Table table =
catalogLoader.loadCatalog().loadTable(TableIdentifier.parse(tableName));
+ if (catalog == null) {
+ catalog = catalogLoader.loadCatalog();
+ }
Review Comment:
Where are we closing this catalog?
##########
flink/v2.1/build.gradle:
##########
@@ -74,6 +74,15 @@ project(":iceberg-flink:iceberg-flink-${flinkMajorVersion}")
{
testImplementation(libs.flink21.test.utilsjunit) {
exclude group: 'junit'
}
+ testImplementation project(':iceberg-aws')
+ testImplementation(platform(libs.awssdk.bom))
+ testImplementation "software.amazon.awssdk:s3"
+ testImplementation "software.amazon.awssdk:auth"
+ testImplementation "software.amazon.awssdk:kms"
+ testImplementation "software.amazon.awssdk:sts"
+ testImplementation libs.testcontainers
+ testImplementation libs.testcontainers.junit.jupiter
+ testImplementation libs.testcontainers.minio
Review Comment:
Do we really need all these dependencies to verify the fix?
##########
flink/v2.1/flink/src/test/java/org/apache/iceberg/flink/sink/dynamic/TestDynamicTableUpdateOperator.java:
##########
@@ -132,6 +139,85 @@ void testDynamicTableUpdateOperatorSchemaChange() throws
Exception {
assertThat(catalog.loadTable(table).schema().schemaId()).isEqualTo(output.schema().schemaId());
}
+ @Test
+ void testCloseOperatorWithNonCloseableCatalog() throws Exception {
+ DynamicTableUpdateOperator operator =
+ new DynamicTableUpdateOperator(
+ CATALOG_EXTENSION.catalogLoader(),
+ 10,
+ 1000,
+ 10,
+ TableCreator.DEFAULT,
+ CASE_SENSITIVE,
+ PRESERVE_COLUMNS);
+ operator.open(null);
+
+ DynamicRecordInternal input =
+ new DynamicRecordInternal(
+ TABLE,
+ "branch",
+ SCHEMA1,
+ GenericRowData.of(1),
+ PartitionSpec.unpartitioned(),
+ 42,
+ false,
+ Collections.emptySet());
+ operator.map(input);
+
+ // HadoopCatalog is not Closeable, so close should complete without error
+ operator.close();
+ }
Review Comment:
Not sure we need this test. There won't be a way to call close on a catalog
which doesn't implement Closeable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
