nastra commented on PR #11262: URL: https://github.com/apache/iceberg/pull/11262#issuecomment-2410267744
> Do you think it's a good idea if I change ParamerizedTest as you suggested, and also add the legacy mode flag to Spark? I took a closer look to see why this happens. It turns out that https://github.com/apache/iceberg/blob/2025e79905488c724a180e25b91b487aeb848811/spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/ExtensionsTestBase.java#L46-L59 is re-using the spark session from `TestBase` due to `getOrCreate()`. I think we haven't seen the issue before, because `TestBase.startMetastoreAndSpark()` hasn't been called in previous JUnit versions. I would suggest to add this diff to `ExtensionstestBase`: ``` --- a/spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/ExtensionsTestBase.java +++ b/spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/ExtensionsTestBase.java @@ -43,6 +43,8 @@ public abstract class ExtensionsTestBase extends CatalogTestBase { metastore.start(); TestBase.hiveConf = metastore.hiveConf(); + TestBase.spark.close(); + TestBase.spark = SparkSession.builder() .master("local[2]") ``` Btw you might also need to update https://github.com/apache/iceberg/blob/892e47cd329b271517dd6ac5cf6b2591092b1146/api/src/test/java/org/apache/iceberg/ParameterizedTestExtension.java#L228 to do a bottom up search. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
