and124578963 commented on issue #13174: URL: https://github.com/apache/iceberg/issues/13174#issuecomment-2921827619
Hey @vanshb03, Typically, I use Iceberg 1.7.2, and my configuration with s3a:// works well. I found the following catalog-related properties in spark-defaults.conf: ``` spark.sql.catalog.iceberg: org.apache.iceberg.spark.SparkCatalog spark.sql.catalog.iceberg.type: hive spark.sql.catalogImplementation=hive spark.sql.catalog.spark_catalog=org.apache.iceberg.spark.SparkSessionCatalog spark.sql.catalog.spark_catalog.type=hive ``` Setting `spark.sql.catalog.iceberg.catalog-impl=org.apache.iceberg.spark.SparkCatalog` leads to some unexpected behavior. For instance, Spark starts 4 executors instead of 1. I cannot use s3:// because I'm using Hive Metastore, which only supports s3a://. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
