nastra commented on code in PR #6058:
URL: https://github.com/apache/iceberg/pull/6058#discussion_r1018846546
##########
spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/SparkCatalog.java:
##########
@@ -532,6 +537,24 @@ public final void initialize(String name,
CaseInsensitiveStringMap options) {
Splitter.on('.').splitToList(options.get("default-namespace")).toArray(new
String[0]);
}
}
+
+ EnvironmentContext.put(EnvironmentContext.ENGINE_NAME, "spark");
+ EnvironmentContext.put(
+ EnvironmentContext.ENGINE_VERSION,
sparkSession.sparkContext().version());
+ EnvironmentContext.put(CatalogProperties.APP_ID,
sparkSession.sparkContext().applicationId());
+ sparkSession
Review Comment:
This version here in `SparkCatalog` where we register a listener to look at
`SparkListenerSQLExecutionStart` events is more for a general purpose, as it
runs in Spark's thread pools.
`SparkScan` + scan reporting on the other hand runs in our own thread pools
and so we're populating the execution id in `SparkScan` and then
extracting/using it in for the scan report.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
