parthchandra opened a new issue, #6026: URL: https://github.com/apache/datafusion-comet/issues/6026
### What is the problem the feature request solves? When Comet runs an Iceberg scan natively (`CometIcebergNativeScanExec`), the Spark SQL UI does not show the scan metrics that plain Spark + Iceberg normally shows on the `BatchScan` node. All the Iceberg planning numbers show up as 0: - number of data manifests, scanned/skipped manifests - number of result/skipped data files - total data file size - total planning duration - all the delete-file counts and sizes The scan also does not report any "scan time", so there is no way to see how long the native read took. Vanilla Iceberg shows all of these (they come from Iceberg's Java planning), so users lose visibility when Comet takes over the scan. Expected: the native Iceberg scan node in the Spark UI shows the same planning numbers Iceberg-Java reports, plus a scan time. ### Describe the potential solution _No response_ ### Additional context _No response_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
