mbutrovich commented on PR #3895:
URL:
https://github.com/apache/datafusion-comet/pull/3895#issuecomment-4281344180
Thanks for improving the Iceberg support, @karuppayya! I took a look at this
today:
The `findMethodInHierarchy` fix is correct, but it widens the set of scans
Comet intercepts. Previously, `getDeclaredMethod("current")` failed for
`InMemoryCatalog`'s operations class (where `current()` is inherited), causing
fallback to Spark. Now the reflection succeeds and Comet runs the scan natively.
The Iceberg CI failures (`TestSparkReaderDeletes`) happen because Comet's
native reader (iceberg-rust/arrow-rs) doesn't expose the `number of row deletes
applied` metric that these tests assert on. That metric is an Iceberg Java
reader concept.
The root cause is `CometScanRule.validateIcebergFileScanTasks` line 809:
```scala
if (scheme != null && !supportedSchemes.contains(scheme))
```
`InMemoryCatalog` returns relative paths where `URI.getScheme` is `null`.
The null check lets them through. Fix: treat null scheme as unsupported so
these scans fall back.
```scala
if (scheme == null || !supportedSchemes.contains(scheme))
```
Same fix needed for the delete file scheme check on line 851.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
