karuppayya opened a new issue, #4332: URL: https://github.com/apache/datafusion-comet/issues/4332
### What is the problem the feature request solves? Comet's scan currently only takes static credentials. Iceberg-rust already supports a credential hook ([AwsCredentialLoad / CustomAwsCredentialLoader in iceberg-storage-opendal](https://github.com/apache/iceberg-rust/blob/main/crates/storage/opendal/src/resolving.rs#L171)). Comet is [not wired](https://github.com/apache/datafusion-comet/blob/main/native/core/src/execution/operators/iceberg_scan.rs#L211) to use it. We should expose a pluggable CometCredentialProvider SPI and connect it to that existing hook. ### Describe the potential solution Thread the Java credential provider already in use with Spark into Comet, and make JNI calls when needed to get creds. ### Additional context _No response_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
