smaheshwar-pltr opened a new issue, #14764: URL: https://github.com/apache/iceberg/issues/14764
### Feature Request / Improvement https://github.com/apache/iceberg/pull/14763 shows a failing test. The Spark purge codepath and other maintenance procedures require finding reachable files, using metadata tables. This doesn't work for encrypted tables currently. Concretely, there are a couple of issues: 1. The [AllManifestsTableScan](https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/AllManifestsTable.java#L104) dispatches manifest lists reads to executors, however the `StandardEncryptionManager`'s fields required for that are not serializable 2. Finding content files requires reading manifests obtained from the manifest metadata tables [here](https://github.com/apache/iceberg/blob/4c0ad422624eb7e2b3b196a4ea77ee8ad83a69e9/spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/actions/BaseSparkAction.java#L156-L169), but the `ManifestFileBean` does not include encryption key metadata to permit decryption during the read ### Query engine None ### Willingness to contribute - [ ] I can contribute this improvement/feature independently - [ ] I would be willing to contribute this improvement/feature with guidance from the Iceberg community - [ ] I cannot contribute this improvement/feature at this time -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
