gaborkaszab opened a new pull request, #18198:
URL: https://github.com/apache/iceberg/pull/18198
Changes this contains:
- More generic interface with parameters: - Iterable<FileScanTask> -
EncryptingFileIO
- Contructs InputFiles lazily when first requested
- EncryptionManager is wrapped into EncryptingFileIO input parameter
- Use EncryptingFileIO.bulkDecrypt() instead of EncryptionManager.decrypt()
- Throw if file location is not found
- Removed usage of old contructor and getInputFile(FileScanTask)
With this design, this class is generic enough to be wired into other
readers as a common implementation for creating InputFiles with bulk decryption
taken into account. Later, GenericReader and Spark's readers can swap to use
this.
One hidden improvement: Previously this class created InputFiles using
location + key metadata without the size. While this works, when it comes to
reading, it requires an extra RPC to get the size of the file. Now, with
bulkDecrypt(), the location + size + key metadata are used together saving an
extra RPC per file.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
