xanderbailey opened a new pull request, #4308: URL: https://github.com/apache/iceberg-python/pull/4308
<!-- Thanks for opening a pull request! --> Working towards https://github.com/apache/iceberg-python/issues/3222 <!-- In the case this PR will resolve an issue, please replace ${GITHUB_ISSUE_ID} below with the actual Github issue id. --> <!-- Closes #${GITHUB_ISSUE_ID} --> # Rationale for this change Now that we can read encrypted manifest lists (#4159), we need to read the manifests they point to. - A manifest list entry's `key_metadata` holds the manifest's plaintext key metadata (key, AAD prefix and file length). That's safe because the manifest list holding it is itself encrypted, so reading a manifest doesn't need the `EncryptionManager` or the KMS. - `ManifestFile.fetch_manifest_entry` decrypts the manifest when `key_metadata` is set, matching iceberg-rust's `ManifestReader` ([8/N] apache/iceberg-rust#2586). Every manifest read goes through it, so scans, `inspect`, validation and purge pick this up without changes. This depends on #4296. iceberg-rust's manifest writer leaves out `avro.codec`, and we currently read a missing codec as gzip (#4295), so the iceberg-rust test below fails until that lands. ## Are these changes tested? Yes. An encrypted v3 manifest written by iceberg-rust is checked in under `tests/table/encryption`, and reading it returns its entries. There's also a round trip of an encrypted manifest, checking that a wrong key fails and that the encrypted bytes don't read as plaintext. ## Are there any user-facing changes? No. Nothing changes for unencrypted tables. <!-- In the case of user-facing changes, please add the changelog label. --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
