hectar-glitches opened a new pull request, #1447:
URL: https://github.com/apache/iceberg-go/pull/1447
## Scope:
Landed the first slice of the encryption package per my earlier plan with
the interfaces + no-op impl + in-memory KMS. Ready for review. EncryptingFileIO
and manifest wiring intentionally deferred to follow-up PRs pending the
key_metadata routing decision.
### Public API:
'EncryptionManager': coordination interface: 'NewEncryptedOutputFile(ctx,
writer, keyID)' and 'NewDecryptedInputFile(ctx, file, keyMetadata)'
'KeyManagementClient': KMS interface: 'WrapKey' / 'UnwrapKey' /
'SupportsKeyGeneration' / 'GenerateKey'
'EncryptionKeyMetadata':opaque '[]byte' type for the per-file blob stored in
'DataFile.KeyMetadata' / 'ManifestFile.KeyMetadata' /
'StatisticsFile.KeyMetadata'
'EncryptedInputFile' / 'EncryptedOutputFile': file wrappers exposing
'KeyMetadata()'; names match the Java 'org.apache.iceberg.encryption' types
'NativeEncryptionInputFile' / 'NativeEncryptionOutputFile': extension
interfaces returning 'any' for format-native crypto properties (so
'parquet.File{En,De}cryptionProperties' don't force a parquet import into a
format-agnostic package)
'PlaintextEncryptionManager': no-op impl for unencrypted tables; matches
Java's 'PlaintextEncryptionManager' (silently ignores 'keyID', passes bytes
through)
'InMemoryKeyManagementClient': AES-GCM wrap/unwrap over an in-memory KEK
map; documented as testing only
Sentinel errors: 'ErrUnknownKeyID', 'ErrInvalidKeyLength',
'ErrCiphertextTooShort', 'ErrAuthenticationFailed' (matches the 'Err*'
convention used in 'io', 'catalog', root 'errors.go')
### Tests
'PlaintextEncryptionManager' round-trip (data & key metadata pass-through)
'InMemoryKeyManagementClient': wrap/unwrap round-trip, non-deterministic
nonces, tamper detection, cross-key confusion, ciphertext-too-short, KEK
overwrite, 'AddKey' input copy isolation, concurrent access via 'errgroup'
(with race detector), key generation + round-trip, invalid length paths — all
using 'errors.Is' and 't.Context()'
### Deferred to follow-ups (per our thread)
'EncryptingFileIO' wrapper — needs the 'key_metadata' routing decision
'encryption.Register' / 'LoadKMS' registry for 'encryption.kms-type' /
'encryption.kms-impl' catalog properties
Bridge between 'table.EncryptionKey.EncryptedKeyMetadata' (base64 'string')
and 'EncryptionKeyMetadata' ('[]byte')
Parquet / Manifest / Puffin native encryption wiring
V3-only gating in table format
### Notes
'EncryptionKeyMetadata' is a defined type ('type EncryptionKeyMetadata
[]byte'), not a type alias. Every call site pulling 'KeyMetadata() []byte' from
a manifest or data file will need an explicit conversion
('encryption.EncryptionKeyMetadata(f.KeyMetadata())'). If the group prefers a
type alias instead, it's a one-line change here before anything downstream is
wired.
'NativeEncryptionInputFile' / 'NativeEncryptionOutputFile' return 'any' for
format-native properties. This avoids a parquet import in a format-agnostic
package as callers type-assert to '*parquet.FileDecryptionProperties' etc.
Happy to discuss alternatives.
'InMemoryKeyManagementClient' uses AES-GCM for key wrapping (matching the
Java standard KMS). It is exported (not test-only file) so other packages can
use it in their own tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
