fallintoplace opened a new pull request, #1326: URL: https://github.com/apache/iceberg-go/pull/1326
Summary - write Hadoop metadata files with suffixes that match the configured metadata compression codec - load the actual discovered metadata file path instead of reconstructing an uncompressed-looking path - teach Hadoop metadata discovery to recognize plain, gzip, and zstd metadata files consistently Why Hadoop currently always writes metadata to vN.metadata.json even when write.metadata.compression-codec requests gzip or zstd. The table metadata reader infers decompression from the filename suffix, so compressed bytes behind a plain .metadata.json path can fail to reload. Using codec-aware filenames on write and carrying the discovered metadata path through LoadTable keeps create, commit, and reload consistent for plain, gzip, and zstd metadata. Testing - go test ./catalog/hadoop -run 'TestHadoopCatalogTestSuite/(TestMetadataFilePathForCompression|TestVersionPatternMatches|TestIsTableDirWithZstdMetadata|TestFindVersionGzipOnlyWithHint|TestFindVersionMixedGzipAndPlain|TestFindVersionZstdOnlyWithHint|TestFindVersionMixedZstdAndPlain|TestCreateTableGzipMetadata|TestCreateTableZstdMetadata|TestCommitTableGzipMetadata|TestCommitTableZstdMetadata)$' -count=1 - go test ./catalog/hadoop -count=1 - go test ./catalog/... -count=1 - go test ./... -run '^$' -count=1 - go run github.com/golangci/golangci-lint/v2/cmd/[email protected] run --timeout=10m ./catalog/hadoop/... - git diff --check -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
