singhpk234 commented on code in PR #14645:
URL: https://github.com/apache/iceberg/pull/14645#discussion_r2550267037
##########
core/src/test/java/org/apache/iceberg/puffin/TestPuffinWriter.java:
##########
@@ -86,6 +98,37 @@ public void testWriteMetricDataCompressedZstd() throws
Exception {
testWriteMetric(ZSTD, "v1/sample-metric-data-compressed-zstd.bin");
}
+ @ParameterizedTest
+ @ValueSource(booleans = {true, false})
+ public void testFileSizeCalculation(boolean isEncrypted) throws Exception {
+ final OutputFile outputFile;
+
+ if (isEncrypted) {
+ File testFile = temp.resolve("test" + System.nanoTime()).toFile();
+ Random random = new Random();
+ byte[] key = new byte[16];
+ random.nextBytes(key);
+ byte[] aadPrefix = new byte[16];
+ random.nextBytes(aadPrefix);
+ outputFile = new AesGcmOutputFile(Files.localOutput(testFile), key,
aadPrefix);
+ } else {
+ outputFile = new InMemoryOutputFile();
+ }
+
+ PuffinWriter writer = Puffin.write(outputFile).build();
+ writer.write(
+ new Blob(
+ "blob",
+ ImmutableList.of(1),
+ 2,
+ 1,
+ ByteBuffer.wrap("blob".getBytes()),
+ null,
+ ImmutableMap.of()));
+ writer.close();
+ assertThat(writer.length()).isEqualTo(isEncrypted ? 158L : 122L);
Review Comment:
It would be nice (if not covered already) that we can check we can read this
file post write
##########
core/src/test/java/org/apache/iceberg/puffin/TestPuffinWriter.java:
##########
@@ -86,6 +98,37 @@ public void testWriteMetricDataCompressedZstd() throws
Exception {
testWriteMetric(ZSTD, "v1/sample-metric-data-compressed-zstd.bin");
}
+ @ParameterizedTest
+ @ValueSource(booleans = {true, false})
Review Comment:
minor : can avoid if checks on length
```
@CsvSource({
"true, 156L", // Test case 1: isEncrypted=true, expectedSize=156L
"false, 122L" // Test case 2: isEncrypted=false, expectedSize=122L
})
```
##########
core/src/main/java/org/apache/iceberg/puffin/PuffinWriter.java:
##########
@@ -144,7 +142,8 @@ public void finish() throws IOException {
long footerOffset = outputStream.getPos();
writeFooter();
this.footerSize = Optional.of(Math.toIntExact(outputStream.getPos() -
footerOffset));
- this.fileSize = Optional.of(outputStream.getPos());
+ outputStream.close();
Review Comment:
seems like the close() can trigger a write to the outputStream and post
writer wrote the footer
https://github.com/apache/iceberg/blob/fb52fdef4fdcaa3cb5779e4d5b1b9f1ea7522e73/core/src/main/java/org/apache/iceberg/encryption/AesGcmOutputStream.java#L162
wondering we should update the description that this patch additionally
includes this fix too ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
