ayushtkn commented on code in PR #16158:
URL: https://github.com/apache/iceberg/pull/16158#discussion_r3240141803
##########
spark/v4.1/spark/src/test/java/org/apache/iceberg/spark/sql/TestTableEncryption.java:
##########
@@ -212,16 +212,22 @@ public void testInsertAndDelete() {
expected,
sql("SELECT * FROM %s ORDER BY id", tableName));
- sql("DELETE FROM %s WHERE id < 4", tableName);
+ sql("DELETE FROM %s WHERE id < 5", tableName);
- expected = ImmutableList.of(row(4L, "d", 4.0F), row(5L, "e", 5.0F),
row(6L, "f", Float.NaN));
+ expected = ImmutableList.of(row(5L, "e", 5.0F), row(6L, "f", Float.NaN));
assertEquals(
"Should return all expected rows",
expected,
sql("SELECT * FROM %s ORDER BY id", tableName));
}
+ @TestTemplate
+ public void testInsertAndDeleteMOR() {
+ sql("ALTER TABLE %s SET TBLPROPERTIES
('write.delete.mode'='merge-on-read')", tableName);
+ testInsertAndDelete();
Review Comment:
Can you check, I did this:
```
ayushsaxena@Q3NW54Y0C5 iceberg % git reset HEAD~1
Unstaged changes after reset:
M core/src/main/java/org/apache/iceberg/DVUtil.java
M core/src/main/java/org/apache/iceberg/deletes/BaseDVFileWriter.java
M
spark/v4.0/spark/src/test/java/org/apache/iceberg/spark/sql/TestTableEncryption.java
M
spark/v4.1/spark/src/test/java/org/apache/iceberg/spark/sql/TestTableEncryption.java
ayushsaxena@Q3NW54Y0C5 iceberg % git checkout
core/src/main/java/org/apache/iceberg/DVUtil.java
core/src/main/java/org/apache/iceberg/deletes/BaseDVFileWriter.java
Updated 2 paths from the index
ayushsaxena@Q3NW54Y0C5 iceberg % ./gradlew clean
:iceberg-spark:iceberg-spark-4.1_2.13:test --tests
"org.apache.iceberg.spark.sql.TestTableEncryption.testInsertAndDeleteMOR"
Configuration on demand is an incubating feature.
> Task :iceberg-api:testJar
file '/Users/ayushsaxena/code/iceberg/build/iceberg-build.properties' will
be copied to 'iceberg-build.properties', overwriting file
'/Users/ayushsaxena/code/iceberg/api/build/resources/test/iceberg-build.properties',
which has already been copied there.
> Task :iceberg-spark:iceberg-spark-4.1_2.13:compileTestJava
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
> Task :iceberg-spark:iceberg-spark-4.1_2.13:test
TestTableEncryption > testInsertAndDeleteMOR() > catalogName = testhive,
implementation = org.apache.iceberg.spark.SparkCatalog, config = {type=hive,
default-namespace=default,
encryption.kms-impl=org.apache.iceberg.encryption.UnitestKMS} FAILED
org.apache.spark.SparkException: Job aborted due to stage failure: Task
0 in stage 9.0 failed 1 times, most recent failure: Lost task 0.0 in stage 9.0
(TID 9) (192.168.29.167 executor driver): java.lang.IllegalStateException: Null
key metadata buffer
at
org.apache.iceberg.encryption.StandardKeyMetadata.castOrParse(StandardKeyMetadata.java:111)
at
org.apache.iceberg.encryption.StandardEncryptionManager$StandardDecryptedInputFile.keyMetadata(StandardEncryptionManager.java:302)
at
org.apache.iceberg.encryption.StandardEncryptionManager$StandardDecryptedInputFile.decrypted(StandardEncryptionManager.java:313)
at
org.apache.iceberg.encryption.StandardEncryptionManager$StandardDecryptedInputFile.location(StandardEncryptionManager.java:333)
at
org.apache.iceberg.encryption.EncryptingFileIO.bulkDecrypt(EncryptingFileIO.java:71)
at
org.apache.iceberg.spark.source.BaseReader.inputFiles(BaseReader.java:186)
at
org.apache.iceberg.spark.source.BaseReader.getInputFile(BaseReader.java:180)
at
org.apache.iceberg.spark.source.BatchDataReader.open(BatchDataReader.java:108)
at
org.apache.iceberg.spark.source.BatchDataReader.open(BatchDataReader.java:43)
at
org.apache.iceberg.spark.source.BaseReader.next(BaseReader.java:144)
at
org.apache.spark.sql.execution.datasources.v2.PartitionIterator.hasNext(DataSourceRDD.scala:148)
at
org.apache.spark.sql.execution.datasources.v2.MetricsIterator.hasNext(DataSourceRDD.scala:186)
at
org.apache.spark.sql.execution.datasources.v2.DataSourceRDD$$anon$1.$anonfun$hasNext$1(DataSourceRDD.scala:72)
at
org.apache.spark.sql.execution.datasources.v2.DataSourceRDD$$anon$1.$anonfun$hasNext$1$adapted(DataSourceRDD.scala:72)
at scala.Option.exists(Option.scala:406)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
