rdblue commented on code in PR #6762:
URL: https://github.com/apache/iceberg/pull/6762#discussion_r1421850518
##########
spark/v3.4/spark/src/main/java/org/apache/iceberg/spark/source/SparkAppenderFactory.java:
##########
@@ -175,15 +181,15 @@ public FileAppender<InternalRow> newAppender(OutputFile
file, FileFormat fileFor
.build();
case AVRO:
- return Avro.write(file)
+ return Avro.write(file.encryptingOutputFile())
Review Comment:
@ggershinsky, it seems to me that with the AES GCM streams set up, Avro
encryption would also work, right? In fact, although the
`StandardEncryptionManager` is not used unless the format is Parquet, I think
you can still request a format in individual writes. Those would work and use
AES GCM stream encryption.
I think we need to change how we prevent Avro and ORC encryption. Instead of
doing the check when creating the encryption manager, it should be done here
instead. What I would do is add `Avro.write(EncryptingOutputStream)` and
`ORC.write(EncryptingOutputStream)` and have them throw
`UnsupportedOperationException` unless the key metadata is null. That will
prevent AES GCM from being used until we want to add the feature.
We should also consider whether this will just work for Avro files!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
