stevenzwu commented on code in PR #10200:
URL: https://github.com/apache/iceberg/pull/10200#discussion_r1583309420
##########
core/src/main/java/org/apache/iceberg/io/BaseTaskWriter.java:
##########
@@ -109,18 +112,34 @@ protected abstract class BaseEqualityDeltaWriter
implements Closeable {
private final StructProjection structProjection;
private RollingFileWriter dataWriter;
private RollingEqDeleteWriter eqDeleteWriter;
- private SortedPosDeleteWriter<T> posDeleteWriter;
+ private FileWriter<PositionDelete<T>, DeleteWriteResult> posDeleteWriter;
private Map<StructLike, PathOffset> insertedRowMap;
protected BaseEqualityDeltaWriter(StructLike partition, Schema schema,
Schema deleteSchema) {
+ this(partition, schema, deleteSchema, DeleteGranularity.PARTITION);
+ }
+
+ protected BaseEqualityDeltaWriter(
+ StructLike partition,
+ Schema schema,
+ Schema deleteSchema,
+ DeleteGranularity deleteGranularity) {
Review Comment:
I am also in favor of no config. It is a "good" internal impl/perf change.
In the data file rollover case, one smaller delete file every data file is
better than one larger delete file for multiple data files.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
