pvary commented on code in PR #14197:
URL: https://github.com/apache/iceberg/pull/14197#discussion_r2393875155
##########
core/src/main/java/org/apache/iceberg/io/BaseTaskWriter.java:
##########
@@ -115,20 +115,32 @@ public WriteResult complete() throws IOException {
protected abstract class BaseEqualityDeltaWriter implements Closeable {
private final StructProjection structProjection;
private final PositionDelete<T> positionDelete;
+ private final StructLike partitionKey;
+ private final boolean useDv;
Review Comment:
The main issue here is that Flink doesn't use the `FileWriterFactory` ATM.
This is mainly because Flink depends on the `BaseEqualityDeltaWriter`
functionality which keeps track of the updated records and either writes an
equality delete or a position delete. There is no similar functionality for the
`FileWriterFactory` API.
I would love to see Flink moving to the `FileWriterFactory` API, but then we
need to implement the complex `EqualityDeltaWriter` functionality above the
writers above it first. That's a bigger undertaking, and if we can find an
"easy" solution to use the TaskWriter API, then we might decide to move forward
with it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
