rdblue commented on code in PR #16936:
URL: https://github.com/apache/iceberg/pull/16936#discussion_r4020861313
##########
core/src/main/java/org/apache/iceberg/TrackedFileAdapters.java:
##########
@@ -552,6 +609,511 @@ public ManifestFile copy() {
}
}
+ /** Shared base for content-file (DATA / EQUALITY_DELETES) write-direction
wrappers. */
+ abstract static class ContentTrackedFile<F extends ContentFile<F>>
+ implements TrackedFile, StructLike {
+ private final Types.StructType partitionType;
+ private final MapBackedContentStats statsWrapper;
+
+ private Tracking tracking;
+ private F file;
+ private StructProjection partition;
+ private ContentStats stats;
+
+ ContentTrackedFile(Schema writeSchema) {
+ Preconditions.checkArgument(writeSchema != null, "Invalid write schema:
null");
+ Types.StructType statsType = nestedStruct(writeSchema,
TrackedFile.CONTENT_STATS_ID);
+ this.partitionType = nestedStruct(writeSchema, TrackedFile.PARTITION_ID);
+ this.statsWrapper = statsType != null ? new
MapBackedContentStats(statsType) : null;
+ }
+
+ TrackedFile wrapWithTracking(F newFile, Tracking newTracking) {
Review Comment:
Since the purpose of this adapter is to adapt from `DataFile` to
`TrackedFile` and not to write, I don't think that this method makes sense.
This should only wrap the data file. The write wrapper will be responsible for
replacing `Tracking`.
The question I have is how we should handle converting `ManifestEntry` and
whether we want to pass that as a second argument here. First, I would not
convert `ManifestEntry` and then pass it in like this because it puts
additional responsibility on the caller and is misleading: the caller should
not use this method to replace or change `ManifestEntry`. As a result, I think
that the `ManifestEntry` variant should accept a `ManifestEntry` only and
convert the data file in it. That's easier to call because you can wrap an
entry that comes from `ManifestReader.entries`.
In addition, I think that we want a way to wrap `DataFile` independent of
the `ManifestEntry`, which is what will be used for data files passed in
through the API. That should set `Tracking` to null since the file has no
tracking info.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
