amogh-jahagirdar commented on code in PR #11948:
URL: https://github.com/apache/iceberg/pull/11948#discussion_r1928121891
##########
core/src/main/java/org/apache/iceberg/SnapshotProducer.java:
##########
@@ -282,6 +283,13 @@ public Snapshot apply() {
throw new RuntimeIOException(e, "Failed to write manifest list file");
}
+ Long addedRows = null;
+ Long lastRowId = null;
+ if (base.rowLineageEnabled()) {
+ addedRows = calculateAddedRows(manifests);
Review Comment:
It's an optimization so we can always just do this later (and arguably makes
it a bit harder to read the code) but instead of waiting until all the
manifests are written what if we set the addedRows as we add manifests to the
writer in the try with-resources-above
Something like
```
// remove the writer.addAll
for (ManifestFile manifest: manifestFIles) {
if (rowLineageEnabled()) {
if (manifest.snapshotId() == null || (manifest.snapshotId() ==
this.snapshotId) {
Preconditions.checkArgument(
manifest.addedRowsCount() != null,
"Cannot determine number of added rows in snapshot because"
+ " the entry for manifest %s is missing the field
`added-rows-count`",
addedRowsCount += manifest.addedRowsCount();
}
}
writer.add(manifest);
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
