amogh-jahagirdar commented on code in PR #12736:
URL: https://github.com/apache/iceberg/pull/12736#discussion_r2060881490
##########
spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/SparkCopyOnWriteOperation.java:
##########
@@ -95,13 +98,18 @@ public WriteBuilder newWriteBuilder(LogicalWriteInfo info) {
@Override
public NamedReference[] requiredMetadataAttributes() {
- NamedReference file = Expressions.column(MetadataColumns.FILE_PATH.name());
- NamedReference pos =
Expressions.column(MetadataColumns.ROW_POSITION.name());
+ List<NamedReference> metadataAttributes = Lists.newArrayList();
+
metadataAttributes.add(Expressions.column(MetadataColumns.FILE_PATH.name()));
- if (command == DELETE || command == UPDATE) {
- return new NamedReference[] {file, pos};
- } else {
- return new NamedReference[] {file};
+ if (TableUtil.supportsRowLineage(table)) {
+
metadataAttributes.add(Expressions.column(MetadataColumns.ROW_POSITION.name()));
+
metadataAttributes.add(Expressions.column(MetadataColumns.ROW_ID.name()));
+ metadataAttributes.add(
+
Expressions.column(MetadataColumns.LAST_UPDATED_SEQUENCE_NUMBER.name()));
+ } else if (command == DELETE || command == UPDATE) {
+
metadataAttributes.add(Expressions.column(MetadataColumns.ROW_POSITION.name()));
}
+
+ return metadataAttributes.toArray(NamedReference[]::new);
Review Comment:
If it's a delete or update add the row position fist. Then if it's row
lineage supported table add those columns. That's easier to parse I think
compared to the else if structure now
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
