chenjunjiedada commented on PR #8164:
URL: https://github.com/apache/iceberg/pull/8164#issuecomment-1656994697
I found #7637 already contains the option 1 logic in Spark 3.4, the unit
test `testRangeCopyOnWriteMergePartitionedUnsortedTableFanout` also verifies
that. The issue mentioned is in our Spark 3.3 production env, so backporting
#7637 should work.
```
// a local ordering within a task is beneficial in two cases:
// - there is a defined table sort order, so it is clear how the data
should be ordered
// - the table is partitioned and fanout writers are disabled,
// so records for one partition must be co-located within a task
private static SortOrder[] writeOrdering(Table table, boolean
fanoutEnabled) {
if (fanoutEnabled && table.sortOrder().isUnsorted()) {
return EMPTY_ORDERING;
} else {
return ordering(table);
}
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
