chenjunjiedada commented on PR #8164: URL: https://github.com/apache/iceberg/pull/8164#issuecomment-1656511292
@aokolnychyi The table doesn't have a sort order. I agree that without repartition/rebalance is not a good idea. It just does not hurt much if the data contains few partitions. >1. Leave as is where no local sort is triggered if fanout writers are enabled and the table is unsorted. >2. Never request a local sort if fanout writers are enabled (even when the table is sorted). >3. Add a SQL property like `spark.sql.iceberg.use-table-ordering-with-fanout-writers` to control this behavior. I prefer option 1, and it would be better if we can combine 2 and 3 so that use an option to control local sorting by table ordering. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
