manuzhang opened a new issue, #18952:
URL: https://github.com/apache/iceberg/issues/18952

   ### Feature Request / Improvement
   
   Add an Iceberg-backed regression for 
[SPARK-60044](https://github.com/apache/spark/pull/59306), which freezes the 
ordering used by a planned `GroupPartitionsExec` k-way merge.
   
   #18294 explicitly leaves the exact upstream scenario untested. Its 
computed-key ORDER BY cases exercise SPARK-59905; they do not establish that a 
planned sorted merge keeps its ordering after a later SQL configuration change.
   
   Acceptance criteria:
   
   - [ ] Use an Iceberg table with mixed identity and transformed partition 
keys, multiple splits per grouping key, and a window such as `ROW_NUMBER() OVER 
(PARTITION BY id ORDER BY name)`.
   - [ ] Enable the subset-key and preserve-ordering-on-coalesce paths, and 
plan a sorted `GroupPartitionsExec` merge.
   - [ ] Change `spark.sql.sources.v2.bucketing.partitionKeyOrdering.enabled` 
after planning and before execution.
   - [ ] Assert exact row numbers, retained merge ordering, and a plan that 
actually uses the sorted merge without an extra sort masking the regression.
   - [ ] Cover AQE on/off and applicable Iceberg planning modes, restoring 
configuration after each case.
   
   Start from the [upstream regression and its Spark 4.2 
backport](https://github.com/apache/spark/pull/59509), adapting it to real 
Iceberg scans in `spark/v4.2`. Current main already pins Spark 4.2.1. Related: 
#17959 and the [existing SPJ 
suite](https://github.com/apache/iceberg/blob/234d55774d3c7355f92c8e93f8e117b70726e556/spark/v4.2/spark/src/test/java/org/apache/iceberg/spark/sql/TestStoragePartitionedJoins.java).
   
   ### Query engine
   
   Spark
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to