rahulsmahadev opened a new pull request, #4390:
URL: https://github.com/apache/iceberg-python/pull/4390

   # Rationale for this change
   
   Two defaults give ascending sort fields the nulls-last order, which 
disagrees with Java and with `SortField`'s own default for ascending fields. 
The same ascending field can therefore be recorded with different null orders 
depending on how it is built.
   
   `Table.update_sort_order().asc(name, transform)` with no `null_order` 
records nulls-last:
   
   ```python
   tbl.update_sort_order().asc("user_id", IdentityTransform()).commit()
   tbl.sort_order().fields[0].null_order  # NullOrder.NULLS_LAST, expected 
NULLS_FIRST
   ```
   
   `SortField` built with a str direction also gets nulls-last, while the enum 
member gets nulls-first:
   
   ```python
   SortField(source_id=1, transform=IdentityTransform(), 
direction="asc").null_order  # NULLS_LAST
   SortField(source_id=1, transform=IdentityTransform(), 
direction=SortDirection.ASC).null_order  # NULLS_FIRST
   ```
   
   Java's `SortOrderBuilder.asc(String)` uses `NullOrder.NULLS_FIRST` (its 
Javadoc says "ascending with nulls first"), and `desc(String)` uses 
`NULLS_LAST`. `SortField` picks nulls-first for `SortDirection.ASC` and 
nulls-last otherwise. So `desc()` agrees in all three places, and only the 
ascending default in `UpdateSortOrder.asc()` differs. The spec requires 
`null-order` in every sort field and does not define a default, so Java and 
`SortField` are the reference.
   
   In `SortField`, the `set_null_order` validator chooses the default by 
comparing the raw `direction` with `SortDirection.ASC`. `SortDirection` is a 
plain `Enum`, so the str `"asc"` never matches and gets the descending default. 
Pydantic converts the value to the enum afterwards, so the field ends up 
ascending with nulls-last.
   
   This change:
   
   - `UpdateSortOrder.asc()` defaults to `NullOrder.NULLS_FIRST`. `desc()` 
keeps `NULLS_LAST`.
   - `set_null_order` converts the direction to `SortDirection` before it 
chooses the default.
   
   An explicit null order is used as given. Existing sort orders are not 
changed, because their null order is already stored. The `asc()` default came 
in with `UpdateSortOrder` in #2552. Calls that omit `null_order` now request 
nulls-first, also on existing tables, where a matching existing sort order may 
be reused. Pass `NullOrder.NULLS_LAST` explicitly to keep the previous 
behaviour.
   
   ## Are these changes tested?
   
   Yes. `test_update_sort_order_default_null_order` in 
`tests/catalog/test_catalog_behaviors.py` runs through the `catalog` fixture 
(memory, SQL, and SQL without rowcount catalogs). It commits `asc()` and 
`desc()` with no null order and checks the stored sort order, then commits 
explicit null orders and checks that they are kept. 
`test_deserialize_sort_field_str_direction` in `tests/table/test_sorting.py` 
checks that `"asc"` and `"desc"` validate to the same `SortField` as the enum 
members, with the matching default null order. Both fail on main with 
`NullOrder.NULLS_LAST` where `NullOrder.NULLS_FIRST` is expected, and both pass 
with this change. The integration tests in 
`tests/integration/test_sort_order_update.py` pass explicit null orders, so 
they do not depend on the default.
   
   ## Are there any user-facing changes?
   
   Yes. `update_sort_order().asc()` without a null order now records 
nulls-first (it recorded nulls-last), and `SortField` with a str direction now 
gets the same default as the enum member. Please add the `changelog` label.
   
   This pull request and its description were written by Isaac.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to