rahulsmahadev opened a new pull request, #4358:
URL: https://github.com/apache/iceberg-python/pull/4358

   # Rationale for this change
   
   The file, entry and partition metadata tables raise on any table with a 
`uuid` column:
   
   ```python
   schema = Schema(NestedField(1, "id", LongType()), NestedField(2, "u", 
UUIDType()))
   tbl = catalog.create_table("default.t", schema=schema)
   tbl.append(pa.table({"id": pa.array([1], pa.int64()), "u": 
pa.array([uuid.UUID(int=1).bytes], pa.uuid())}))
   tbl.inspect.files()
   # pyarrow.lib.ArrowNotImplementedError: extension
   ```
   
   `readable_metrics` holds the uuid bounds as `arrow.uuid` values inside a 
struct, and pyarrow cannot build an extension type nested in a struct from 
Python values, so `pa.Table.from_pylist` raises. This affects `entries()`, 
`files()`, `data_files()`, `delete_files()`, `all_files()`, `all_data_files()`, 
`all_delete_files()`, and `partitions()` for a table partitioned by identity on 
a uuid column, even when the bounds are null.
   
   This change builds these tables with the storage types 
(`fixed_size_binary(16)` for a uuid) and casts the result to the unchanged 
schema. UUID partition values are passed as their 16 bytes, which pyarrow 18 
(the minimum supported version) needs. UUID row filters (#2372) are a separate 
issue and are not touched.
   
   ## Are these changes tested?
   
   Yes. Three new tests in `tests/table/test_inspect.py` fail with 
`ArrowNotImplementedError: extension` without the fix and pass with it, on 
pyarrow 25 and 18:
   
   - `test_inspect_files_and_entries_render_uuid_bounds`: the bounds and the 
`arrow.uuid` output type for the file and entry tables.
   - `test_inspect_files_of_empty_table_with_uuid_column`: `files()` and 
`all_files()` on a table without snapshots.
   - `test_inspect_partitions_and_files_render_uuid_partition`: `partitions()`, 
`files()` and `entries()` for a table partitioned by identity on a uuid column.
   
   ## Are there any user-facing changes?
   
   Bug fix only: these metadata tables no longer raise on tables with a uuid 
column. The output schema is unchanged.
   
   This pull request and its description were written by Isaac.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to