Re: [I] Error reading table after appending pyarrow table [iceberg-python]

[via GitHub]

kevinjqliu commented on issue #1798:
URL: 
https://github.com/apache/iceberg-python/issues/1798#issuecomment-2764856799

   alternatively, this can be resolved by setting the table's name-mapping. 
This is interesting because `name-mapping` is not set by default.
   
   But [Spark ensure that it is 
set](https://github.com/apache/iceberg/blob/main/spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/BaseTableCreationSparkAction.java#L168-L174)
 
   
   ```
   from pyiceberg.catalog import load_catalog
   
   catalog = load_catalog(**dict(type="in-memory"))
   
   from pyiceberg.schema import Schema
   from pyiceberg.types import NestedField, StringType, ListType
   from pyiceberg.table import TableProperties
   
   schema = Schema(
       NestedField(field_id=1, name="name", field_type=StringType(), 
required=False),
       NestedField(
           field_id=3,
           name="my_list",
           field_type=ListType(
               element_id=45, element=StringType(), element_required=False
           ),
           required=False,
       ),
   )
   catalog.create_namespace_if_not_exists("test")
   tbl = catalog.create_table_if_not_exists("test.table", schema)
   print(f"name-mapping: {tbl.metadata.name_mapping()}")
   
   if tbl.metadata.name_mapping() is None:
       with tbl.transaction() as txn:
           txn.set_properties(
               **{TableProperties.DEFAULT_NAME_MAPPING: 
tbl.metadata.schema().name_mapping.model_dump_json()}
           )
   
   print(f"name-mapping: {tbl.metadata.name_mapping()}")
   
   import pyarrow as pa
   df_1 = pa.Table.from_pylist([
       {"name": "one", "my_list": ["test"]},
       {"name": "another", "my_list": ["test"]},
   ])
   catalog.load_table("test.table").append(df_1)
   catalog.load_table("test.table").scan().to_arrow()
   
   import pyarrow as pa
   df_2 = pa.Table.from_pylist([
       {"name": "one"},
       {"name": "another"},
   ])
   catalog.load_table("test.table").append(df_2)
   catalog.load_table("test.table").scan().to_arrow()
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org