Re: [PR] Partitioned Append on Identity Transform [iceberg-python]

2024-04-05 Thread via GitHub
Fokko merged PR #555: URL: https://github.com/apache/iceberg-python/pull/555 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] Partitioned Append on Identity Transform [iceberg-python]

2024-04-04 Thread via GitHub
Fokko commented on code in PR #555: URL: https://github.com/apache/iceberg-python/pull/555#discussion_r1552302617 ## pyiceberg/manifest.py: ## @@ -283,31 +277,12 @@ def __repr__(self) -> str: } -@singledispatch -def partition_field_to_data_file_partition_field(partition_fie

Re: [PR] Partitioned Append on Identity Transform [iceberg-python]

2024-04-03 Thread via GitHub
Fokko commented on code in PR #555: URL: https://github.com/apache/iceberg-python/pull/555#discussion_r1550332689 ## pyiceberg/typedef.py: ## @@ -199,3 +199,7 @@ def __repr__(self) -> str: def record_fields(self) -> List[str]: """Return values of all the fields of

Re: [PR] Partitioned Append on Identity Transform [iceberg-python]

2024-04-02 Thread via GitHub
jqin61 commented on code in PR #555: URL: https://github.com/apache/iceberg-python/pull/555#discussion_r1548140319 ## pyiceberg/table/__init__.py: ## @@ -2526,25 +2537,44 @@ def _dataframe_to_data_files( """ from pyiceberg.io.pyarrow import bin_pack_arrow_table, write_

Re: [PR] Partitioned Append on Identity Transform [iceberg-python]

2024-04-02 Thread via GitHub
jqin61 commented on code in PR #555: URL: https://github.com/apache/iceberg-python/pull/555#discussion_r1548140319 ## pyiceberg/table/__init__.py: ## @@ -2526,25 +2537,44 @@ def _dataframe_to_data_files( """ from pyiceberg.io.pyarrow import bin_pack_arrow_table, write_

Re: [PR] Partitioned Append on Identity Transform [iceberg-python]

2024-04-02 Thread via GitHub
jqin61 commented on code in PR #555: URL: https://github.com/apache/iceberg-python/pull/555#discussion_r1548140319 ## pyiceberg/table/__init__.py: ## @@ -2526,25 +2537,44 @@ def _dataframe_to_data_files( """ from pyiceberg.io.pyarrow import bin_pack_arrow_table, write_

Re: [PR] Partitioned Append on Identity Transform [iceberg-python]

2024-04-02 Thread via GitHub
jqin61 commented on code in PR #555: URL: https://github.com/apache/iceberg-python/pull/555#discussion_r1548068306 ## pyiceberg/typedef.py: ## @@ -199,3 +199,7 @@ def __repr__(self) -> str: def record_fields(self) -> List[str]: """Return values of all the fields of

Re: [PR] Partitioned Append on Identity Transform [iceberg-python]

2024-04-02 Thread via GitHub
Fokko commented on code in PR #555: URL: https://github.com/apache/iceberg-python/pull/555#discussion_r1547810906 ## pyiceberg/manifest.py: ## @@ -289,10 +286,7 @@ def partition_field_to_data_file_partition_field(partition_field_type: IcebergTy @partition_field_to_data_fil

Re: [PR] Partitioned Append on Identity Transform [iceberg-python]

2024-03-29 Thread via GitHub
jqin61 commented on code in PR #555: URL: https://github.com/apache/iceberg-python/pull/555#discussion_r1544785650 ## pyiceberg/table/__init__.py: ## @@ -3108,3 +3138,127 @@ def snapshots(self) -> "pa.Table": snapshots, schema=snapshots_schema,

Re: [PR] Partitioned Append on Identity Transform [iceberg-python]

2024-03-29 Thread via GitHub
jqin61 commented on code in PR #555: URL: https://github.com/apache/iceberg-python/pull/555#discussion_r1544785650 ## pyiceberg/table/__init__.py: ## @@ -3108,3 +3138,127 @@ def snapshots(self) -> "pa.Table": snapshots, schema=snapshots_schema,

Re: [PR] Partitioned Append on Identity Transform [iceberg-python]

2024-03-29 Thread via GitHub
jqin61 commented on code in PR #555: URL: https://github.com/apache/iceberg-python/pull/555#discussion_r1544783412 ## pyiceberg/table/__init__.py: ## @@ -3108,3 +3138,127 @@ def snapshots(self) -> "pa.Table": snapshots, schema=snapshots_schema,

Re: [PR] Partitioned Append on Identity Transform [iceberg-python]

2024-03-29 Thread via GitHub
jqin61 commented on code in PR #555: URL: https://github.com/apache/iceberg-python/pull/555#discussion_r1544770723 ## pyiceberg/table/__init__.py: ## @@ -3108,3 +3138,127 @@ def snapshots(self) -> "pa.Table": snapshots, schema=snapshots_schema,

Re: [PR] Partitioned Append on Identity Transform [iceberg-python]

2024-03-29 Thread via GitHub
jqin61 commented on code in PR #555: URL: https://github.com/apache/iceberg-python/pull/555#discussion_r1544770143 ## pyiceberg/table/__init__.py: ## @@ -3108,3 +3138,127 @@ def snapshots(self) -> "pa.Table": snapshots, schema=snapshots_schema,

Re: [PR] Partitioned Append on Identity Transform [iceberg-python]

2024-03-29 Thread via GitHub
jqin61 commented on code in PR #555: URL: https://github.com/apache/iceberg-python/pull/555#discussion_r1544756546 ## pyiceberg/table/__init__.py: ## @@ -3108,3 +3138,127 @@ def snapshots(self) -> "pa.Table": snapshots, schema=snapshots_schema,

Re: [PR] Partitioned Append on Identity Transform [iceberg-python]

2024-03-29 Thread via GitHub
jqin61 commented on code in PR #555: URL: https://github.com/apache/iceberg-python/pull/555#discussion_r1544754857 ## pyiceberg/table/__init__.py: ## @@ -2526,25 +2537,44 @@ def _dataframe_to_data_files( """ from pyiceberg.io.pyarrow import bin_pack_arrow_table, write_

Re: [PR] Partitioned Append on Identity Transform [iceberg-python]

2024-03-29 Thread via GitHub
jqin61 commented on code in PR #555: URL: https://github.com/apache/iceberg-python/pull/555#discussion_r1544754492 ## pyiceberg/table/__init__.py: ## @@ -2526,25 +2537,44 @@ def _dataframe_to_data_files( """ from pyiceberg.io.pyarrow import bin_pack_arrow_table, write_

Re: [PR] Partitioned Append on Identity Transform [iceberg-python]

2024-03-29 Thread via GitHub
jqin61 commented on code in PR #555: URL: https://github.com/apache/iceberg-python/pull/555#discussion_r1544752616 ## pyiceberg/table/__init__.py: ## @@ -2489,16 +2488,28 @@ def _add_and_move_fields( class WriteTask: write_uuid: uuid.UUID task_id: int +schema: Sch

Re: [PR] Partitioned Append on Identity Transform [iceberg-python]

2024-03-28 Thread via GitHub
syun64 commented on code in PR #555: URL: https://github.com/apache/iceberg-python/pull/555#discussion_r1544039629 ## pyiceberg/table/__init__.py: ## @@ -3108,3 +3138,127 @@ def snapshots(self) -> "pa.Table": snapshots, schema=snapshots_schema,

Re: [PR] Partitioned Append on Identity Transform [iceberg-python]

2024-03-28 Thread via GitHub
syun64 commented on code in PR #555: URL: https://github.com/apache/iceberg-python/pull/555#discussion_r1544015221 ## pyiceberg/table/__init__.py: ## @@ -2526,25 +2537,44 @@ def _dataframe_to_data_files( """ from pyiceberg.io.pyarrow import bin_pack_arrow_table, write_

Re: [PR] Partitioned Append on Identity Transform [iceberg-python]

2024-03-28 Thread via GitHub
jqin61 commented on code in PR #555: URL: https://github.com/apache/iceberg-python/pull/555#discussion_r1543796831 ## tests/integration/test_partitioned_write.py: ## @@ -0,0 +1,533 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license ag

Re: [PR] Partitioned Append on Identity Transform [iceberg-python]

2024-03-28 Thread via GitHub
jqin61 commented on code in PR #555: URL: https://github.com/apache/iceberg-python/pull/555#discussion_r1543797231 ## tests/integration/test_partitioned_write.py: ## @@ -0,0 +1,533 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license ag

Re: [PR] Partitioned Append on Identity Transform [iceberg-python]

2024-03-28 Thread via GitHub
jqin61 commented on code in PR #555: URL: https://github.com/apache/iceberg-python/pull/555#discussion_r1543796831 ## tests/integration/test_partitioned_write.py: ## @@ -0,0 +1,533 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license ag

Re: [PR] Partitioned Append on Identity Transform [iceberg-python]

2024-03-28 Thread via GitHub
jqin61 commented on code in PR #555: URL: https://github.com/apache/iceberg-python/pull/555#discussion_r1543796652 ## tests/integration/test_partitioned_write.py: ## @@ -0,0 +1,533 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license ag

Re: [PR] Partitioned Append on Identity Transform [iceberg-python]

2024-03-28 Thread via GitHub
jqin61 commented on code in PR #555: URL: https://github.com/apache/iceberg-python/pull/555#discussion_r1543793736 ## pyiceberg/table/__init__.py: ## @@ -2526,25 +2537,44 @@ def _dataframe_to_data_files( """ from pyiceberg.io.pyarrow import bin_pack_arrow_table, write_