Re: [PR] Arrow: Set field-id with prefix [iceberg-python]

2024-01-12 Thread via GitHub
HonahX merged PR #227: URL: https://github.com/apache/iceberg-python/pull/227 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [PR] Arrow: Set field-id with prefix [iceberg-python]

2023-12-21 Thread via GitHub
Fokko commented on code in PR #227: URL: https://github.com/apache/iceberg-python/pull/227#discussion_r1433978923 ## tests/io/test_pyarrow_visitor.py: ## @@ -209,9 +209,9 @@ def test_pyarrow_variable_binary_to_iceberg() -> None: def test_pyarrow_struct_to_iceberg() -> None:

Re: [PR] Arrow: Set field-id with prefix [iceberg-python]

2023-12-20 Thread via GitHub
HonahX commented on code in PR #227: URL: https://github.com/apache/iceberg-python/pull/227#discussion_r1433377086 ## tests/io/test_pyarrow_visitor.py: ## @@ -209,9 +209,9 @@ def test_pyarrow_variable_binary_to_iceberg() -> None: def test_pyarrow_struct_to_iceberg() -> None:

Re: [PR] Arrow: Set field-id with prefix [iceberg-python]

2023-12-20 Thread via GitHub
Fokko commented on code in PR #227: URL: https://github.com/apache/iceberg-python/pull/227#discussion_r1433111604 ## pyiceberg/io/pyarrow.py: ## @@ -720,25 +719,17 @@ def primitive(self, primitive: pa.DataType) -> Optional[T]: def _get_field_id(field: pa.Field) -> Optional

Re: [PR] Arrow: Set field-id with prefix [iceberg-python]

2023-12-20 Thread via GitHub
Fokko commented on code in PR #227: URL: https://github.com/apache/iceberg-python/pull/227#discussion_r1433110576 ## pyiceberg/io/pyarrow.py: ## @@ -720,25 +719,17 @@ def primitive(self, primitive: pa.DataType) -> Optional[T]: def _get_field_id(field: pa.Field) -> Optional

Re: [PR] Arrow: Set field-id with prefix [iceberg-python]

2023-12-20 Thread via GitHub
Fokko commented on code in PR #227: URL: https://github.com/apache/iceberg-python/pull/227#discussion_r1433110149 ## tests/io/test_pyarrow_visitor.py: ## @@ -209,9 +209,9 @@ def test_pyarrow_variable_binary_to_iceberg() -> None: def test_pyarrow_struct_to_iceberg() -> None:

Re: [PR] Arrow: Set field-id with prefix [iceberg-python]

2023-12-20 Thread via GitHub
Fokko commented on code in PR #227: URL: https://github.com/apache/iceberg-python/pull/227#discussion_r1433109602 ## pyiceberg/io/pyarrow.py: ## @@ -154,10 +154,9 @@ ONE_MEGABYTE = 1024 * 1024 BUFFER_SIZE = "buffer-size" ICEBERG_SCHEMA = b"iceberg.schema" -FIELD_ID = "field_i

Re: [PR] Arrow: Set field-id with prefix [iceberg-python]

2023-12-20 Thread via GitHub
HonahX commented on code in PR #227: URL: https://github.com/apache/iceberg-python/pull/227#discussion_r1432482150 ## pyiceberg/io/pyarrow.py: ## @@ -720,25 +719,17 @@ def primitive(self, primitive: pa.DataType) -> Optional[T]: def _get_field_id(field: pa.Field) -> Optiona

Re: [PR] Arrow: Set field-id with prefix [iceberg-python]

2023-12-20 Thread via GitHub
HonahX commented on code in PR #227: URL: https://github.com/apache/iceberg-python/pull/227#discussion_r1432471641 ## pyiceberg/io/pyarrow.py: ## @@ -154,10 +154,9 @@ ONE_MEGABYTE = 1024 * 1024 BUFFER_SIZE = "buffer-size" ICEBERG_SCHEMA = b"iceberg.schema" -FIELD_ID = "field_

Re: [PR] Arrow: Set field-id with prefix [iceberg-python]

2023-12-20 Thread via GitHub
HonahX commented on code in PR #227: URL: https://github.com/apache/iceberg-python/pull/227#discussion_r1432471641 ## pyiceberg/io/pyarrow.py: ## @@ -154,10 +154,9 @@ ONE_MEGABYTE = 1024 * 1024 BUFFER_SIZE = "buffer-size" ICEBERG_SCHEMA = b"iceberg.schema" -FIELD_ID = "field_

[PR] Arrow: Set field-id with prefix [iceberg-python]

2023-12-19 Thread via GitHub
Fokko opened a new pull request, #227: URL: https://github.com/apache/iceberg-python/pull/227 While working on the write support, I started to understand the meaning of the Arrow metadata. The `PARQUET:` prefix means that it is specific to Parquet, and setting `PARQUET:field_id` will actual