Re: [PR] Add support for orc format [iceberg-python]

2024-06-17 Thread via GitHub
MehulBatra commented on code in PR #790: URL: https://github.com/apache/iceberg-python/pull/790#discussion_r1642734352 ## pyiceberg/io/pyarrow.py: ## @@ -912,6 +916,9 @@ def primitive(self, primitive: pa.DataType) -> PrimitiveType: return TimestamptzType()

Re: [PR] Add support for orc format [iceberg-python]

2024-06-17 Thread via GitHub
MehulBatra commented on PR #790: URL: https://github.com/apache/iceberg-python/pull/790#issuecomment-2172597173 > I've added some comments for the read side. > > We may try to merge the read support first and make write support a separate PR. WDYT? @HonahX Works for me and I be

Re: [PR] Add support for orc format [iceberg-python]

2024-06-17 Thread via GitHub
MehulBatra commented on code in PR #790: URL: https://github.com/apache/iceberg-python/pull/790#discussion_r1642376476 ## pyiceberg/io/pyarrow.py: ## @@ -799,11 +802,12 @@ def primitive(self, primitive: pa.DataType) -> T: def _get_field_id(field: pa.Field) -> Optional[int]:

Re: [PR] Add support for orc format [iceberg-python]

2024-06-11 Thread via GitHub
HonahX commented on code in PR #790: URL: https://github.com/apache/iceberg-python/pull/790#discussion_r1632797941 ## pyiceberg/io/pyarrow.py: ## @@ -912,6 +916,9 @@ def primitive(self, primitive: pa.DataType) -> PrimitiveType: return TimestamptzType()

Re: [PR] Add support for orc format [iceberg-python]

2024-06-11 Thread via GitHub
HonahX commented on code in PR #790: URL: https://github.com/apache/iceberg-python/pull/790#discussion_r1632769062 ## pyiceberg/io/pyarrow.py: ## @@ -799,11 +802,12 @@ def primitive(self, primitive: pa.DataType) -> T: def _get_field_id(field: pa.Field) -> Optional[int]: -

Re: [PR] Add support for orc format [iceberg-python]

2024-06-10 Thread via GitHub
MehulBatra commented on PR #790: URL: https://github.com/apache/iceberg-python/pull/790#issuecomment-2159094452 Thanks, @HonahX for the feedback, I will consider all this while moving forward! -- This is an automated message from the Apache Git Service. To respond to the message, please l

Re: [PR] Add support for orc format [iceberg-python]

2024-06-10 Thread via GitHub
HonahX commented on PR #790: URL: https://github.com/apache/iceberg-python/pull/790#issuecomment-2157590281 Hi @MehulBatra. Thanks for taking this! It looks like a great start. > I believe we need to make a change in this write_file method to support ORC writes, as the link goes

Re: [PR] Add support for orc format [iceberg-python]

2024-06-05 Thread via GitHub
MehulBatra commented on PR #790: URL: https://github.com/apache/iceberg-python/pull/790#issuecomment-2150324091 I believe we need to make a change here to support ORC writes, please correct me if I am pointing towards the wrong direction https://github.com/apache/iceberg-python/blob/a11

Re: [PR] Add support for orc format [iceberg-python]

2024-06-05 Thread via GitHub
MehulBatra commented on PR #790: URL: https://github.com/apache/iceberg-python/pull/790#issuecomment-2149033357 Hi @Fokko and @HonahX ✅ I have modified the read logic to read the orc file-based iceberg table and wrote an integration test too it is working great. Would love Some g