Re: [PR] PyArrow: Avoid buffer-overflow by avoid doing a sort [iceberg-python]

Fokko merged PR #1555: URL: https://github.com/apache/iceberg-python/pull/1555 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [PR] PyArrow: Avoid buffer-overflow by avoid doing a sort [iceberg-python]

Fokko commented on code in PR #1555: URL: https://github.com/apache/iceberg-python/pull/1555#discussion_r1926067731 ## pyiceberg/partitioning.py: ## @@ -413,8 +413,10 @@ def partition_record_value(partition_field: PartitionField, value: Any, schema: the final partition rec

Re: [PR] PyArrow: Avoid buffer-overflow by avoid doing a sort [iceberg-python]

kevinjqliu commented on code in PR #1555: URL: https://github.com/apache/iceberg-python/pull/1555#discussion_r1925983653 ## pyiceberg/partitioning.py: ## @@ -413,8 +413,10 @@ def partition_record_value(partition_field: PartitionField, value: Any, schema: the final partitio

Re: [PR] PyArrow: Avoid buffer-overflow by avoid doing a sort [iceberg-python]

kevinjqliu commented on code in PR #1555: URL: https://github.com/apache/iceberg-python/pull/1555#discussion_r1925983134 ## pyiceberg/partitioning.py: ## @@ -413,8 +413,10 @@ def partition_record_value(partition_field: PartitionField, value: Any, schema: the final partitio

Re: [PR] PyArrow: Avoid buffer-overflow by avoid doing a sort [iceberg-python]

Fokko commented on code in PR #1555: URL: https://github.com/apache/iceberg-python/pull/1555#discussion_r1925902812 ## pyiceberg/partitioning.py: ## @@ -413,8 +413,10 @@ def partition_record_value(partition_field: PartitionField, value: Any, schema: the final partition rec

Re: [PR] PyArrow: Avoid buffer-overflow by avoid doing a sort [iceberg-python]

Fokko commented on code in PR #1555: URL: https://github.com/apache/iceberg-python/pull/1555#discussion_r1925454227 ## pyiceberg/partitioning.py: ## @@ -413,8 +413,10 @@ def partition_record_value(partition_field: PartitionField, value: Any, schema: the final partition rec

Re: [PR] PyArrow: Avoid buffer-overflow by avoid doing a sort [iceberg-python]

kevinjqliu commented on PR #1555: URL: https://github.com/apache/iceberg-python/pull/1555#issuecomment-2605648387 2^32 (4_294_967_296) is around 4GB, we just need to test a scenario greater than that -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] PyArrow: Avoid buffer-overflow by avoid doing a sort [iceberg-python]

Fokko commented on PR #1555: URL: https://github.com/apache/iceberg-python/pull/1555#issuecomment-2605603737 @kevinjqliu I think the test is a bit too much, according to your comment here https://github.com/apache/iceberg-python/pull/1539#discussion_r1922705843 the test allocates almost 5gb

Re: [PR] PyArrow: Avoid buffer-overflow by avoid doing a sort [iceberg-python]

Fokko commented on code in PR #1555: URL: https://github.com/apache/iceberg-python/pull/1555#discussion_r1924223584 ## tests/integration/test_writes/test_partitioned_writes.py: ## @@ -1126,3 +1127,25 @@ def test_append_multiple_partitions( """ ) assert files_d

Re: [PR] PyArrow: Avoid buffer-overflow by avoid doing a sort [iceberg-python]

Fokko commented on code in PR #1555: URL: https://github.com/apache/iceberg-python/pull/1555#discussion_r1924222167 ## tests/integration/test_writes/test_partitioned_writes.py: ## @@ -1126,3 +1127,25 @@ def test_append_multiple_partitions( """ ) assert files_d

Re: [PR] PyArrow: Avoid buffer-overflow by avoid doing a sort [iceberg-python]

kevinjqliu commented on code in PR #1555: URL: https://github.com/apache/iceberg-python/pull/1555#discussion_r1924080871 ## tests/integration/test_writes/test_partitioned_writes.py: ## @@ -1126,3 +1127,25 @@ def test_append_multiple_partitions( """ ) assert fi

Re: [PR] PyArrow: Avoid buffer-overflow by avoid doing a sort [iceberg-python]

kevinjqliu commented on PR #1555: URL: https://github.com/apache/iceberg-python/pull/1555#issuecomment-2605266133 > make: *** [Makefile:55: test-integration] Aborted (core dumped) uh oh -- This is an automated message from the Apache Git Service. To respond to the message, please lo