Re: [PR] Add name-mapping [iceberg-python]

HonahX commented on code in PR #212: URL: https://github.com/apache/iceberg-python/pull/212#discussion_r1427656072 ## pyiceberg/table/name_mapping.py: ## @@ -0,0 +1,203 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. S

Re: [PR] Add name-mapping [iceberg-python]

HonahX commented on code in PR #212: URL: https://github.com/apache/iceberg-python/pull/212#discussion_r1427656072 ## pyiceberg/table/name_mapping.py: ## @@ -0,0 +1,203 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. S

Re: [I] Documentation [iceberg-rust]

Xuanwo commented on issue #114: URL: https://github.com/apache/iceberg-rust/issues/114#issuecomment-1857400642 > cc @Xuanwo Are you working on this? I think we are ok to ship the first release now, all blockers has been resolved. I expect to finish it in next week. -- This is an au

Re: [I] Documentation [iceberg-rust]

liurenjie1024 commented on issue #114: URL: https://github.com/apache/iceberg-rust/issues/114#issuecomment-1857396621 cc @Xuanwo Are you working on this? I think we are ok to ship the first release now, all blockers has been resolved. -- This is an automated message from the Apache Git Se

Re: [I] website: Setup rust.iceberg.apache.org [iceberg-rust]

liurenjie1024 commented on issue #8: URL: https://github.com/apache/iceberg-rust/issues/8#issuecomment-1857387581 Duplicated with #114 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

Re: [I] website: Setup rust.iceberg.apache.org [iceberg-rust]

liurenjie1024 closed issue #8: website: Setup rust.iceberg.apache.org URL: https://github.com/apache/iceberg-rust/issues/8 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

Re: [PR] Core: Look up targeted position deletes by path [iceberg]

aokolnychyi commented on code in PR #9251: URL: https://github.com/apache/iceberg/pull/9251#discussion_r1427628834 ## core/src/main/java/org/apache/iceberg/DeleteFileIndex.java: ## @@ -582,93 +513,187 @@ private Iterable>> deleteManifestRea } } - // a group of indexe

Re: [PR] Core: Look up targeted position deletes by path [iceberg]

aokolnychyi commented on code in PR #9251: URL: https://github.com/apache/iceberg/pull/9251#discussion_r1427628834 ## core/src/main/java/org/apache/iceberg/DeleteFileIndex.java: ## @@ -582,93 +513,187 @@ private Iterable>> deleteManifestRea } } - // a group of indexe

Re: [PR] Support parameterized tests at class-level with JUnit5 [iceberg]

nastra commented on PR #9161: URL: https://github.com/apache/iceberg/pull/9161#issuecomment-1857383702 @Fokko @pvary could you review this one please? Once this is merged we can use it for https://github.com/apache/iceberg/pull/9185 -- This is an automated message from the Apache Git Serv

Re: [I] An exception occurred while writing iceberg data through Spark: org. apache. iceberg. exceptions. CommitFailedException: metadata location has changed [iceberg]

Zhangg7723 commented on issue #9178: URL: https://github.com/apache/iceberg/issues/9178#issuecomment-1857382064 From https://iceberg.apache.org/docs/1.2.1/configuration/#write-properties write.update.isolation-level, write.merge.isolation-level -- This is an automated message from the A

Re: [PR] Core: Look up targeted position deletes by path [iceberg]

aokolnychyi commented on code in PR #9251: URL: https://github.com/apache/iceberg/pull/9251#discussion_r1427621708 ## core/src/main/java/org/apache/iceberg/DeleteFileIndex.java: ## @@ -582,93 +513,187 @@ private Iterable>> deleteManifestRea } } - // a group of indexe

Re: [PR] Core: Look up targeted position deletes by path [iceberg]

aokolnychyi commented on code in PR #9251: URL: https://github.com/apache/iceberg/pull/9251#discussion_r1427619933 ## core/src/main/java/org/apache/iceberg/DeleteFileIndex.java: ## @@ -582,93 +513,187 @@ private Iterable>> deleteManifestRea } } - // a group of indexe

Re: [PR] Core: Look up targeted position deletes by path [iceberg]

aokolnychyi commented on code in PR #9251: URL: https://github.com/apache/iceberg/pull/9251#discussion_r1427617315 ## core/src/main/java/org/apache/iceberg/DeleteFileIndex.java: ## @@ -474,68 +420,64 @@ private Collection loadDeleteFiles() { DeleteFileIndex build() {

Re: [I] position delete in BaseEqualityDeltaWriter write function will lead to unstable result when equalityFieldColumns is not null and upsert is false [iceberg]

pvary commented on issue #9299: URL: https://github.com/apache/iceberg/issues/9299#issuecomment-1857355625 Do I understand correctly, that you have 2 different jobs writing to the same table in update mode? I think this situation should be avoided. Flink update mode deletes the

Re: [PR] Add Hive integration tests [iceberg-python]

HonahX commented on code in PR #207: URL: https://github.com/apache/iceberg-python/pull/207#discussion_r1427569648 ## tests/integration/test_hive.py: ## @@ -0,0 +1,409 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. Se

Re: [PR] Add Hive integration tests [iceberg-python]

HonahX commented on code in PR #207: URL: https://github.com/apache/iceberg-python/pull/207#discussion_r1427569648 ## tests/integration/test_hive.py: ## @@ -0,0 +1,409 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. Se

Re: [PR] WIP: Glue catalog commit [iceberg-python]

HonahX commented on code in PR #140: URL: https://github.com/apache/iceberg-python/pull/140#discussion_r1427597141 ## pyiceberg/catalog/glue.py: ## @@ -59,21 +61,33 @@ from pyiceberg.partitioning import UNPARTITIONED_PARTITION_SPEC, PartitionSpec from pyiceberg.schema import S

Re: [PR] Add Hive integration tests [iceberg-python]

HonahX commented on code in PR #207: URL: https://github.com/apache/iceberg-python/pull/207#discussion_r1427591155 ## tests/integration/__init__.py: ## @@ -0,0 +1,16 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See

Re: [PR] Add Hive integration tests [iceberg-python]

HonahX commented on code in PR #207: URL: https://github.com/apache/iceberg-python/pull/207#discussion_r1427591155 ## tests/integration/__init__.py: ## @@ -0,0 +1,16 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See

Re: [PR] BugFix: ORC reader is not closed when SortedMerge iterator is used for positional deletes [iceberg]

pvary commented on code in PR #9301: URL: https://github.com/apache/iceberg/pull/9301#discussion_r1427582938 ## orc/src/main/java/org/apache/iceberg/orc/OrcIterable.java: ## @@ -134,7 +134,13 @@ private static VectorizedRowBatchIterator newOrcIterator( try { return

Re: [PR] BugFix: ORC reader is not closed when SortedMerge iterator is used for positional deletes [iceberg]

pvary commented on code in PR #9301: URL: https://github.com/apache/iceberg/pull/9301#discussion_r1427582845 ## orc/src/main/java/org/apache/iceberg/orc/OrcIterable.java: ## @@ -134,7 +134,13 @@ private static VectorizedRowBatchIterator newOrcIterator( try { return

Re: [PR] BugFix: ORC reader is not closed when SortedMerge iterator is used for positional deletes [iceberg]

pvary commented on code in PR #9301: URL: https://github.com/apache/iceberg/pull/9301#discussion_r1427582320 ## orc/src/main/java/org/apache/iceberg/orc/OrcIterable.java: ## @@ -134,7 +134,13 @@ private static VectorizedRowBatchIterator newOrcIterator( try { return

Re: [PR] BugFix: ORC reader is not closed when SortedMerge iterator is used for positional deletes [iceberg]

pvary commented on code in PR #9301: URL: https://github.com/apache/iceberg/pull/9301#discussion_r1427579685 ## orc/src/main/java/org/apache/iceberg/orc/OrcIterable.java: ## @@ -134,7 +134,13 @@ private static VectorizedRowBatchIterator newOrcIterator( try { return

Re: [PR] Add Hive integration tests [iceberg-python]

HonahX commented on code in PR #207: URL: https://github.com/apache/iceberg-python/pull/207#discussion_r1427569648 ## tests/integration/test_hive.py: ## @@ -0,0 +1,409 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. Se

Re: [PR] Build: Bump coverage from 7.3.2 to 7.3.3 [iceberg-python]

Fokko merged PR #214: URL: https://github.com/apache/iceberg-python/pull/214 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] Build: Bump actions/upload-artifact from 3 to 4 [iceberg-python]

Fokko merged PR #215: URL: https://github.com/apache/iceberg-python/pull/215 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] Core: Look up targeted position deletes by path [iceberg]

szehon-ho commented on code in PR #9251: URL: https://github.com/apache/iceberg/pull/9251#discussion_r1427495126 ## core/src/main/java/org/apache/iceberg/DeleteFileIndex.java: ## @@ -582,93 +513,187 @@ private Iterable>> deleteManifestRea } } - // a group of indexed

Re: [PR] fix when equalityFieldColumns is not null and upsert is false, position delete in write function will lead to unstable result if flink checkpoint interval is not same [iceberg]

sunnyzhuzhu commented on code in PR #9300: URL: https://github.com/apache/iceberg/pull/9300#discussion_r1427494930 ## core/src/main/java/org/apache/iceberg/io/BaseTaskWriter.java: ## @@ -137,11 +137,7 @@ public void write(T row) throws IOException { StructLike copiedKey =

Re: [PR] Core: Look up targeted position deletes by path [iceberg]

szehon-ho commented on code in PR #9251: URL: https://github.com/apache/iceberg/pull/9251#discussion_r1427495126 ## core/src/main/java/org/apache/iceberg/DeleteFileIndex.java: ## @@ -582,93 +513,187 @@ private Iterable>> deleteManifestRea } } - // a group of indexed

Re: [PR] fix when equalityFieldColumns is not null and upsert is false, position delete in write function will lead to unstable result if flink checkpoint interval is not same [iceberg]

sunnyzhuzhu commented on code in PR #9300: URL: https://github.com/apache/iceberg/pull/9300#discussion_r1427494930 ## core/src/main/java/org/apache/iceberg/io/BaseTaskWriter.java: ## @@ -137,11 +137,7 @@ public void write(T row) throws IOException { StructLike copiedKey =

Re: [PR] fix when equalityFieldColumns is not null and upsert is false, position delete in write function will lead to unstable result if flink checkpoint interval is not same [iceberg]

sunnyzhuzhu commented on code in PR #9300: URL: https://github.com/apache/iceberg/pull/9300#discussion_r1427490308 ## core/src/main/java/org/apache/iceberg/io/BaseTaskWriter.java: ## @@ -137,11 +137,7 @@ public void write(T row) throws IOException { StructLike copiedKey =

Re: [PR] Flink: fix flaky test that might fail due to classloader check [iceberg]

stevenzwu commented on PR #9216: URL: https://github.com/apache/iceberg/pull/9216#issuecomment-1857145512 @pvary can you follow up? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

Re: [PR] Write support [iceberg-python]

rdblue commented on code in PR #41: URL: https://github.com/apache/iceberg-python/pull/41#discussion_r1427433153 ## pyiceberg/io/pyarrow.py: ## @@ -1565,13 +1564,54 @@ def fill_parquet_file_metadata( del upper_bounds[field_id] del null_value_counts[field_id]

Re: [PR] Write support [iceberg-python]

rdblue commented on code in PR #41: URL: https://github.com/apache/iceberg-python/pull/41#discussion_r1427432722 ## pyiceberg/io/pyarrow.py: ## @@ -1565,13 +1564,54 @@ def fill_parquet_file_metadata( del upper_bounds[field_id] del null_value_counts[field_id]

Re: [PR] Write support [iceberg-python]

rdblue commented on code in PR #41: URL: https://github.com/apache/iceberg-python/pull/41#discussion_r1427432401 ## pyiceberg/io/pyarrow.py: ## @@ -1565,13 +1564,54 @@ def fill_parquet_file_metadata( del upper_bounds[field_id] del null_value_counts[field_id]

Re: [PR] Write support [iceberg-python]

rdblue commented on code in PR #41: URL: https://github.com/apache/iceberg-python/pull/41#discussion_r1427432264 ## pyiceberg/io/pyarrow.py: ## @@ -1565,13 +1564,54 @@ def fill_parquet_file_metadata( del upper_bounds[field_id] del null_value_counts[field_id]

Re: [PR] Write support [iceberg-python]

rdblue commented on code in PR #41: URL: https://github.com/apache/iceberg-python/pull/41#discussion_r1427431948 ## pyiceberg/io/pyarrow.py: ## @@ -1565,13 +1564,54 @@ def fill_parquet_file_metadata( del upper_bounds[field_id] del null_value_counts[field_id]

Re: [PR] Write support [iceberg-python]

rdblue commented on code in PR #41: URL: https://github.com/apache/iceberg-python/pull/41#discussion_r1427431827 ## pyiceberg/io/pyarrow.py: ## @@ -1565,13 +1564,54 @@ def fill_parquet_file_metadata( del upper_bounds[field_id] del null_value_counts[field_id]

Re: [PR] Write support [iceberg-python]

rdblue commented on code in PR #41: URL: https://github.com/apache/iceberg-python/pull/41#discussion_r1427431303 ## pyiceberg/io/pyarrow.py: ## @@ -1565,13 +1564,54 @@ def fill_parquet_file_metadata( del upper_bounds[field_id] del null_value_counts[field_id]

Re: [PR] Write support [iceberg-python]

rdblue commented on code in PR #41: URL: https://github.com/apache/iceberg-python/pull/41#discussion_r1427431067 ## pyiceberg/io/pyarrow.py: ## @@ -1565,13 +1564,54 @@ def fill_parquet_file_metadata( del upper_bounds[field_id] del null_value_counts[field_id]

Re: [PR] Write support [iceberg-python]

rdblue commented on code in PR #41: URL: https://github.com/apache/iceberg-python/pull/41#discussion_r1427430202 ## pyiceberg/io/pyarrow.py: ## @@ -1565,13 +1564,54 @@ def fill_parquet_file_metadata( del upper_bounds[field_id] del null_value_counts[field_id]

Re: [PR] Write support [iceberg-python]

rdblue commented on code in PR #41: URL: https://github.com/apache/iceberg-python/pull/41#discussion_r1427426778 ## pyiceberg/io/pyarrow.py: ## @@ -1565,13 +1564,54 @@ def fill_parquet_file_metadata( del upper_bounds[field_id] del null_value_counts[field_id]

Re: [I] TestMetricsRowGroupFilterTypes fails on certain machines, bisects to ORC lib upgrade [iceberg]

github-actions[bot] commented on issue #7303: URL: https://github.com/apache/iceberg/issues/7303#issuecomment-1857049772 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] TestMetricsRowGroupFilterTypes fails on certain machines, bisects to ORC lib upgrade [iceberg]

github-actions[bot] closed issue #7303: TestMetricsRowGroupFilterTypes fails on certain machines, bisects to ORC lib upgrade URL: https://github.com/apache/iceberg/issues/7303 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

Re: [I] When using flink dataframe api for upsert, the deduplication effect is not achieved [iceberg]

github-actions[bot] commented on issue #7639: URL: https://github.com/apache/iceberg/issues/7639#issuecomment-1857049525 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] When using flink dataframe api for upsert, the deduplication effect is not achieved [iceberg]

github-actions[bot] closed issue #7639: When using flink dataframe api for upsert, the deduplication effect is not achieved URL: https://github.com/apache/iceberg/issues/7639 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and us

Re: [PR] Adding documentation for flink iceberg connector for version 1.18 [iceberg]

stevenzwu merged PR #9304: URL: https://github.com/apache/iceberg/pull/9304 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.a

[PR] Build: Bump actions/upload-artifact from 3 to 4 [iceberg-python]

dependabot[bot] opened a new pull request, #215: URL: https://github.com/apache/iceberg-python/pull/215 Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 3 to 4. Release notes Sourced from https://github.com/actions/upload-artifact/releases";>actions/

Re: [PR] Core: Fixed certain operations failing to add new data files during retries [iceberg]

jasonf20 commented on code in PR #9230: URL: https://github.com/apache/iceberg/pull/9230#discussion_r1427386841 ## core/src/main/java/org/apache/iceberg/FastAppend.java: ## @@ -187,8 +187,7 @@ protected void cleanUncommitted(Set committed) { deleteFile(manifest.path(

Re: [PR] Core: Fixed certain operations failing to add new data files during retries [iceberg]

jasonf20 commented on code in PR #9230: URL: https://github.com/apache/iceberg/pull/9230#discussion_r1427375881 ## core/src/main/java/org/apache/iceberg/SnapshotProducer.java: ## @@ -487,7 +487,11 @@ protected void cleanAll() { } protected void deleteFile(String path) {

Re: [PR] Core: Fixed certain operations failing to add new data files during retries [iceberg]

jasonf20 commented on code in PR #9230: URL: https://github.com/apache/iceberg/pull/9230#discussion_r1427383977 ## core/src/main/java/org/apache/iceberg/SnapshotProducer.java: ## @@ -487,7 +487,11 @@ protected void cleanAll() { } protected void deleteFile(String path) {

Re: [PR] Core: Fixed certain operations failing to add new data files during retries [iceberg]

jasonf20 commented on code in PR #9230: URL: https://github.com/apache/iceberg/pull/9230#discussion_r1427375881 ## core/src/main/java/org/apache/iceberg/SnapshotProducer.java: ## @@ -487,7 +487,11 @@ protected void cleanAll() { } protected void deleteFile(String path) {

Re: [PR] Core: Fixed certain operations failing to add new data files during retries [iceberg]

jasonf20 commented on PR #9230: URL: https://github.com/apache/iceberg/pull/9230#issuecomment-1856811895 Based on further Slack discussions I re-added the FastApply fix. Couple of notes: * I think it’s better to clear the lists instead of removing one item at a time. If the list i

[PR] Build: Bump coverage from 7.3.2 to 7.3.3 [iceberg-python]

dependabot[bot] opened a new pull request, #214: URL: https://github.com/apache/iceberg-python/pull/214 Bumps [coverage](https://github.com/nedbat/coveragepy) from 7.3.2 to 7.3.3. Changelog Sourced from https://github.com/nedbat/coveragepy/blob/master/CHANGES.rst";>coverage's chang

Re: [PR] API, Core: Add sqlFor API to views to handle basic resolution of dialect [iceberg]

rdblue commented on code in PR #9247: URL: https://github.com/apache/iceberg/pull/9247#discussion_r1427362141 ## core/src/main/java/org/apache/iceberg/view/BaseView.java: ## @@ -103,4 +105,29 @@ public UpdateLocation updateLocation() { public UUID uuid() { return UUID.fr

Re: [PR] API, Core: Add sqlFor API to views to handle basic resolution of dialect [iceberg]

amogh-jahagirdar commented on code in PR #9247: URL: https://github.com/apache/iceberg/pull/9247#discussion_r1427181875 ## core/src/main/java/org/apache/iceberg/view/BaseView.java: ## @@ -103,4 +105,27 @@ public UpdateLocation updateLocation() { public UUID uuid() { retu

Re: [PR] API, Core: Add sqlFor API to views to handle basic resolution of dialect [iceberg]

amogh-jahagirdar commented on code in PR #9247: URL: https://github.com/apache/iceberg/pull/9247#discussion_r1427359104 ## core/src/test/java/org/apache/iceberg/view/ViewCatalogTests.java: ## @@ -1669,4 +1669,52 @@ public void concurrentReplaceViewVersion() {

Re: [PR] API, Core: Add sqlFor API to views to handle basic resolution of dialect [iceberg]

amogh-jahagirdar commented on code in PR #9247: URL: https://github.com/apache/iceberg/pull/9247#discussion_r1427359104 ## core/src/test/java/org/apache/iceberg/view/ViewCatalogTests.java: ## @@ -1669,4 +1669,52 @@ public void concurrentReplaceViewVersion() {

Re: [PR] API, Core: Add sqlFor API to views to handle basic resolution of dialect [iceberg]

amogh-jahagirdar commented on code in PR #9247: URL: https://github.com/apache/iceberg/pull/9247#discussion_r1427357997 ## core/src/test/java/org/apache/iceberg/view/ViewCatalogTests.java: ## @@ -200,6 +200,13 @@ public void completeCreateView() { .build

Re: [PR] Write support [iceberg-python]

jqin61 commented on code in PR #41: URL: https://github.com/apache/iceberg-python/pull/41#discussion_r1427302782 ## pyiceberg/io/pyarrow.py: ## @@ -1565,13 +1564,54 @@ def fill_parquet_file_metadata( del upper_bounds[field_id] del null_value_counts[field_id]

Re: [PR] Core: Fixed certain operations failing to add new data files during retries [iceberg]

jasonf20 commented on code in PR #9230: URL: https://github.com/apache/iceberg/pull/9230#discussion_r1427260537 ## core/src/main/java/org/apache/iceberg/MergingSnapshotProducer.java: ## @@ -895,7 +895,7 @@ private void cleanUncommittedAppends(Set committed) { }

Re: [PR] Core: Fixed certain operations failing to add new data files during retries [iceberg]

jasonf20 commented on code in PR #9230: URL: https://github.com/apache/iceberg/pull/9230#discussion_r1427260537 ## core/src/main/java/org/apache/iceberg/MergingSnapshotProducer.java: ## @@ -895,7 +895,7 @@ private void cleanUncommittedAppends(Set committed) { }

Re: [PR] Core: remove statistic files in CatalogUtil:dropTableData [iceberg]

dramaticlly commented on code in PR #9305: URL: https://github.com/apache/iceberg/pull/9305#discussion_r1427226753 ## core/src/main/java/org/apache/iceberg/CatalogUtil.java: ## @@ -100,7 +100,7 @@ public static void dropTableData(FileIO io, TableMetadata metadata) { LOG.

Re: [PR] Core: remove statistic files in CatalogUtil:dropTableData [iceberg]

RussellSpitzer commented on code in PR #9305: URL: https://github.com/apache/iceberg/pull/9305#discussion_r1427225369 ## core/src/main/java/org/apache/iceberg/CatalogUtil.java: ## @@ -100,7 +100,7 @@ public static void dropTableData(FileIO io, TableMetadata metadata) { L

Re: [PR] Api: Track partition statistics via TableMetadata [iceberg]

aokolnychyi commented on PR #8502: URL: https://github.com/apache/iceberg/pull/8502#issuecomment-1856508439 @ajantha-bhat, I'll take a look at other PRs once this is in. I feel this one is almost ready to go. -- This is an automated message from the Apache Git Service. To respond to the m

Re: [PR] Api: Track partition statistics via TableMetadata [iceberg]

aokolnychyi commented on code in PR #8502: URL: https://github.com/apache/iceberg/pull/8502#discussion_r1427180249 ## core/src/main/java/org/apache/iceberg/TableMetadata.java: ## @@ -946,15 +955,25 @@ private Builder(TableMetadata base) { this.previousFileLocation = base.

Re: [PR] WIP: Glue catalog commit [iceberg-python]

jackye1995 commented on code in PR #140: URL: https://github.com/apache/iceberg-python/pull/140#discussion_r1427178174 ## pyiceberg/catalog/glue.py: ## @@ -59,21 +61,33 @@ from pyiceberg.partitioning import UNPARTITIONED_PARTITION_SPEC, PartitionSpec from pyiceberg.schema impo

Re: [PR] Add Hive integration tests [iceberg-python]

Fokko commented on code in PR #207: URL: https://github.com/apache/iceberg-python/pull/207#discussion_r1427172141 ## dev/Dockerfile: ## @@ -62,7 +60,7 @@ RUN chmod u+x /opt/spark/sbin/* && \ RUN pip3 install -q ipython -RUN pip3 install "pyiceberg[s3fs]==${PYICEBERG_VERSION

Re: [PR] Add Hive integration tests [iceberg-python]

Fokko commented on code in PR #207: URL: https://github.com/apache/iceberg-python/pull/207#discussion_r1427153974 ## tests/integration/test_hive.py: ## @@ -0,0 +1,409 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See

Re: [PR] Add Hive integration tests [iceberg-python]

Fokko commented on code in PR #207: URL: https://github.com/apache/iceberg-python/pull/207#discussion_r1427150320 ## tests/integration/__init__.py: ## @@ -0,0 +1,16 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See t

Re: [PR] Build: Bump mypy-boto3-glue from 1.33.5 to 1.34.0 [iceberg-python]

Fokko merged PR #213: URL: https://github.com/apache/iceberg-python/pull/213 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] BugFix: ORC reader is not closed when SortedMerge iterator is used for positional deletes [iceberg]

deniskuzZ commented on code in PR #9301: URL: https://github.com/apache/iceberg/pull/9301#discussion_r1427131740 ## orc/src/main/java/org/apache/iceberg/orc/OrcIterable.java: ## @@ -134,7 +134,13 @@ private static VectorizedRowBatchIterator newOrcIterator( try { re

Re: [PR] BugFix: ORC reader is not closed when SortedMerge iterator is used for positional deletes [iceberg]

deniskuzZ commented on code in PR #9301: URL: https://github.com/apache/iceberg/pull/9301#discussion_r1427131740 ## orc/src/main/java/org/apache/iceberg/orc/OrcIterable.java: ## @@ -134,7 +134,13 @@ private static VectorizedRowBatchIterator newOrcIterator( try { re

Re: [PR] BugFix: ORC reader is not closed when SortedMerge iterator is used for positional deletes [iceberg]

deniskuzZ commented on code in PR #9301: URL: https://github.com/apache/iceberg/pull/9301#discussion_r1427131740 ## orc/src/main/java/org/apache/iceberg/orc/OrcIterable.java: ## @@ -134,7 +134,13 @@ private static VectorizedRowBatchIterator newOrcIterator( try { re

Re: [PR] Core: Look up targeted position deletes by path [iceberg]

aokolnychyi commented on code in PR #9251: URL: https://github.com/apache/iceberg/pull/9251#discussion_r1427120373 ## core/src/main/java/org/apache/iceberg/DeleteFileIndex.java: ## @@ -582,93 +513,187 @@ private Iterable>> deleteManifestRea } } - // a group of indexe

Re: [PR] Core: Look up targeted position deletes by path [iceberg]

aokolnychyi commented on code in PR #9251: URL: https://github.com/apache/iceberg/pull/9251#discussion_r1427120373 ## core/src/main/java/org/apache/iceberg/DeleteFileIndex.java: ## @@ -582,93 +513,187 @@ private Iterable>> deleteManifestRea } } - // a group of indexe

Re: [PR] API, Core, Spark 3.5: Parallelize reading of deletes and cache them on executors [iceberg]

aokolnychyi commented on code in PR #8755: URL: https://github.com/apache/iceberg/pull/8755#discussion_r1427103384 ## core/src/main/java/org/apache/iceberg/util/ThreadPools.java: ## @@ -68,8 +68,9 @@ public static ExecutorService getWorkerPool() { /** * Return an {@link E

Re: [PR] BugFix: ORC reader is not closed when SortedMerge iterator is used for positional deletes [iceberg]

deniskuzZ commented on code in PR #9301: URL: https://github.com/apache/iceberg/pull/9301#discussion_r1426947971 ## orc/src/main/java/org/apache/iceberg/orc/OrcIterable.java: ## @@ -134,7 +134,13 @@ private static VectorizedRowBatchIterator newOrcIterator( try { re

Re: [PR] BugFix: ORC reader is not closed when SortedMerge iterator is used for positional deletes [iceberg]

deniskuzZ commented on code in PR #9301: URL: https://github.com/apache/iceberg/pull/9301#discussion_r1426947971 ## orc/src/main/java/org/apache/iceberg/orc/OrcIterable.java: ## @@ -134,7 +134,13 @@ private static VectorizedRowBatchIterator newOrcIterator( try { re

Re: [PR] BugFix: ORC reader is not closed when SortedMerge iterator is used for positional deletes [iceberg]

deniskuzZ commented on code in PR #9301: URL: https://github.com/apache/iceberg/pull/9301#discussion_r1426947971 ## orc/src/main/java/org/apache/iceberg/orc/OrcIterable.java: ## @@ -134,7 +134,13 @@ private static VectorizedRowBatchIterator newOrcIterator( try { re

Re: [PR] BugFix: ORC reader is not closed when SortedMerge iterator is used for positional deletes [iceberg]

deniskuzZ commented on code in PR #9301: URL: https://github.com/apache/iceberg/pull/9301#discussion_r1426947971 ## orc/src/main/java/org/apache/iceberg/orc/OrcIterable.java: ## @@ -134,7 +134,13 @@ private static VectorizedRowBatchIterator newOrcIterator( try { re

Re: [PR] BugFix: ORC reader is not closed when SortedMerge iterator is used for positional deletes [iceberg]

deniskuzZ commented on code in PR #9301: URL: https://github.com/apache/iceberg/pull/9301#discussion_r1426947971 ## orc/src/main/java/org/apache/iceberg/orc/OrcIterable.java: ## @@ -134,7 +134,13 @@ private static VectorizedRowBatchIterator newOrcIterator( try { re

Re: [PR] BugFix: ORC reader is not closed when SortedMerge iterator is used for positional deletes [iceberg]

deniskuzZ commented on code in PR #9301: URL: https://github.com/apache/iceberg/pull/9301#discussion_r1426947971 ## orc/src/main/java/org/apache/iceberg/orc/OrcIterable.java: ## @@ -134,7 +134,13 @@ private static VectorizedRowBatchIterator newOrcIterator( try { re

Re: [PR] BugFix: ORC reader is not closed when SortedMerge iterator is used for positional deletes [iceberg]

deniskuzZ commented on code in PR #9301: URL: https://github.com/apache/iceberg/pull/9301#discussion_r1426947971 ## orc/src/main/java/org/apache/iceberg/orc/OrcIterable.java: ## @@ -134,7 +134,13 @@ private static VectorizedRowBatchIterator newOrcIterator( try { re

Re: [PR] Rest Catalog: Add RESTful AppendFiles data operation [iceberg]

geruh commented on code in PR #9292: URL: https://github.com/apache/iceberg/pull/9292#discussion_r1426221963 ## open-api/rest-catalog-open-api.yaml: ## @@ -2367,6 +2368,21 @@ components: items: type: string +AppendFilesUpdate: + allOf: +

Re: [PR] Rest Catalog: Add RESTful AppendFiles data operation [iceberg]

geruh commented on code in PR #9292: URL: https://github.com/apache/iceberg/pull/9292#discussion_r1426221963 ## open-api/rest-catalog-open-api.yaml: ## @@ -2367,6 +2368,21 @@ components: items: type: string +AppendFilesUpdate: + allOf: +

Re: [PR] API, Core, Spark 3.5: Parallelize reading of deletes and cache them on executors [iceberg]

RussellSpitzer commented on code in PR #8755: URL: https://github.com/apache/iceberg/pull/8755#discussion_r1427064092 ## core/src/main/java/org/apache/iceberg/util/ThreadPools.java: ## @@ -68,8 +68,9 @@ public static ExecutorService getWorkerPool() { /** * Return an {@lin

Re: [PR] BugFix: ORC reader is not closed when SortedMerge iterator is used for positional deletes [iceberg]

deniskuzZ commented on code in PR #9301: URL: https://github.com/apache/iceberg/pull/9301#discussion_r1427023155 ## orc/src/main/java/org/apache/iceberg/orc/OrcIterable.java: ## @@ -134,7 +134,13 @@ private static VectorizedRowBatchIterator newOrcIterator( try { re

Re: [PR] BugFix: ORC reader is not closed when SortedMerge iterator is used for positional deletes [iceberg]

deniskuzZ commented on code in PR #9301: URL: https://github.com/apache/iceberg/pull/9301#discussion_r1427023155 ## orc/src/main/java/org/apache/iceberg/orc/OrcIterable.java: ## @@ -134,7 +134,13 @@ private static VectorizedRowBatchIterator newOrcIterator( try { re

Re: [PR] Add validations for the rewritten data [iceberg]

RussellSpitzer commented on code in PR #9303: URL: https://github.com/apache/iceberg/pull/9303#discussion_r1427019533 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/SparkSizeBasedDataRewriter.java: ## @@ -57,7 +60,30 @@ public Set rewrite(List group) {

Re: [PR] BugFix: ORC reader is not closed when SortedMerge iterator is used for positional deletes [iceberg]

pvary commented on code in PR #9301: URL: https://github.com/apache/iceberg/pull/9301#discussion_r1427017701 ## orc/src/main/java/org/apache/iceberg/orc/OrcIterable.java: ## @@ -134,7 +134,13 @@ private static VectorizedRowBatchIterator newOrcIterator( try { return

Re: [PR] Add validations for the rewritten data [iceberg]

RussellSpitzer commented on code in PR #9303: URL: https://github.com/apache/iceberg/pull/9303#discussion_r1427015161 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/SparkSizeBasedDataRewriter.java: ## @@ -57,7 +60,30 @@ public Set rewrite(List group) {

Re: [PR] BugFix: ORC reader is not closed when SortedMerge iterator is used for positional deletes [iceberg]

deniskuzZ commented on code in PR #9301: URL: https://github.com/apache/iceberg/pull/9301#discussion_r1427000705 ## orc/src/main/java/org/apache/iceberg/orc/OrcIterable.java: ## @@ -134,7 +134,13 @@ private static VectorizedRowBatchIterator newOrcIterator( try { re

Re: [I] Unable to merge CDC data into snapshot data. java.lang.ClassCastException: org.apache.spark.unsafe.types.UTF8String cannot be cast to java.lang.Long [iceberg]

sabyasachinandy commented on issue #8333: URL: https://github.com/apache/iceberg/issues/8333#issuecomment-1856243795 Adding to this, I did a quick check of the stackstrace, seems like something is happening after the addMetadataRows function is getting called. It creates a generated cla

Re: [PR] Core: Look up targeted position deletes by path [iceberg]

szehon-ho commented on code in PR #9251: URL: https://github.com/apache/iceberg/pull/9251#discussion_r1426773779 ## core/src/main/java/org/apache/iceberg/DeleteFileIndex.java: ## @@ -582,93 +513,187 @@ private Iterable>> deleteManifestRea } } - // a group of indexed

Re: [PR] BugFix: ORC reader is not closed when SortedMerge iterator is used for positional deletes [iceberg]

deniskuzZ commented on code in PR #9301: URL: https://github.com/apache/iceberg/pull/9301#discussion_r1427000705 ## orc/src/main/java/org/apache/iceberg/orc/OrcIterable.java: ## @@ -134,7 +134,13 @@ private static VectorizedRowBatchIterator newOrcIterator( try { re

Re: [PR] Core, API: Move SQLViewRepresentation to API [iceberg]

rdblue closed pull request #9278: Core, API: Move SQLViewRepresentation to API URL: https://github.com/apache/iceberg/pull/9278 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] Core, API: Move SQLViewRepresentation to API [iceberg]

rdblue commented on PR #9278: URL: https://github.com/apache/iceberg/pull/9278#issuecomment-1856181463 Closing in favor of #9302 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

Re: [PR] API, Core: Move SQLViewRepresentation to API [iceberg]

nastra commented on PR #9302: URL: https://github.com/apache/iceberg/pull/9302#issuecomment-1856177487 thanks for the review @amogh-jahagirdar -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] API, Core: Move SQLViewRepresentation to API [iceberg]

nastra merged PR #9302: URL: https://github.com/apache/iceberg/pull/9302 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

  1   2   >