Re: [PR] Publish Apache Iceberg 1.6.0 release [iceberg]

2024-07-23 Thread via GitHub
ajantha-bhat commented on code in PR #10752: URL: https://github.com/apache/iceberg/pull/10752#discussion_r1689229611 ## site/docs/releases.md: ## @@ -71,6 +71,90 @@ To add a dependency on Iceberg in Maven, add the following to your `pom.xml`: ... ``` + +### 1.6.0 release

Re: [PR] mr:Fix issues 10639 [iceberg]

2024-07-23 Thread via GitHub
lurnagao-dahua commented on PR #10661: URL: https://github.com/apache/iceberg/pull/10661#issuecomment-2247028070 Gentle ping for reviews @pvary Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] View Spec implementation [iceberg-rust]

2024-07-23 Thread via GitHub
c-thiel commented on code in PR #331: URL: https://github.com/apache/iceberg-rust/pull/331#discussion_r1688304774 ## crates/iceberg/src/spec/view_metadata.rs: ## @@ -0,0 +1,675 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agre

Re: [PR] Publish Apache Iceberg 1.6.0 release [iceberg]

2024-07-23 Thread via GitHub
jbonofre commented on code in PR #10752: URL: https://github.com/apache/iceberg/pull/10752#discussion_r1689221137 ## site/docs/releases.md: ## @@ -71,6 +71,35 @@ To add a dependency on Iceberg in Maven, add the following to your `pom.xml`: ... ``` + +### 1.6.0 release +Ap

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-07-23 Thread via GitHub
rahil-c commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1689200439 ## open-api/rest-catalog-open-api.yaml: ## @@ -3647,6 +3818,176 @@ components: type: integer description: "List of equality field IDs" +Pr

Re: [PR] UpdatePartitionSpec: Added ability to not set the new partition spec as default [iceberg]

2024-07-23 Thread via GitHub
shanielh commented on PR #10736: URL: https://github.com/apache/iceberg/pull/10736#issuecomment-2246992983 > This seems reasonable but we do need to add some tests to TestUpdatePartitionSpec to validate the behavior I had to add the spec to TestTableUpdatePartitionSpec since that's th

Re: [PR] AddPartitionSpec: A new way to set new partition specs [iceberg]

2024-07-23 Thread via GitHub
shanielh commented on code in PR #10737: URL: https://github.com/apache/iceberg/pull/10737#discussion_r1689195839 ## api/src/main/java/org/apache/iceberg/AddPartitionSpec.java: ## @@ -0,0 +1,91 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more con

Re: [PR] Publish Apache Iceberg 1.6.0 release [iceberg]

2024-07-23 Thread via GitHub
ajantha-bhat commented on code in PR #10752: URL: https://github.com/apache/iceberg/pull/10752#discussion_r1689181588 ## site/docs/releases.md: ## @@ -71,6 +71,35 @@ To add a dependency on Iceberg in Maven, add the following to your `pom.xml`: ... ``` + +### 1.6.0 release

Re: [PR] REST: AuthManager API [iceberg]

2024-07-23 Thread via GitHub
ajantha-bhat commented on code in PR #10753: URL: https://github.com/apache/iceberg/pull/10753#discussion_r1689182347 ## .palantir/revapi.yml: ## @@ -1056,6 +1056,9 @@ acceptedBreaks: - code: "java.method.removed" old: "method org.apache.iceberg.DataFiles.Builder or

Re: [PR] Docs: Add bodo to iceberg vendors [iceberg]

2024-07-23 Thread via GitHub
ajantha-bhat commented on code in PR #10756: URL: https://github.com/apache/iceberg/pull/10756#discussion_r1689178731 ## site/docs/vendors.md: ## @@ -21,6 +21,12 @@ title: "Vendors" ## Vendors Supporting Iceberg Tables This page contains some of the vendors who are shipping

Re: [PR] Update .asf.yaml [iceberg]

2024-07-23 Thread via GitHub
jbonofre commented on code in PR #10767: URL: https://github.com/apache/iceberg/pull/10767#discussion_r1689177588 ## .asf.yaml: ## @@ -49,7 +49,6 @@ github: - jun-he - marton-bod - samarthjain -- findepi Review Comment: I agree with the 6 months period. It

Re: [PR] Update .asf.yaml [iceberg]

2024-07-23 Thread via GitHub
jbonofre commented on code in PR #10767: URL: https://github.com/apache/iceberg/pull/10767#discussion_r1689177048 ## .asf.yaml: ## @@ -49,7 +49,6 @@ github: - jun-he - marton-bod - samarthjain -- findepi Review Comment: @ajantha-bhat 😄 Do you want to add m

Re: [I] flink iceberg may occur duplication when succeed to write datafile and commit but checkpoint fail [iceberg]

2024-07-23 Thread via GitHub
pvary commented on issue #10765: URL: https://github.com/apache/iceberg/issues/10765#issuecomment-2246955538 Is this similar to #10526? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

Re: [PR] Update .asf.yaml [iceberg]

2024-07-23 Thread via GitHub
ajantha-bhat commented on code in PR #10767: URL: https://github.com/apache/iceberg/pull/10767#discussion_r1689170878 ## .asf.yaml: ## @@ -49,7 +49,6 @@ github: - jun-he - marton-bod - samarthjain -- findepi Review Comment: @findepi: Congratulations on bei

Re: [PR] Bump mkdocs-material from 9.5.29 to 9.5.30 [iceberg-python]

2024-07-23 Thread via GitHub
Fokko merged PR #959: URL: https://github.com/apache/iceberg-python/pull/959 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] Update .asf.yaml [iceberg]

2024-07-23 Thread via GitHub
ajantha-bhat commented on code in PR #10767: URL: https://github.com/apache/iceberg/pull/10767#discussion_r1689171429 ## .asf.yaml: ## @@ -49,7 +49,6 @@ github: - jun-he - marton-bod - samarthjain -- findepi Review Comment: And @jbonofre: Since you are mor

Re: [PR] Update .asf.yaml [iceberg]

2024-07-23 Thread via GitHub
ajantha-bhat commented on code in PR #10767: URL: https://github.com/apache/iceberg/pull/10767#discussion_r1689170878 ## .asf.yaml: ## @@ -49,7 +49,6 @@ github: - jun-he - marton-bod - samarthjain -- findepi Review Comment: @findepi: Congratulations on bei

Re: [PR] Flink: backport PR #10331 and PR #10457 [iceberg]

2024-07-23 Thread via GitHub
pvary commented on PR #10757: URL: https://github.com/apache/iceberg/pull/10757#issuecomment-2246943273 @stevenzwu: Could you please help me understand why are the changes in flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/SortKeySerializer.java are needed? Than

Re: [PR] Publish Apache Iceberg 1.6.0 release [iceberg]

2024-07-23 Thread via GitHub
ajantha-bhat commented on code in PR #10752: URL: https://github.com/apache/iceberg/pull/10752#discussion_r1689160541 ## site/docs/releases.md: ## @@ -71,6 +71,35 @@ To add a dependency on Iceberg in Maven, add the following to your `pom.xml`: ... ``` + +### 1.6.0 release

Re: [PR] Flink: support limit pushdown in FLIP-27 source [iceberg]

2024-07-23 Thread via GitHub
pvary commented on code in PR #10748: URL: https://github.com/apache/iceberg/pull/10748#discussion_r1689160834 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/source/reader/RowDataReaderFunction.java: ## @@ -61,19 +86,30 @@ public RowDataReaderFunction( this.io

Re: [PR] Publish Apache Iceberg 1.6.0 release [iceberg]

2024-07-23 Thread via GitHub
jbonofre commented on code in PR #10752: URL: https://github.com/apache/iceberg/pull/10752#discussion_r1689157041 ## site/docs/releases.md: ## @@ -71,6 +71,35 @@ To add a dependency on Iceberg in Maven, add the following to your `pom.xml`: ... ``` + +### 1.6.0 release +Ap

Re: [PR] Support for Flink's SpeculativeExecution in batch execution mode [iceberg]

2024-07-23 Thread via GitHub
pvary commented on PR #10548: URL: https://github.com/apache/iceberg/pull/10548#issuecomment-2246926879 Merged to main. Thanks for the PR @venkata91! Could you please create the backport PR to the other Flink versions? The PR could be generated like this: ``` git diff ^ | se

Re: [PR] Support for Flink's SpeculativeExecution in batch execution mode [iceberg]

2024-07-23 Thread via GitHub
pvary merged PR #10548: URL: https://github.com/apache/iceberg/pull/10548 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] Core: Add estimateRowCount for Files and Entries Metadata Tables [iceberg]

2024-07-23 Thread via GitHub
szehon-ho commented on PR #10759: URL: https://github.com/apache/iceberg/pull/10759#issuecomment-2246857283 cc @amogh-jahagirdar @RussellSpitzer does it make sense? Thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

Re: [PR] Add `IcebergAnalysisException` in iceberg-spark module [iceberg]

2024-07-23 Thread via GitHub
szehon-ho commented on PR #10766: URL: https://github.com/apache/iceberg/pull/10766#issuecomment-2246856254 Hi @huaxingao thanks, this mostly makes sense to me, but just wanted to get the motivation. Is it so the user can see the error comes from Iceberg vs other part of Spark? Also, is t

Re: [PR] API: Add RemoveUnusedSpecs in Table [iceberg]

2024-07-23 Thread via GitHub
advancedxy commented on code in PR #10755: URL: https://github.com/apache/iceberg/pull/10755#discussion_r1689069772 ## core/src/main/java/org/apache/iceberg/TableMetadata.java: ## @@ -597,6 +597,12 @@ public TableMetadata replaceProperties(Map rawProperties) { .build()

Re: [PR] API: Add RemoveUnusedSpecs in Table [iceberg]

2024-07-23 Thread via GitHub
advancedxy commented on PR #10755: URL: https://github.com/apache/iceberg/pull/10755#issuecomment-2246811037 > I don't think I'd go with a general SetPartitionSpecs update, I think I'd have a RemovePartitionSpec, and the TableMetadata builder APIs to remove a given spec (which will have va

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-07-23 Thread via GitHub
rahil-c commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1689067372 ## open-api/rest-catalog-open-api.yaml: ## @@ -3647,6 +3818,176 @@ components: type: integer description: "List of equality field IDs" +Pr

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-07-23 Thread via GitHub
rahil-c commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1689057944 ## open-api/rest-catalog-open-api.yaml: ## @@ -3647,6 +3818,176 @@ components: type: integer description: "List of equality field IDs" +Pr

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-07-23 Thread via GitHub
rahil-c commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1689054787 ## open-api/rest-catalog-open-api.yaml: ## @@ -3647,6 +3818,176 @@ components: type: integer description: "List of equality field IDs" +Pr

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-07-23 Thread via GitHub
rahil-c commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1689052919 ## open-api/rest-catalog-open-api.yaml: ## @@ -3647,6 +3818,176 @@ components: type: integer description: "List of equality field IDs" +Pr

Re: [I] [feat] Ability to read/write table using `version-hint.txt` [iceberg-python]

2024-07-23 Thread via GitHub
djouallah commented on issue #763: URL: https://github.com/apache/iceberg-python/issues/763#issuecomment-2246766948 it is quite ironic, it seems the only iceberg vendor who generate hint.text is snowflake !!! go figure -- This is an automated message from the Apache Git Service. To respo

Re: [PR] Add `IcebergAnalysisException` in iceberg-spark module [iceberg]

2024-07-23 Thread via GitHub
huaxingao commented on PR #10766: URL: https://github.com/apache/iceberg/pull/10766#issuecomment-2246758141 cc @szehon-ho -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

Re: [I] [feat] Ability to read/write table using `version-hint.txt` [iceberg-python]

2024-07-23 Thread via GitHub
kevinjqliu commented on issue #763: URL: https://github.com/apache/iceberg-python/issues/763#issuecomment-2246747456 It would be great if duckdb_iceberg could support reading directly from the catalog. -- This is an automated message from the Apache Git Service. To respond to the mes

Re: [I] [feat] Ability to read/write table using `version-hint.txt` [iceberg-python]

2024-07-23 Thread via GitHub
kevinjqliu commented on issue #763: URL: https://github.com/apache/iceberg-python/issues/763#issuecomment-2246746210 @lamb-russell duckdb_iceberg can read the "metadata json file" directly. See https://github.com/steven-luabase/duckdb-iceberg-demo/issues/1#issuecomment-2215482225 -

Re: [I] [feat] Ability to read/write table using `version-hint.txt` [iceberg-python]

2024-07-23 Thread via GitHub
lamb-russell commented on issue #763: URL: https://github.com/apache/iceberg-python/issues/763#issuecomment-2246727745 DuckDB appears to depend on the version-hint.text file when scanning iceberg. https://github.com/user-attachments/assets/9aea6acb-e909-4866-9b7f-a1826b2a79c8";>

Re: [I] Is dataFiles() Method Retryable? [iceberg]

2024-07-23 Thread via GitHub
osy497 commented on issue #10750: URL: https://github.com/apache/iceberg/issues/10750#issuecomment-2246694505 @nk1506 I got something like these: ``` Caused by: software.amazon.awssdk.services.s3.model.S3Exception: Remote backend is unreachable (ConcurrentModification: concurrent m

Re: [PR] AddPartitionSpec: A new way to set new partition specs [iceberg]

2024-07-23 Thread via GitHub
shanielh commented on code in PR #10737: URL: https://github.com/apache/iceberg/pull/10737#discussion_r1688994691 ## api/src/main/java/org/apache/iceberg/AddPartitionSpec.java: ## @@ -0,0 +1,91 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more con

Re: [PR] AddPartitionSpec: A new way to set new partition specs [iceberg]

2024-07-23 Thread via GitHub
shanielh commented on code in PR #10737: URL: https://github.com/apache/iceberg/pull/10737#discussion_r1688993988 ## api/src/main/java/org/apache/iceberg/Transaction.java: ## @@ -44,6 +44,13 @@ public interface Transaction { */ UpdatePartitionSpec updateSpec(); + /** +

Re: [PR] fix table.delete()/overwrite() with null values [iceberg-python]

2024-07-23 Thread via GitHub
syun64 commented on code in PR #955: URL: https://github.com/apache/iceberg-python/pull/955#discussion_r1688973596 ## tests/io/test_pyarrow_visitor.py: ## @@ -580,3 +594,127 @@ def test_pyarrow_schema_ensure_large_types(pyarrow_schema_nested_without_ids: pa ), ])

[PR] Add `IcebergAnalysisException` in iceberg-spark module [iceberg]

2024-07-23 Thread via GitHub
huaxingao opened a new pull request, #10766: URL: https://github.com/apache/iceberg/pull/10766 In Iceberg's `spark-extensions`, if we encounter any invalid SQL statements, we throw Spark's `AnalysisException`. However, it would be better to have Iceberg's own `IcebergAnalysisException` to d

Re: [PR] fix table.delete()/overwrite() with null values [iceberg-python]

2024-07-23 Thread via GitHub
syun64 commented on code in PR #955: URL: https://github.com/apache/iceberg-python/pull/955#discussion_r1688971266 ## pyiceberg/io/pyarrow.py: ## @@ -638,10 +634,165 @@ def visit_or(self, left_result: pc.Expression, right_result: pc.Expression) -> p return left_result

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-07-23 Thread via GitHub
rahil-c commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1688966381 ## open-api/rest-catalog-open-api.yaml: ## @@ -3647,6 +3818,176 @@ components: type: integer description: "List of equality field IDs" +Pr

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-07-23 Thread via GitHub
rahil-c commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1688966126 ## open-api/rest-catalog-open-api.yaml: ## @@ -3647,6 +3818,176 @@ components: type: integer description: "List of equality field IDs" +Pr

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-07-23 Thread via GitHub
jeremysong commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1688961718 ## open-api/rest-catalog-open-api.yaml: ## @@ -3647,6 +3818,176 @@ components: type: integer description: "List of equality field IDs" +

Re: [PR] Hive: Push filtering for Iceberg table type to Hive MetaStore when listing tables [iceberg]

2024-07-23 Thread via GitHub
github-actions[bot] closed pull request #2722: Hive: Push filtering for Iceberg table type to Hive MetaStore when listing tables URL: https://github.com/apache/iceberg/pull/2722 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] Fix the problem that the metastore-client is not closed [iceberg]

2024-07-23 Thread via GitHub
github-actions[bot] commented on PR #2785: URL: https://github.com/apache/iceberg/pull/2785#issuecomment-2246614278 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Flink: merge the configurations of hive-conf-dir to CatalogLoader [iceberg]

2024-07-23 Thread via GitHub
github-actions[bot] commented on PR #2799: URL: https://github.com/apache/iceberg/pull/2799#issuecomment-2246614297 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Spark: Add Multi-thread to construct ReadTask [iceberg]

2024-07-23 Thread via GitHub
github-actions[bot] closed pull request #2803: Spark: Add Multi-thread to construct ReadTask URL: https://github.com/apache/iceberg/pull/2803 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

Re: [PR] Docs : Add snapshot incremental read on spark doc [iceberg]

2024-07-23 Thread via GitHub
github-actions[bot] closed pull request #2657: Docs : Add snapshot incremental read on spark doc URL: https://github.com/apache/iceberg/pull/2657 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] Fix the problem that the metastore-client is not closed [iceberg]

2024-07-23 Thread via GitHub
github-actions[bot] closed pull request #2785: Fix the problem that the metastore-client is not closed URL: https://github.com/apache/iceberg/pull/2785 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

Re: [PR] Support all operations in incremental scan [iceberg]

2024-07-23 Thread via GitHub
github-actions[bot] closed pull request #2782: Support all operations in incremental scan URL: https://github.com/apache/iceberg/pull/2782 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

Re: [PR] Spark3: Disable catalog cache-enabled. [iceberg]

2024-07-23 Thread via GitHub
github-actions[bot] commented on PR #2659: URL: https://github.com/apache/iceberg/pull/2659#issuecomment-2246614127 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] fix wrong position to get field from record [iceberg]

2024-07-23 Thread via GitHub
github-actions[bot] commented on PR #2711: URL: https://github.com/apache/iceberg/pull/2711#issuecomment-2246614143 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] [WIP] Core: Support relative paths in metadata [iceberg]

2024-07-23 Thread via GitHub
github-actions[bot] closed pull request #2658: [WIP] Core: Support relative paths in metadata URL: https://github.com/apache/iceberg/pull/2658 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

Re: [PR] [WIP] Core: Support relative paths in metadata [iceberg]

2024-07-23 Thread via GitHub
github-actions[bot] commented on PR #2658: URL: https://github.com/apache/iceberg/pull/2658#issuecomment-2246614109 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Docs : Add snapshot incremental read on spark doc [iceberg]

2024-07-23 Thread via GitHub
github-actions[bot] commented on PR #2657: URL: https://github.com/apache/iceberg/pull/2657#issuecomment-2246614085 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Core: Skip delete data files when commit-state is unknown. [iceberg]

2024-07-23 Thread via GitHub
github-actions[bot] closed pull request #2621: Core: Skip delete data files when commit-state is unknown. URL: https://github.com/apache/iceberg/pull/2621 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

Re: [PR] Fix when dropping partition fields for the same source field [iceberg]

2024-07-23 Thread via GitHub
github-actions[bot] closed pull request #2570: Fix when dropping partition fields for the same source field URL: https://github.com/apache/iceberg/pull/2570 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] Flink : vectorized read of orc format in flink [iceberg]

2024-07-23 Thread via GitHub
github-actions[bot] closed pull request #2566: Flink : vectorized read of orc format in flink URL: https://github.com/apache/iceberg/pull/2566 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

Re: [PR] Core: add key_metadata in ManifestFile [iceberg]

2024-07-23 Thread via GitHub
github-actions[bot] commented on PR #2520: URL: https://github.com/apache/iceberg/pull/2520#issuecomment-2246613967 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Doc: fix error and enhanced iceberg catalog description for the flink DataStream API [iceberg]

2024-07-23 Thread via GitHub
github-actions[bot] closed pull request #2389: Doc: fix error and enhanced iceberg catalog description for the flink DataStream API URL: https://github.com/apache/iceberg/pull/2389 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] Core: introduce envelope encryption API and spec [iceberg]

2024-07-23 Thread via GitHub
github-actions[bot] commented on PR #2443: URL: https://github.com/apache/iceberg/pull/2443#issuecomment-2246613857 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Spark: Add Multi-thread to construct ReadTask [iceberg]

2024-07-23 Thread via GitHub
github-actions[bot] commented on PR #2803: URL: https://github.com/apache/iceberg/pull/2803#issuecomment-2246614317 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Flink: merge the configurations of hive-conf-dir to CatalogLoader [iceberg]

2024-07-23 Thread via GitHub
github-actions[bot] closed pull request #2799: Flink: merge the configurations of hive-conf-dir to CatalogLoader URL: https://github.com/apache/iceberg/pull/2799 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

Re: [PR] Support all operations in incremental scan [iceberg]

2024-07-23 Thread via GitHub
github-actions[bot] commented on PR #2782: URL: https://github.com/apache/iceberg/pull/2782#issuecomment-2246614257 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Avro inferred mapping issues with nested struct [iceberg]

2024-07-23 Thread via GitHub
github-actions[bot] commented on PR #2775: URL: https://github.com/apache/iceberg/pull/2775#issuecomment-2246614236 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Core: Use current user ugi to run in WORKER_POOL to prevent wrong cre… [iceberg]

2024-07-23 Thread via GitHub
github-actions[bot] closed pull request #2754: Core: Use current user ugi to run in WORKER_POOL to prevent wrong cre… URL: https://github.com/apache/iceberg/pull/2754 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

Re: [PR] Avro inferred mapping issues with nested struct [iceberg]

2024-07-23 Thread via GitHub
github-actions[bot] closed pull request #2775: Avro inferred mapping issues with nested struct URL: https://github.com/apache/iceberg/pull/2775 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

Re: [PR] Core: Use current user ugi to run in WORKER_POOL to prevent wrong cre… [iceberg]

2024-07-23 Thread via GitHub
github-actions[bot] commented on PR #2754: URL: https://github.com/apache/iceberg/pull/2754#issuecomment-2246614204 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Hive: Push filtering for Iceberg table type to Hive MetaStore when listing tables [iceberg]

2024-07-23 Thread via GitHub
github-actions[bot] commented on PR #2722: URL: https://github.com/apache/iceberg/pull/2722#issuecomment-2246614176 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] fix wrong position to get field from record [iceberg]

2024-07-23 Thread via GitHub
github-actions[bot] closed pull request #2711: fix wrong position to get field from record URL: https://github.com/apache/iceberg/pull/2711 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

Re: [PR] Spark3: Disable catalog cache-enabled. [iceberg]

2024-07-23 Thread via GitHub
github-actions[bot] closed pull request #2659: Spark3: Disable catalog cache-enabled. URL: https://github.com/apache/iceberg/pull/2659 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

Re: [PR] Core : Repair manifests [iceberg]

2024-07-23 Thread via GitHub
github-actions[bot] commented on PR #2608: URL: https://github.com/apache/iceberg/pull/2608#issuecomment-2246614043 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Core: Skip delete data files when commit-state is unknown. [iceberg]

2024-07-23 Thread via GitHub
github-actions[bot] commented on PR #2621: URL: https://github.com/apache/iceberg/pull/2621#issuecomment-2246614062 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Core : Repair manifests [iceberg]

2024-07-23 Thread via GitHub
github-actions[bot] closed pull request #2608: Core : Repair manifests URL: https://github.com/apache/iceberg/pull/2608 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsub

Re: [PR] MR: remove Hive dependencies on Iceberg de/serialization utility functions [iceberg]

2024-07-23 Thread via GitHub
github-actions[bot] closed pull request #2598: MR: remove Hive dependencies on Iceberg de/serialization utility functions URL: https://github.com/apache/iceberg/pull/2598 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

Re: [PR] MR: remove Hive dependencies on Iceberg de/serialization utility functions [iceberg]

2024-07-23 Thread via GitHub
github-actions[bot] commented on PR #2598: URL: https://github.com/apache/iceberg/pull/2598#issuecomment-2246614023 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Fix when dropping partition fields for the same source field [iceberg]

2024-07-23 Thread via GitHub
github-actions[bot] commented on PR #2570: URL: https://github.com/apache/iceberg/pull/2570#issuecomment-2246614006 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Flink : vectorized read of orc format in flink [iceberg]

2024-07-23 Thread via GitHub
github-actions[bot] commented on PR #2566: URL: https://github.com/apache/iceberg/pull/2566#issuecomment-2246613986 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Core: add key_metadata in ManifestFile [iceberg]

2024-07-23 Thread via GitHub
github-actions[bot] closed pull request #2520: Core: add key_metadata in ManifestFile URL: https://github.com/apache/iceberg/pull/2520 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

Re: [PR] Spark: Remove deletefiles when expiring snapshots. [iceberg]

2024-07-23 Thread via GitHub
github-actions[bot] commented on PR #2518: URL: https://github.com/apache/iceberg/pull/2518#issuecomment-2246613949 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Spark: Remove deletefiles when expiring snapshots. [iceberg]

2024-07-23 Thread via GitHub
github-actions[bot] closed pull request #2518: Spark: Remove deletefiles when expiring snapshots. URL: https://github.com/apache/iceberg/pull/2518 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] Core: Add content field to ManifestTable Schema. [iceberg]

2024-07-23 Thread via GitHub
github-actions[bot] closed pull request #2510: Core: Add content field to ManifestTable Schema. URL: https://github.com/apache/iceberg/pull/2510 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

Re: [PR] Core: Add content field to ManifestTable Schema. [iceberg]

2024-07-23 Thread via GitHub
github-actions[bot] commented on PR #2510: URL: https://github.com/apache/iceberg/pull/2510#issuecomment-2246613929 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Core : fix bugs in RewriteDataFilesAction when datafile size greater than targetFileSize [iceberg]

2024-07-23 Thread via GitHub
github-actions[bot] commented on PR #2508: URL: https://github.com/apache/iceberg/pull/2508#issuecomment-2246613903 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Core : fix bugs in RewriteDataFilesAction when datafile size greater than targetFileSize [iceberg]

2024-07-23 Thread via GitHub
github-actions[bot] closed pull request #2508: Core : fix bugs in RewriteDataFilesAction when datafile size greater than targetFileSize URL: https://github.com/apache/iceberg/pull/2508 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Gi

Re: [PR] Core: add API for table metadata file encryption [iceberg]

2024-07-23 Thread via GitHub
github-actions[bot] closed pull request #2444: Core: add API for table metadata file encryption URL: https://github.com/apache/iceberg/pull/2444 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

Re: [PR] Core: add API for table metadata file encryption [iceberg]

2024-07-23 Thread via GitHub
github-actions[bot] commented on PR #2444: URL: https://github.com/apache/iceberg/pull/2444#issuecomment-2246613880 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Core: introduce envelope encryption API and spec [iceberg]

2024-07-23 Thread via GitHub
github-actions[bot] closed pull request #2443: Core: introduce envelope encryption API and spec URL: https://github.com/apache/iceberg/pull/2443 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

Re: [PR] Doc: fix error and enhanced iceberg catalog description for the flink DataStream API [iceberg]

2024-07-23 Thread via GitHub
github-actions[bot] commented on PR #2389: URL: https://github.com/apache/iceberg/pull/2389#issuecomment-2246613841 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] GCP: Add Iceberg Catalog for GCP BigLake Metastore [iceberg]

2024-07-23 Thread via GitHub
coufon closed pull request #7412: GCP: Add Iceberg Catalog for GCP BigLake Metastore URL: https://github.com/apache/iceberg/pull/7412 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

[I] flink iceberg may occur duplication when succeed to write datafile and commit but checkpoint fail [iceberg]

2024-07-23 Thread via GitHub
maekchi opened a new issue, #10765: URL: https://github.com/apache/iceberg/issues/10765 ### Apache Iceberg version 1.4.3 ### Query engine Flink ### Please describe the bug 🐞 It seems like very rare duplicates occur in flink iceberg. Let me explain the

Re: [PR] Introduces the new IcebergSink based on the new V2 Flink Sink Abstraction [iceberg]

2024-07-23 Thread via GitHub
stevenzwu commented on code in PR #10179: URL: https://github.com/apache/iceberg/pull/10179#discussion_r1688804698 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/sink/IcebergSink.java: ## @@ -0,0 +1,811 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

Re: [PR] Ensure that RestCatalog passes user config to FileIO [iceberg-rust]

2024-07-23 Thread via GitHub
sdd closed pull request #476: Ensure that RestCatalog passes user config to FileIO URL: https://github.com/apache/iceberg-rust/pull/476 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-07-23 Thread via GitHub
rahil-c commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1688800465 ## open-api/rest-catalog-open-api.yaml: ## @@ -2774,6 +2920,30 @@ components: additionalProperties: type: string +PreplanTableResult: +

[PR] Bump mkdocs-material from 9.5.29 to 9.5.30 [iceberg-python]

2024-07-23 Thread via GitHub
dependabot[bot] opened a new pull request, #959: URL: https://github.com/apache/iceberg-python/pull/959 Bumps [mkdocs-material](https://github.com/squidfunk/mkdocs-material) from 9.5.29 to 9.5.30. Release notes Sourced from https://github.com/squidfunk/mkdocs-material/releases";>mk

Re: [PR] Core: Add estimateRowCount for Files and Entries Metadata Tables [iceberg]

2024-07-23 Thread via GitHub
szehon-ho commented on code in PR #10759: URL: https://github.com/apache/iceberg/pull/10759#discussion_r1688784074 ## core/src/test/java/org/apache/iceberg/TestMetadataTableScans.java: ## @@ -1753,4 +1753,35 @@ public void testPositionDeletesManyColumns() { assertThat(scan

Re: [I] Support for Default Values [iceberg]

2024-07-23 Thread via GitHub
RussellSpitzer commented on issue #10761: URL: https://github.com/apache/iceberg/issues/10761#issuecomment-2246394403 I think we will probably need specific Issues for Flink and Spark Support. I may narrow this one down to just the Generics API or just make some sub-issues? -- This is an

Re: [PR] Core: Adds Basic Classes for Iceberg Table Version 3 [iceberg]

2024-07-23 Thread via GitHub
RussellSpitzer commented on code in PR #10760: URL: https://github.com/apache/iceberg/pull/10760#discussion_r1688772674 ## core/src/main/java/org/apache/iceberg/V3Metadata.java: ## @@ -0,0 +1,575 @@ +/* Review Comment: This is a direct copy of V2Metadata.json -- This is

  1   2   3   >