Re: [PR] Data: Add a util to read write partition stats [iceberg]

2024-08-15 Thread via GitHub
ajantha-bhat commented on PR #10176: URL: https://github.com/apache/iceberg/pull/10176#issuecomment-2292897105 Thanks a lot for the review @lirui-apache, I hope we can get this in after a review from @RussellSpitzer and @aokolnychyi. -- This is an automated message from the Apa

Re: [PR] Flink: Fix duplicate data in Flink's upsert writer for format V2 [iceberg]

2024-08-15 Thread via GitHub
zhongqishang commented on code in PR #10526: URL: https://github.com/apache/iceberg/pull/10526#discussion_r1719379441 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/sink/IcebergFilesCommitter.java: ## @@ -426,30 +425,45 @@ private void commitOperation( } @Ov

Re: [PR] Flink: Fix duplicate data in Flink's upsert writer for format V2 [iceberg]

2024-08-15 Thread via GitHub
zhongqishang commented on code in PR #10526: URL: https://github.com/apache/iceberg/pull/10526#discussion_r1719379441 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/sink/IcebergFilesCommitter.java: ## @@ -426,30 +425,45 @@ private void commitOperation( } @Ov

Re: [PR] Flink: Fix duplicate data in Flink's upsert writer for format V2 [iceberg]

2024-08-15 Thread via GitHub
zhongqishang commented on code in PR #10526: URL: https://github.com/apache/iceberg/pull/10526#discussion_r1719368722 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/sink/IcebergFilesCommitter.java: ## @@ -426,30 +425,45 @@ private void commitOperation( } @Ov

Re: [PR] Flink: put everything together for range distribution in Flink sink [iceberg]

2024-08-15 Thread via GitHub
pvary commented on code in PR #10859: URL: https://github.com/apache/iceberg/pull/10859#discussion_r1719365548 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/sink/FlinkSink.java: ## @@ -548,21 +609,52 @@ private DataStream distributeDataStream( }

Re: [PR] Flink: put everything together for range distribution in Flink sink [iceberg]

2024-08-15 Thread via GitHub
pvary commented on code in PR #10859: URL: https://github.com/apache/iceberg/pull/10859#discussion_r1719362536 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/sink/FlinkSink.java: ## @@ -233,15 +239,66 @@ public Builder flinkConf(ReadableConfig config) { * @ret

Re: [PR] Flink: put everything together for range distribution in Flink sink [iceberg]

2024-08-15 Thread via GitHub
pvary commented on code in PR #10859: URL: https://github.com/apache/iceberg/pull/10859#discussion_r1719357623 ## docs/docs/flink-writes.md: ## @@ -262,6 +262,107 @@ INSERT INTO tableName /*+ OPTIONS('upsert-enabled'='true') */ Check out all the options here: [write-options

Re: [PR] Flink: put everything together for range distribution in Flink sink [iceberg]

2024-08-15 Thread via GitHub
pvary commented on code in PR #10859: URL: https://github.com/apache/iceberg/pull/10859#discussion_r1719348803 ## docs/docs/flink-configuration.md: ## @@ -146,14 +146,54 @@ INSERT INTO tableName /*+ OPTIONS('upsert-enabled'='true') */ ... ``` -| Flink option | Def

Re: [PR] doc: Add statement for contributors to avoid force push. [iceberg]

2024-08-15 Thread via GitHub
pvary commented on PR #10939: URL: https://github.com/apache/iceberg/pull/10939#issuecomment-2292839168 I also often use force push myself. The main cases being: 1. CI catches errors between reviews - immediate fix, and easier to review 2. Need to rebase on small upstream changes - if t

Re: [PR] Flink: put everything together for range distribution in Flink sink [iceberg]

2024-08-15 Thread via GitHub
stevenzwu commented on code in PR #10859: URL: https://github.com/apache/iceberg/pull/10859#discussion_r1719333729 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/FlinkWriteOptions.java: ## @@ -60,6 +61,14 @@ private FlinkWriteOptions() {} public static final Conf

Re: [PR] Flink: put everything together for range distribution in Flink sink [iceberg]

2024-08-15 Thread via GitHub
stevenzwu commented on code in PR #10859: URL: https://github.com/apache/iceberg/pull/10859#discussion_r1719333729 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/FlinkWriteOptions.java: ## @@ -60,6 +61,14 @@ private FlinkWriteOptions() {} public static final Conf

Re: [PR] Flink: put everything together for range distribution in Flink sink [iceberg]

2024-08-15 Thread via GitHub
stevenzwu commented on code in PR #10859: URL: https://github.com/apache/iceberg/pull/10859#discussion_r1719331652 ## flink/v1.19/flink/src/test/java/org/apache/iceberg/flink/sink/TestFlinkIcebergSinkDistributionMode.java: ## @@ -177,4 +185,288 @@ public void testOverrideWriteC

Re: [PR] 1 Cloudera blog in February 2023 [iceberg]

2024-08-15 Thread via GitHub
pvary merged PR #10947: URL: https://github.com/apache/iceberg/pull/10947 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] Spec: Minor modifications for v3 [iceberg]

2024-08-15 Thread via GitHub
emkornfield commented on code in PR #10948: URL: https://github.com/apache/iceberg/pull/10948#discussion_r1719268387 ## format/spec.md: ## @@ -1308,7 +1321,7 @@ Default values are added to struct fields in v3. Types `timestamp_ns` and `timestamptz_ns` are added in v3. -All

Re: [PR] Spec: Minor modifications for v3 [iceberg]

2024-08-15 Thread via GitHub
emkornfield commented on code in PR #10948: URL: https://github.com/apache/iceberg/pull/10948#discussion_r1719262786 ## format/spec.md: ## @@ -193,16 +204,14 @@ Supported primitive types are defined in the table below. Primitive types added Notes: -1. Decimal scale is fixe

Re: [PR] Spec: Minor modifications for v3 [iceberg]

2024-08-15 Thread via GitHub
emkornfield commented on code in PR #10948: URL: https://github.com/apache/iceberg/pull/10948#discussion_r1719261154 ## format/spec.md: ## @@ -113,9 +122,9 @@ Tables do not require random-access writes. Once written, data and metadata file Tables do not require rename, except

Re: [PR] Spec: Minor modifications for v3 [iceberg]

2024-08-15 Thread via GitHub
emkornfield commented on code in PR #10948: URL: https://github.com/apache/iceberg/pull/10948#discussion_r1719260843 ## format/spec.md: ## @@ -44,6 +44,15 @@ The primary change in version 2 adds delete files to encode rows that are delete In addition to row-level deletes, ve

Re: [PR] Spec: Minor modifications for v3 [iceberg]

2024-08-15 Thread via GitHub
emkornfield commented on code in PR #10948: URL: https://github.com/apache/iceberg/pull/10948#discussion_r1719260575 ## format/spec.md: ## @@ -44,6 +44,15 @@ The primary change in version 2 adds delete files to encode rows that are delete In addition to row-level deletes, ve

Re: [PR] doc: Add statement for contributors to avoid force push. [iceberg]

2024-08-15 Thread via GitHub
stevenzwu commented on code in PR #10939: URL: https://github.com/apache/iceberg/pull/10939#discussion_r1719259458 ## site/docs/contribute.md: ## @@ -44,6 +44,7 @@ The Iceberg community prefers to receive contributions as [Github pull requests] * It's helpful to include a pref

Re: [PR] Spec: Minor modifications for v3 [iceberg]

2024-08-15 Thread via GitHub
emkornfield commented on code in PR #10948: URL: https://github.com/apache/iceberg/pull/10948#discussion_r1719259618 ## format/spec.md: ## @@ -44,6 +44,15 @@ The primary change in version 2 adds delete files to encode rows that are delete In addition to row-level deletes, ve

Re: [PR] [DRAFT] Support changelog scan for table with delete files [iceberg]

2024-08-15 Thread via GitHub
wypoon commented on code in PR #10935: URL: https://github.com/apache/iceberg/pull/10935#discussion_r1719244230 ## core/src/main/java/org/apache/iceberg/BaseIncrementalChangelogScan.java: ## @@ -133,51 +131,155 @@ private static Map computeSnapshotOrdinals(Deque snapsh ret

Re: [PR] Table Scan Performance Tests [iceberg-rust]

2024-08-15 Thread via GitHub
Xuanwo commented on code in PR #497: URL: https://github.com/apache/iceberg-rust/pull/497#discussion_r1719222033 ## justfile: ## @@ -0,0 +1,53 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# dist

Re: [PR] doc: Add statement for contributors to avoid force push. [iceberg]

2024-08-15 Thread via GitHub
liurenjie1024 commented on code in PR #10939: URL: https://github.com/apache/iceberg/pull/10939#discussion_r1719212282 ## site/docs/contribute.md: ## @@ -44,6 +44,7 @@ The Iceberg community prefers to receive contributions as [Github pull requests] * It's helpful to include a

Re: [PR] doc: Add statement for contributors to avoid force push. [iceberg]

2024-08-15 Thread via GitHub
liurenjie1024 commented on PR #10939: URL: https://github.com/apache/iceberg/pull/10939#issuecomment-2292614179 > I believe I may have caused some confusion. The problem with force pushing arises when someone fixes the requested changes using --amend. If the person uses a new commit, it’s n

Re: [I] Spark configuration for amazon access key and secret key with glue catalog for apache Iceberg is not honoring [iceberg]

2024-08-15 Thread via GitHub
andythsu commented on issue #10078: URL: https://github.com/apache/iceberg/issues/10078#issuecomment-2292613192 facing the same issue as well. I'm on `iceberg-aws-bundle-1.6.0` but it still complains region not set -- This is an automated message from the Apache Git Service. To respond to

Re: [PR] Fix Install Issues with `docutils = 0.21.post1` and exclude 3.12 from supported python dependencies [iceberg-python]

2024-08-15 Thread via GitHub
kevinjqliu commented on PR #1067: URL: https://github.com/apache/iceberg-python/pull/1067#issuecomment-2292588035 > The proposed solution excludes 3.12 from the supported versions in the dependencies Do we not support 3.12 at all? https://github.com/apache/iceberg-python/blob/9e2

Re: [I] Running `cargo test` on `main` at the moment results in an error [iceberg-rust]

2024-08-15 Thread via GitHub
Xuanwo closed issue #551: Running `cargo test` on `main` at the moment results in an error URL: https://github.com/apache/iceberg-rust/issues/551 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] fix: ensure S3 and GCS integ tests are conditionally compiled only when the storage-s3 and storage-gcs features are enabled [iceberg-rust]

2024-08-15 Thread via GitHub
Xuanwo merged PR #552: URL: https://github.com/apache/iceberg-rust/pull/552 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.a

Re: [PR] Add metadata tables for `data_files` and `delete_files` [iceberg-python]

2024-08-15 Thread via GitHub
ndrluis commented on code in PR #1066: URL: https://github.com/apache/iceberg-python/pull/1066#discussion_r1719186698 ## pyiceberg/table/__init__.py: ## @@ -4393,12 +4397,12 @@ def _readable_metrics_struct(bound_type: PrimitiveType) -> pa.StructType: "spec_

Re: [PR] Add metadata tables for `data_files` and `delete_files` [iceberg-python]

2024-08-15 Thread via GitHub
ndrluis commented on code in PR #1066: URL: https://github.com/apache/iceberg-python/pull/1066#discussion_r1719185549 ## pyiceberg/table/__init__.py: ## @@ -4365,6 +4365,10 @@ def _readable_metrics_struct(bound_type: PrimitiveType) -> pa.StructType: for manifest_list i

Re: [PR] Add metadata tables for `data_files` and `delete_files` [iceberg-python]

2024-08-15 Thread via GitHub
ndrluis commented on code in PR #1066: URL: https://github.com/apache/iceberg-python/pull/1066#discussion_r1719185549 ## pyiceberg/table/__init__.py: ## @@ -4365,6 +4365,10 @@ def _readable_metrics_struct(bound_type: PrimitiveType) -> pa.StructType: for manifest_list i

Re: [PR] io: add support for role arn and external id s3 props [iceberg-rust]

2024-08-15 Thread via GitHub
Xuanwo merged PR #553: URL: https://github.com/apache/iceberg-rust/pull/553 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.a

Re: [PR] Spark Action to Analyze table [iceberg]

2024-08-15 Thread via GitHub
aokolnychyi commented on code in PR #10288: URL: https://github.com/apache/iceberg/pull/10288#discussion_r1719162511 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/ComputeTableStatsSparkAction.java: ## @@ -0,0 +1,176 @@ +/* + * Licensed to the Apache Software

Re: [PR] Spark Action to Analyze table [iceberg]

2024-08-15 Thread via GitHub
aokolnychyi commented on code in PR #10288: URL: https://github.com/apache/iceberg/pull/10288#discussion_r1719162717 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/ComputeTableStatsSparkAction.java: ## @@ -0,0 +1,176 @@ +/* + * Licensed to the Apache Software

Re: [PR] Spark Action to Analyze table [iceberg]

2024-08-15 Thread via GitHub
aokolnychyi commented on code in PR #10288: URL: https://github.com/apache/iceberg/pull/10288#discussion_r1719162376 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/ComputeTableStatsSparkAction.java: ## @@ -0,0 +1,176 @@ +/* + * Licensed to the Apache Software

Re: [PR] Spark Action to Analyze table [iceberg]

2024-08-15 Thread via GitHub
aokolnychyi commented on code in PR #10288: URL: https://github.com/apache/iceberg/pull/10288#discussion_r1719138348 ## spark/v3.5/spark/src/main/scala/org/apache/spark/sql/stats/ThetaSketchAggregator.scala: ## @@ -0,0 +1,96 @@ +/* + * Licensed to the Apache Software Foundation

Re: [I] Migrate to Spark DS V2 Filter [iceberg]

2024-08-15 Thread via GitHub
github-actions[bot] commented on issue #5273: URL: https://github.com/apache/iceberg/issues/5273#issuecomment-2292492533 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [PR] Doc:Add parquet,orc,avro delete codec and level [iceberg]

2024-08-15 Thread via GitHub
github-actions[bot] commented on PR #5158: URL: https://github.com/apache/iceberg/pull/5158#issuecomment-2292492463 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull

Re: [PR] Flink: add sql partition functions [iceberg]

2024-08-15 Thread via GitHub
github-actions[bot] commented on PR #5120: URL: https://github.com/apache/iceberg/pull/5120#issuecomment-2292492395 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull

Re: [I] Reduce CI Workload by Removing Some Spark Variants and Using Callable Workflows for Github Actions [iceberg]

2024-08-15 Thread via GitHub
github-actions[bot] commented on issue #5153: URL: https://github.com/apache/iceberg/issues/5153#issuecomment-2292492435 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [PR] Data: delete compaction optimization by bloom filter [iceberg]

2024-08-15 Thread via GitHub
github-actions[bot] commented on PR #5100: URL: https://github.com/apache/iceberg/pull/5100#issuecomment-2292492358 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull

Re: [I] Flink: FLIP-143 & FLIP-191 based Iceberg sink [iceberg]

2024-08-15 Thread via GitHub
github-actions[bot] commented on issue #5119: URL: https://github.com/apache/iceberg/issues/5119#issuecomment-2292492377 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [PR] Reduce the number of equity-deletes using bloom filter [iceberg]

2024-08-15 Thread via GitHub
github-actions[bot] commented on PR #5026: URL: https://github.com/apache/iceberg/pull/5026#issuecomment-2292492240 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull

Re: [PR] AWS: support configure region [iceberg]

2024-08-15 Thread via GitHub
github-actions[bot] commented on PR #5046: URL: https://github.com/apache/iceberg/pull/5046#issuecomment-2292492256 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull

Re: [PR] [CORE] - Fix snapshot summary generating partition summary metrics for unpartitioned tables [iceberg]

2024-08-15 Thread via GitHub
github-actions[bot] commented on PR #5009: URL: https://github.com/apache/iceberg/pull/5009#issuecomment-2292492187 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull

Re: [PR] Format: Clarify valid position delete file path [iceberg]

2024-08-15 Thread via GitHub
github-actions[bot] commented on PR #4988: URL: https://github.com/apache/iceberg/pull/4988#issuecomment-2292492046 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull

Re: [PR] [WIP] API/Core: View support [iceberg]

2024-08-15 Thread via GitHub
github-actions[bot] closed pull request #4657: [WIP] API/Core: View support URL: https://github.com/apache/iceberg/pull/4657 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] [WIP] API/Core: View support [iceberg]

2024-08-15 Thread via GitHub
github-actions[bot] commented on PR #4657: URL: https://github.com/apache/iceberg/pull/4657#issuecomment-2292491902 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Support non-optional union types for ORC [iceberg]

2024-08-15 Thread via GitHub
github-actions[bot] closed pull request #4654: Support non-optional union types for ORC URL: https://github.com/apache/iceberg/pull/4654 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] Flink 1.14: Implement LookupTableSource for IcebergTableSource [iceberg]

2024-08-15 Thread via GitHub
github-actions[bot] commented on PR #4649: URL: https://github.com/apache/iceberg/pull/4649#issuecomment-2292491860 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Flink: support watermark and computed columns [iceberg]

2024-08-15 Thread via GitHub
github-actions[bot] commented on PR #4625: URL: https://github.com/apache/iceberg/pull/4625#issuecomment-2292491839 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Flink: support watermark and computed columns [iceberg]

2024-08-15 Thread via GitHub
github-actions[bot] closed pull request #4625: Flink: support watermark and computed columns URL: https://github.com/apache/iceberg/pull/4625 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

Re: [PR] Checkstyle: Add Checkstyle Rule to prevent Map and Set [iceberg]

2024-08-15 Thread via GitHub
github-actions[bot] closed pull request #4620: Checkstyle: Add Checkstyle Rule to prevent Map and Set URL: https://github.com/apache/iceberg/pull/4620 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Core:Optimize Error msg in validateDeletedManifests (fix #3466) [iceberg]

2024-08-15 Thread via GitHub
github-actions[bot] commented on PR #4612: URL: https://github.com/apache/iceberg/pull/4612#issuecomment-2292491799 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Core:Optimize Error msg in validateDeletedManifests (fix #3466) [iceberg]

2024-08-15 Thread via GitHub
github-actions[bot] closed pull request #4612: Core:Optimize Error msg in validateDeletedManifests (fix #3466) URL: https://github.com/apache/iceberg/pull/4612 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] Support insert overwrite table for hive-mr engine. [iceberg]

2024-08-15 Thread via GitHub
github-actions[bot] closed pull request #4605: Support insert overwrite table for hive-mr engine. URL: https://github.com/apache/iceberg/pull/4605 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] Support insert overwrite table for hive-mr engine. [iceberg]

2024-08-15 Thread via GitHub
github-actions[bot] commented on PR #4605: URL: https://github.com/apache/iceberg/pull/4605#issuecomment-2292491777 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] API: Drop column of deleted partitioned field to Unbound partitionSpec [iceberg]

2024-08-15 Thread via GitHub
github-actions[bot] closed pull request #4602: API: Drop column of deleted partitioned field to Unbound partitionSpec URL: https://github.com/apache/iceberg/pull/4602 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

Re: [PR] Orc : Fix inner struct field as partition (#4604) [iceberg]

2024-08-15 Thread via GitHub
github-actions[bot] commented on PR #4599: URL: https://github.com/apache/iceberg/pull/4599#issuecomment-2292491739 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Orc : Fix inner struct field as partition (#4604) [iceberg]

2024-08-15 Thread via GitHub
github-actions[bot] closed pull request #4599: Orc : Fix inner struct field as partition (#4604) URL: https://github.com/apache/iceberg/pull/4599 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] Spark: Spark SQL Extensions for SnapshotRef DDL [iceberg]

2024-08-15 Thread via GitHub
github-actions[bot] commented on PR #5209: URL: https://github.com/apache/iceberg/pull/5209#issuecomment-2292492512 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull

Re: [I] java.lang.IllegalArgumentException: Table identifier not set [iceberg]

2024-08-15 Thread via GitHub
github-actions[bot] commented on issue #5175: URL: https://github.com/apache/iceberg/issues/5175#issuecomment-2292492483 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] org.apache.flink.connectors.hive.FlinkHiveException: Unable to instantiate the hadoop input format [iceberg]

2024-08-15 Thread via GitHub
github-actions[bot] commented on issue #5145: URL: https://github.com/apache/iceberg/issues/5145#issuecomment-2292492416 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [PR] Flink: upsert table join failed [iceberg]

2024-08-15 Thread via GitHub
github-actions[bot] commented on PR #5061: URL: https://github.com/apache/iceberg/pull/5061#issuecomment-2292492307 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull

Re: [PR] Dell: Package dell module into runtime JARs. [iceberg]

2024-08-15 Thread via GitHub
github-actions[bot] commented on PR #5060: URL: https://github.com/apache/iceberg/pull/5060#issuecomment-2292492283 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull

Re: [PR] Docs: use 'SET sql-client.execution.result-mode = tableau' instead of 'SET … [iceberg]

2024-08-15 Thread via GitHub
github-actions[bot] commented on PR #5012: URL: https://github.com/apache/iceberg/pull/5012#issuecomment-2292492217 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull

Re: [PR] Flink: FlinkSQL supports partition transform by computed columns [iceberg]

2024-08-15 Thread via GitHub
github-actions[bot] commented on PR #5001: URL: https://github.com/apache/iceberg/pull/5001#issuecomment-2292492163 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull

Re: [I] rewritedatafile: Cannot commit, found new position delete for replaced data [iceberg]

2024-08-15 Thread via GitHub
github-actions[bot] commented on issue #4996: URL: https://github.com/apache/iceberg/issues/4996#issuecomment-2292492111 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] Proposal: FlinkSQL supports partition transform by computed columns [iceberg]

2024-08-15 Thread via GitHub
github-actions[bot] commented on issue #5000: URL: https://github.com/apache/iceberg/issues/5000#issuecomment-2292492137 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [PR] API: Support computed comlumns [iceberg]

2024-08-15 Thread via GitHub
github-actions[bot] commented on PR #4994: URL: https://github.com/apache/iceberg/pull/4994#issuecomment-2292492081 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull

Re: [PR] Support non-optional union types for ORC [iceberg]

2024-08-15 Thread via GitHub
github-actions[bot] commented on PR #4654: URL: https://github.com/apache/iceberg/pull/4654#issuecomment-2292491882 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Flink 1.14: Implement LookupTableSource for IcebergTableSource [iceberg]

2024-08-15 Thread via GitHub
github-actions[bot] closed pull request #4649: Flink 1.14: Implement LookupTableSource for IcebergTableSource URL: https://github.com/apache/iceberg/pull/4649 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] Checkstyle: Add Checkstyle Rule to prevent Map and Set [iceberg]

2024-08-15 Thread via GitHub
github-actions[bot] commented on PR #4620: URL: https://github.com/apache/iceberg/pull/4620#issuecomment-2292491822 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] API: Drop column of deleted partitioned field to Unbound partitionSpec [iceberg]

2024-08-15 Thread via GitHub
github-actions[bot] commented on PR #4602: URL: https://github.com/apache/iceberg/pull/4602#issuecomment-2292491761 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] [Spark][MR]: Reuse InputFilesDecryptor [iceberg]

2024-08-15 Thread via GitHub
github-actions[bot] commented on PR #4595: URL: https://github.com/apache/iceberg/pull/4595#issuecomment-2292491715 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] [Spark][MR]: Reuse InputFilesDecryptor [iceberg]

2024-08-15 Thread via GitHub
github-actions[bot] closed pull request #4595: [Spark][MR]: Reuse InputFilesDecryptor URL: https://github.com/apache/iceberg/pull/4595 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

[PR] Spec: Minor modifications for v3 [iceberg]

2024-08-15 Thread via GitHub
rdblue opened a new pull request, #10948: URL: https://github.com/apache/iceberg/pull/10948 This adds some minor changes to prepare for v3 spec work: * Adds a section to summarize the high-level v3 changes * Reduces some heading levels to leave room for better organization. The previou

Re: [PR] Add list_views to rest catalog [iceberg-python]

2024-08-15 Thread via GitHub
sungwy commented on code in PR #817: URL: https://github.com/apache/iceberg-python/pull/817#discussion_r1719107560 ## pyiceberg/catalog/__init__.py: ## @@ -537,6 +537,22 @@ def list_namespaces(self, namespace: Union[str, Identifier] = ()) -> List[Identi NoSuchNames

Re: [PR] Introduces the new IcebergSink based on the new V2 Flink Sink Abstraction [iceberg]

2024-08-15 Thread via GitHub
stevenzwu commented on code in PR #10179: URL: https://github.com/apache/iceberg/pull/10179#discussion_r1719099167 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/sink/IcebergSink.java: ## @@ -0,0 +1,752 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

Re: [PR] Introduces the new IcebergSink based on the new V2 Flink Sink Abstraction [iceberg]

2024-08-15 Thread via GitHub
stevenzwu commented on code in PR #10179: URL: https://github.com/apache/iceberg/pull/10179#discussion_r1719099167 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/sink/IcebergSink.java: ## @@ -0,0 +1,752 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

Re: [PR] Introduces the new IcebergSink based on the new V2 Flink Sink Abstraction [iceberg]

2024-08-15 Thread via GitHub
stevenzwu commented on code in PR #10179: URL: https://github.com/apache/iceberg/pull/10179#discussion_r1719099167 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/sink/IcebergSink.java: ## @@ -0,0 +1,752 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

Re: [PR] Introduces the new IcebergSink based on the new V2 Flink Sink Abstraction [iceberg]

2024-08-15 Thread via GitHub
stevenzwu commented on code in PR #10179: URL: https://github.com/apache/iceberg/pull/10179#discussion_r1719097127 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/sink/IcebergSink.java: ## @@ -0,0 +1,811 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

Re: [PR] Introduces the new IcebergSink based on the new V2 Flink Sink Abstraction [iceberg]

2024-08-15 Thread via GitHub
stevenzwu commented on code in PR #10179: URL: https://github.com/apache/iceberg/pull/10179#discussion_r1719095758 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/sink/IcebergSink.java: ## @@ -0,0 +1,811 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

Re: [PR] Introduces the new IcebergSink based on the new V2 Flink Sink Abstraction [iceberg]

2024-08-15 Thread via GitHub
stevenzwu commented on code in PR #10179: URL: https://github.com/apache/iceberg/pull/10179#discussion_r1719092107 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/sink/IcebergSink.java: ## @@ -0,0 +1,752 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

Re: [PR] Introduces the new IcebergSink based on the new V2 Flink Sink Abstraction [iceberg]

2024-08-15 Thread via GitHub
rodmeneses commented on code in PR #10179: URL: https://github.com/apache/iceberg/pull/10179#discussion_r1719090494 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/sink/IcebergSink.java: ## @@ -0,0 +1,811 @@ +/* + * Licensed to the Apache Software Foundation (ASF) un

Re: [PR] Refactor PyArrow DataFiles Projection functions [iceberg-python]

2024-08-15 Thread via GitHub
kevinjqliu commented on code in PR #1043: URL: https://github.com/apache/iceberg-python/pull/1043#discussion_r1719085905 ## pyiceberg/io/pyarrow.py: ## @@ -1398,6 +1588,11 @@ def project_table( return result +@deprecated( +deprecated_in="0.8.0", +removed_in="0.9

Re: [PR] Introduces the new IcebergSink based on the new V2 Flink Sink Abstraction [iceberg]

2024-08-15 Thread via GitHub
rodmeneses commented on code in PR #10179: URL: https://github.com/apache/iceberg/pull/10179#discussion_r1719087855 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/sink/IcebergSink.java: ## @@ -0,0 +1,811 @@ +/* + * Licensed to the Apache Software Foundation (ASF) un

Re: [PR] Add REST Compatibility Kit [iceberg]

2024-08-15 Thread via GitHub
danielcweeks commented on code in PR #10908: URL: https://github.com/apache/iceberg/pull/10908#discussion_r1719084899 ## open-api/src/test/java/org/apache/iceberg/rest/RESTCompatibilityKitCatalogTests.java: ## @@ -0,0 +1,77 @@ +/* + * Licensed to the Apache Software Foundation (

Re: [PR] io: add support for role arn and external id s3 props [iceberg-rust]

2024-08-15 Thread via GitHub
mattheusv commented on PR #553: URL: https://github.com/apache/iceberg-rust/pull/553#issuecomment-2292409776 Upgrading opendal to [v0.49.0](https://github.com/apache/opendal/releases/tag/v0.49.0) we can use role_session_name -- This is an automated message from the Apache Git Service. To

Re: [PR] Introduces the new IcebergSink based on the new V2 Flink Sink Abstraction [iceberg]

2024-08-15 Thread via GitHub
rodmeneses commented on code in PR #10179: URL: https://github.com/apache/iceberg/pull/10179#discussion_r1719082354 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/sink/IcebergSink.java: ## @@ -0,0 +1,811 @@ +/* + * Licensed to the Apache Software Foundation (ASF) un

Re: [PR] Introduces the new IcebergSink based on the new V2 Flink Sink Abstraction [iceberg]

2024-08-15 Thread via GitHub
rodmeneses commented on code in PR #10179: URL: https://github.com/apache/iceberg/pull/10179#discussion_r1719081266 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/sink/IcebergSink.java: ## @@ -0,0 +1,752 @@ +/* + * Licensed to the Apache Software Foundation (ASF) un

[PR] Fix Install Issues with `docutils = 0.21.post1` and exclude 3.12 from supported python dependencies [iceberg-python]

2024-08-15 Thread via GitHub
sungwy opened a new pull request, #1067: URL: https://github.com/apache/iceberg-python/pull/1067 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe

[PR] io: add support for role arn and external id s3 props [iceberg-rust]

2024-08-15 Thread via GitHub
mattheusv opened a new pull request, #553: URL: https://github.com/apache/iceberg-rust/pull/553 Add support for client.assume-role.arn and client.assume-role.external-id s3 config properties. Partial fix for #527 -- This is an automated message from the Apache Git Service. To res

Re: [I] [feat] add missing metadata tables [iceberg-python]

2024-08-15 Thread via GitHub
soumya-ghosh commented on issue #1053: URL: https://github.com/apache/iceberg-python/issues/1053#issuecomment-2292395623 @kevinjqliu added PR #1066 for `data_files` and `delete_files`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[PR] Add metadata tables for `data_files` and `delete_files` [iceberg-python]

2024-08-15 Thread via GitHub
soumya-ghosh opened a new pull request, #1066: URL: https://github.com/apache/iceberg-python/pull/1066 Implements metadata tables for `data_files` and `delete_files` - https://github.com/apache/iceberg-python/issues/1053 Have reused the logic of `files` to derive `data_files` and `del

Re: [PR] Add REST Compatibility Kit [iceberg]

2024-08-15 Thread via GitHub
dimas-b commented on code in PR #10908: URL: https://github.com/apache/iceberg/pull/10908#discussion_r1719071732 ## open-api/src/test/java/org/apache/iceberg/rest/RESTCompatibilityKitCatalogTests.java: ## @@ -0,0 +1,77 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] doc: Add statement for contributors to avoid force push. [iceberg]

2024-08-15 Thread via GitHub
stevenzwu commented on code in PR #10939: URL: https://github.com/apache/iceberg/pull/10939#discussion_r1719050896 ## site/docs/contribute.md: ## @@ -44,6 +44,7 @@ The Iceberg community prefers to receive contributions as [Github pull requests] * It's helpful to include a pref

Re: [PR] Spark partial limit push down [iceberg]

2024-08-15 Thread via GitHub
RussellSpitzer commented on code in PR #10943: URL: https://github.com/apache/iceberg/pull/10943#discussion_r1719059739 ## parquet/src/main/java/org/apache/iceberg/parquet/ParquetReader.java: ## @@ -117,7 +122,12 @@ private static class FileIterator implements CloseableIterator

Re: [PR] Spark partial limit push down [iceberg]

2024-08-15 Thread via GitHub
RussellSpitzer commented on code in PR #10943: URL: https://github.com/apache/iceberg/pull/10943#discussion_r1719058057 ## arrow/src/main/java/org/apache/iceberg/arrow/vectorized/parquet/VectorizedColumnIterator.java: ## @@ -69,12 +69,14 @@ public boolean producesDictionaryEncod

Re: [PR] Spark partial limit push down [iceberg]

2024-08-15 Thread via GitHub
RussellSpitzer commented on code in PR #10943: URL: https://github.com/apache/iceberg/pull/10943#discussion_r171905 ## arrow/src/main/java/org/apache/iceberg/arrow/vectorized/parquet/VectorizedColumnIterator.java: ## @@ -69,12 +69,14 @@ public boolean producesDictionaryEncod

  1   2   3   >