Re: [PR] Table Scan Delete File Handling: Positional and Equality Delete Support [iceberg-rust]

2024-12-10 Thread via GitHub
sdd commented on code in PR #652: URL: https://github.com/apache/iceberg-rust/pull/652#discussion_r1879516829 ## crates/iceberg/src/scan.rs: ## @@ -951,6 +1077,82 @@ impl FileScanTask { } } +type DeleteFileManagerResult = Result>>>; + +/// Manages async retrieval of all

[PR] doc: add RisingWave to users [iceberg-rust]

2024-12-10 Thread via GitHub
xxchan opened a new pull request, #775: URL: https://github.com/apache/iceberg-rust/pull/775 Signed-off-by: xxchan -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsub

[I] Comment and assertion mismatch in PartitionedWritesTestBase [iceberg]

2024-12-10 Thread via GitHub
wzx140 opened a new issue, #11747: URL: https://github.com/apache/iceberg/issues/11747 ### Apache Iceberg version main (development) ### Query engine Spark ### Please describe the bug 🐞 There is a mismatch between the test comment and the assertion in the f

[I] Discussion: make DataFile Serializable && Deserializable [iceberg-rust]

2024-12-10 Thread via GitHub
ZENOTME opened a new issue, #774: URL: https://github.com/apache/iceberg-rust/issues/774 ## Context Make Datafile Serializable && Deserializable is useful, e.g. In distributed compute engine, it will create multiple writers in multiple machines and write the data in parallel and get

Re: [PR] refactor: avoid async_trait for FileWrite and provide extra dyn methods [iceberg-rust]

2024-12-10 Thread via GitHub
wenym1 commented on code in PR #762: URL: https://github.com/apache/iceberg-rust/pull/762#discussion_r1879426201 ## crates/iceberg/src/writer/file_writer/parquet_writer.rs: ## @@ -213,7 +213,7 @@ impl SchemaVisitor for IndexByParquetPathName { pub struct ParquetWriter { sc

Re: [I] Implement nan_value_counts && distinct_counts metrics in parquet writer [iceberg-rust]

2024-12-10 Thread via GitHub
feniljain commented on issue #417: URL: https://github.com/apache/iceberg-rust/issues/417#issuecomment-2533731058 >but according to the spec it's supposed to be an estimated value using sketch That sounds interesting, thanks for the link up to spec! >I think we could start with

Re: [I] Implement nan_value_counts && distinct_counts metrics in parquet writer [iceberg-rust]

2024-12-10 Thread via GitHub
liurenjie1024 commented on issue #417: URL: https://github.com/apache/iceberg-rust/issues/417#issuecomment-2533708338 Hi, @feniljain I also didn't find how distinct counts are implemented in java, but according to the spec it's supposed to be an estimated value using sketch. I think we coul

Re: [PR] Flink: Backport #11662 Fix range distribution npe when value is null to Flink 1.18 and 1.19 [iceberg]

2024-12-10 Thread via GitHub
pvary commented on PR #11745: URL: https://github.com/apache/iceberg/pull/11745#issuecomment-2533701394 @Guosmilesmile: Was this a clean backport, or you had to modify things to apply the changes? -- This is an automated message from the Apache Git Service. To respond to the message, plea

Re: [PR] Flink: Fix range distribution npe when value is null [iceberg]

2024-12-10 Thread via GitHub
Guosmilesmile commented on PR #11662: URL: https://github.com/apache/iceberg/pull/11662#issuecomment-2533696010 > Thanks @Guosmilesmile for the PR, and @mxm and @stevenzwu for the review! > > @Guosmilesmile please prepare the backport commits to the other Flink versions. > > Th

Re: [PR] Flink: Backport #11662 Fix range distribution npe when value is null to Flink 1.18 and 1.19 [iceberg]

2024-12-10 Thread via GitHub
Guosmilesmile commented on PR #11745: URL: https://github.com/apache/iceberg/pull/11745#issuecomment-2533693064 @pvary here's the PR to backport for Fix range distribution npe when value is null to Flink v1.18 and v1.19. Could you please take some time to help review it? -- This

Re: [I] Use apache/spark image in the quickstart [iceberg]

2024-12-10 Thread via GitHub
ajantha-bhat commented on issue #11746: URL: https://github.com/apache/iceberg/issues/11746#issuecomment-2533652976 Will keep it open if anyone wants to take it up. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[I] Use apache/spark image in the quickstart [iceberg]

2024-12-10 Thread via GitHub
ajantha-bhat opened a new issue, #11746: URL: https://github.com/apache/iceberg/issues/11746 ### Feature Request / Improvement more information https://lists.apache.org/thread/4kknk8mvnffbmhdt63z8t4ps0mt1jbf4 ### Query engine None ### Willingness to contribute

Re: [PR] Core: Fix failure when reading files table with branch [iceberg]

2024-12-10 Thread via GitHub
dramaticlly commented on PR #11719: URL: https://github.com/apache/iceberg/pull/11719#issuecomment-2533650691 > This makes sense to me. How about other metadata tables? @dramaticlly do you also want take a look? Happy to help with other metadata tables experienced similar problem if i

Re: [PR] Bump Spark 3.5.4 RC1 [iceberg]

2024-12-10 Thread via GitHub
pan3793 commented on PR #11731: URL: https://github.com/apache/iceberg/pull/11731#issuecomment-2533597688 FYI @LuciferYang @viirya @dongjoon-hyun @wForget CI is green now, and I think the current Spark `branch-3.5` is in good shape for Iceberg. -- This is an automated message from

Re: [PR] Spark 3.5: Add ignore-invalid-options to RewriteDataFilesSparkAction and RewritePositionDeleteFilesSparkAction [iceberg]

2024-12-10 Thread via GitHub
manuzhang commented on code in PR #11737: URL: https://github.com/apache/iceberg/pull/11737#discussion_r1879291362 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/actions/TestRewriteDataFilesAction.java: ## @@ -1214,6 +1214,15 @@ public void testInvalidOptions() {

Re: [I] Partition Spec Evolution API [iceberg-rust]

2024-12-10 Thread via GitHub
liurenjie1024 commented on issue #732: URL: https://github.com/apache/iceberg-rust/issues/732#issuecomment-2533556499 Thanks for @c-thiel for raising this. I agree that we should focus on v2 first, and add v1 support later. -- This is an automated message from the Apache Git Service. To r

Re: [PR] Bump Spark 3.5.4 RC1 [iceberg]

2024-12-10 Thread via GitHub
pan3793 commented on code in PR #11731: URL: https://github.com/apache/iceberg/pull/11731#discussion_r1879272199 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/data/vectorized/IcebergArrowColumnVector.java: ## @@ -59,6 +59,11 @@ public void close() { accessor.cl

Re: [PR] Bump Spark 3.5.4 RC1 [iceberg]

2024-12-10 Thread via GitHub
pan3793 commented on code in PR #11731: URL: https://github.com/apache/iceberg/pull/11731#discussion_r1879272199 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/data/vectorized/IcebergArrowColumnVector.java: ## @@ -59,6 +59,11 @@ public void close() { accessor.cl

Re: [PR] Spark 3.5: Add ignore-invalid-options to RewriteDataFilesSparkAction and RewritePositionDeleteFilesSparkAction [iceberg]

2024-12-10 Thread via GitHub
huaxingao commented on code in PR #11737: URL: https://github.com/apache/iceberg/pull/11737#discussion_r1879274285 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/actions/TestRewriteDataFilesAction.java: ## @@ -1214,6 +1214,15 @@ public void testInvalidOptions() {

Re: [PR] Bump Spark 3.5.4 RC1 [iceberg]

2024-12-10 Thread via GitHub
singhpk234 commented on code in PR #11731: URL: https://github.com/apache/iceberg/pull/11731#discussion_r1879265358 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/data/vectorized/IcebergArrowColumnVector.java: ## @@ -59,6 +59,11 @@ public void close() { accessor

Re: [I] Add s3tables catalog support [iceberg-rust]

2024-12-10 Thread via GitHub
liurenjie1024 commented on issue #754: URL: https://github.com/apache/iceberg-rust/issues/754#issuecomment-2533507061 > please assign this to me if it's not a feature that needs to be released soon. I can work on it part-time, but I cannot guarantee the completion time Done, thanks!

Re: [PR] Spark3.4,3.5: Fix the BUG of iceberg views when resolved "group/order… [iceberg]

2024-12-10 Thread via GitHub
Ppei-Wang commented on code in PR #11729: URL: https://github.com/apache/iceberg/pull/11729#discussion_r1879246100 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestViews.java: ## @@ -562,6 +562,67 @@ public void readFromViewWithCTE() throws N

Re: [PR] Bump Spark 3.5.4 RC1 [iceberg]

2024-12-10 Thread via GitHub
pan3793 commented on code in PR #11731: URL: https://github.com/apache/iceberg/pull/11731#discussion_r1879240394 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/data/vectorized/IcebergArrowColumnVector.java: ## @@ -59,6 +59,11 @@ public void close() { accessor.cl

Re: [PR] Add clang format [iceberg-cpp]

2024-12-10 Thread via GitHub
zhjwpku commented on code in PR #4: URL: https://github.com/apache/iceberg-cpp/pull/4#discussion_r1879216891 ## .clang-format: ## @@ -0,0 +1,22 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# dis

Re: [PR] AWS: Enable Adaptive Retries for AWS KMS client [iceberg]

2024-12-10 Thread via GitHub
hsiang-c commented on PR #11420: URL: https://github.com/apache/iceberg/pull/11420#issuecomment-2533470077 Thank you @amogh-jahagirdar -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

Re: [PR] Bump Spark 3.5.4 RC1 [iceberg]

2024-12-10 Thread via GitHub
pan3793 commented on PR #11731: URL: https://github.com/apache/iceberg/pull/11731#issuecomment-2533468089 @viirya @LuciferYang thank you for the guidance, let me try -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] Add clang format [iceberg-cpp]

2024-12-10 Thread via GitHub
zhjwpku commented on code in PR #4: URL: https://github.com/apache/iceberg-cpp/pull/4#discussion_r1879216891 ## .clang-format: ## @@ -0,0 +1,22 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# dis

Re: [PR] AWS: Enable RetryMode for AWS KMS client [iceberg]

2024-12-10 Thread via GitHub
amogh-jahagirdar merged PR #11420: URL: https://github.com/apache/iceberg/pull/11420 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@

Re: [PR] Add clang format [iceberg-cpp]

2024-12-10 Thread via GitHub
zhjwpku commented on code in PR #4: URL: https://github.com/apache/iceberg-cpp/pull/4#discussion_r1879216891 ## .clang-format: ## @@ -0,0 +1,22 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# dis

Re: [PR] docker: The `archive` seems unstable [iceberg-rust]

2024-12-10 Thread via GitHub
liurenjie1024 commented on PR #773: URL: https://github.com/apache/iceberg-rust/pull/773#issuecomment-2533460061 Thanks @Fokko for fixing this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] docker: The `archive` seems unstable [iceberg-rust]

2024-12-10 Thread via GitHub
liurenjie1024 merged PR #773: URL: https://github.com/apache/iceberg-rust/pull/773 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [PR] Core: Fix failure when reading files table with branch [iceberg]

2024-12-10 Thread via GitHub
szehon-ho commented on PR #11719: URL: https://github.com/apache/iceberg/pull/11719#issuecomment-2533424881 This makes sense to me. How about other metadata tables? @dramaticlly do you also want take a look? -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] Core: Fix failure when reading files table with branch [iceberg]

2024-12-10 Thread via GitHub
szehon-ho commented on PR #11719: URL: https://github.com/apache/iceberg/pull/11719#issuecomment-2533425414 Is it ready for review? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

Re: [PR] Core, Flink, Spark, KafkaConnect: Remove remaining usage of deprecated path API [iceberg]

2024-12-10 Thread via GitHub
amogh-jahagirdar commented on code in PR #11744: URL: https://github.com/apache/iceberg/pull/11744#discussion_r1879171773 ## spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/source/BaseReader.java: ## @@ -148,7 +148,7 @@ public boolean next() throws IOException { i

Re: [PR] Core, Flink, Spark, KafkaConnect: Remove remaining usage of deprecated path API [iceberg]

2024-12-10 Thread via GitHub
amogh-jahagirdar commented on code in PR #11744: URL: https://github.com/apache/iceberg/pull/11744#discussion_r1879171908 ## spark/v3.3/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestExpireSnapshotsProcedure.java: ## @@ -300,7 +300,7 @@ public void testEx

Re: [PR] AWS: Enable RetryMode for AWS KMS client [iceberg]

2024-12-10 Thread via GitHub
amogh-jahagirdar commented on PR #11420: URL: https://github.com/apache/iceberg/pull/11420#issuecomment-2533397879 I'll go ahead and merge -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

Re: [I] Rest Catalog: spark catalog api fails to work with rest based catalog [iceberg]

2024-12-10 Thread via GitHub
dramaticlly commented on issue #11741: URL: https://github.com/apache/iceberg/issues/11741#issuecomment-2533365254 I think use this we can reproduce the problem in a unit test from `org.apache.iceberg.spark.sql.TestCreateTable` as it support all of hive/spark/hadoop/rest catalog, I think re

Re: [PR] Spark3.4,3.5: Fix the BUG of iceberg views when resolved "group/order… [iceberg]

2024-12-10 Thread via GitHub
ebyhr commented on code in PR #11729: URL: https://github.com/apache/iceberg/pull/11729#discussion_r1879148802 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestViews.java: ## @@ -562,6 +562,67 @@ public void readFromViewWithCTE() throws NoSuc

Re: [PR] Core, Flink, Spark, KafkaConnect: Remove remaining usage of deprecated path API [iceberg]

2024-12-10 Thread via GitHub
ebyhr commented on code in PR #11744: URL: https://github.com/apache/iceberg/pull/11744#discussion_r1879109181 ## flink/v1.19/flink/src/test/java/org/apache/iceberg/flink/TestHelpers.java: ## @@ -602,7 +602,7 @@ public static void assertEquals(ContentFile expected, ContentFile

Re: [PR] core: support support move a column with same name after rename column [iceberg]

2024-12-10 Thread via GitHub
github-actions[bot] commented on PR #10862: URL: https://github.com/apache/iceberg/pull/10862#issuecomment-2533325429 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] [Flink] the precondition judgment is incorrect in the FlinkSink class. [iceberg]

2024-12-10 Thread via GitHub
github-actions[bot] commented on PR #10669: URL: https://github.com/apache/iceberg/pull/10669#issuecomment-2533325348 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [I] Support for loading different hive-metastore versions at Runtime [iceberg]

2024-12-10 Thread via GitHub
github-actions[bot] closed issue #10401: Support for loading different hive-metastore versions at Runtime URL: https://github.com/apache/iceberg/issues/10401 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

[PR] Fix `Table.scan` to enable case sensitivity argument [iceberg-python]

2024-12-10 Thread via GitHub
jiakai-li opened a new pull request, #1423: URL: https://github.com/apache/iceberg-python/pull/1423 **This pull request fixes below issue:** - #1421 The change modified the `DataScan._build_partition_projection` method to pass `case_sensitive` argument when calling `inclusive_proj

Re: [I] Support for loading different hive-metastore versions at Runtime [iceberg]

2024-12-10 Thread via GitHub
github-actions[bot] commented on issue #10401: URL: https://github.com/apache/iceberg/issues/10401#issuecomment-2533325043 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache

Re: [I] Can't add column with the same name as a deleted and previously partitioned key column [iceberg]

2024-12-10 Thread via GitHub
github-actions[bot] commented on issue #10487: URL: https://github.com/apache/iceberg/issues/10487#issuecomment-2533325230 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [PR] Spec: Support geo type [iceberg]

2024-12-10 Thread via GitHub
szehon-ho commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1879110520 ## format/spec.md: ## @@ -1102,6 +1105,7 @@ Hash results are not dependent on decimal scale, which is part of the type, not 4. UUIDs are encoded using big endian.

Re: [PR] Spec: Support geo type [iceberg]

2024-12-10 Thread via GitHub
jiayuasu commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r187907 ## format/spec.md: ## @@ -1633,3 +1656,15 @@ might indicate different snapshot IDs for a specific timestamp. The discrepancie When processing point in time queri

Re: [PR] Spec: Support geo type [iceberg]

2024-12-10 Thread via GitHub
jiayuasu commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1879080772 ## format/spec.md: ## @@ -1633,3 +1656,15 @@ might indicate different snapshot IDs for a specific timestamp. The discrepancie When processing point in time queri

Re: [PR] Spec: Support geo type [iceberg]

2024-12-10 Thread via GitHub
jiayuasu commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1879088058 ## format/spec.md: ## @@ -1633,3 +1656,15 @@ might indicate different snapshot IDs for a specific timestamp. The discrepancie When processing point in time queri

Re: [PR] Hive: Optimize tableExists API in hive catalog [iceberg]

2024-12-10 Thread via GitHub
dramaticlly commented on code in PR #11597: URL: https://github.com/apache/iceberg/pull/11597#discussion_r1879086639 ## hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java: ## @@ -412,6 +412,43 @@ private void validateTableIsIcebergTableOrView( } } +

Re: [PR] Hive: Optimize tableExists API in hive catalog [iceberg]

2024-12-10 Thread via GitHub
danielcweeks commented on PR #11597: URL: https://github.com/apache/iceberg/pull/11597#issuecomment-2533223559 @dramaticlly minor comment, but other than that LGTM. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

Re: [PR] Hive: Optimize tableExists API in hive catalog [iceberg]

2024-12-10 Thread via GitHub
danielcweeks commented on code in PR #11597: URL: https://github.com/apache/iceberg/pull/11597#discussion_r1879074770 ## hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java: ## @@ -412,6 +412,43 @@ private void validateTableIsIcebergTableOrView( } } +

Re: [PR] Add CMake format [iceberg-cpp]

2024-12-10 Thread via GitHub
Fokko commented on code in PR #5: URL: https://github.com/apache/iceberg-cpp/pull/5#discussion_r1879041334 ## cmake-format.py: ## @@ -0,0 +1,74 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# dis

Re: [PR] Add Strict projection [iceberg-python]

2024-12-10 Thread via GitHub
Fokko commented on code in PR #539: URL: https://github.com/apache/iceberg-python/pull/539#discussion_r1879028369 ## pyiceberg/transforms.py: ## @@ -766,6 +858,47 @@ def _truncate_number( return None +def _truncate_number_strict( +name: str, pred: BoundLiteralPr

Re: [PR] Update StrictProjection tests [iceberg-python]

2024-12-10 Thread via GitHub
Fokko commented on code in PR #1422: URL: https://github.com/apache/iceberg-python/pull/1422#discussion_r1879026523 ## pyiceberg/transforms.py: ## @@ -985,7 +985,7 @@ def _truncate_number_strict( elif isinstance(pred, BoundGreaterThanOrEqual): return GreaterThan(Re

Re: [PR] Spec: Support geo type [iceberg]

2024-12-10 Thread via GitHub
szehon-ho commented on PR #10981: URL: https://github.com/apache/iceberg/pull/10981#issuecomment-2533103710 Update, there was a sync with @jiayuasu @flyrain @dmitrykoval @paleolimbot @rdblue and Menelaos, it was decided the following (meeting notes): My summary is that we decided to h

Re: [PR] Add Strict projection [iceberg-python]

2024-12-10 Thread via GitHub
Fokko commented on code in PR #539: URL: https://github.com/apache/iceberg-python/pull/539#discussion_r1879010689 ## pyiceberg/transforms.py: ## @@ -766,6 +858,47 @@ def _truncate_number( return None +def _truncate_number_strict( +name: str, pred: BoundLiteralPr

Re: [PR] Hive: Optimize tableExists API in hive catalog [iceberg]

2024-12-10 Thread via GitHub
dramaticlly commented on PR #11597: URL: https://github.com/apache/iceberg/pull/11597#issuecomment-2533061166 @rdblue @danielcweeks can you help take anther look to see if we can move forward on this PR? -- This is an automated message from the Apache Git Service. To respond to the messa

Re: [PR] Flink: Fix range distribution npe when value is null [iceberg]

2024-12-10 Thread via GitHub
pvary commented on PR #11662: URL: https://github.com/apache/iceberg/pull/11662#issuecomment-2532940822 Thanks @Guosmilesmile for the PR, and @mxm and @stevenzwu for the review! @Guosmilesmile please prepare the backport commits to the other Flink versions. Thanks, Peter --

Re: [PR] Flink: Fix range distribution npe when value is null [iceberg]

2024-12-10 Thread via GitHub
pvary merged PR #11662: URL: https://github.com/apache/iceberg/pull/11662 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

[PR] Core, Flink, Spark, KafkaConnect: Remove remaining usage of deprecated path API [iceberg]

2024-12-10 Thread via GitHub
amogh-jahagirdar opened a new pull request, #11744: URL: https://github.com/apache/iceberg/pull/11744 Same as https://github.com/apache/iceberg/pull/11563, removing the remaining usages of the path API -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] API, Core: Add scan planning apis to REST Catalog [iceberg]

2024-12-10 Thread via GitHub
rahil-c commented on PR #11180: URL: https://github.com/apache/iceberg/pull/11180#issuecomment-2532789316 @rdblue @danielcweeks Was wondering if you guys can take a look whenever you get a chance? -- This is an automated message from the Apache Git Service. To respond to the message, plea

Re: [PR] Add scan planning api request and response models, parsers [iceberg]

2024-12-10 Thread via GitHub
rahil-c commented on PR #11369: URL: https://github.com/apache/iceberg/pull/11369#issuecomment-2532787657 @amogh-jahagirdar Was wondering if there is another issues, or if you think we can land this? -- This is an automated message from the Apache Git Service. To respond to the message, p

Re: [PR] REST: AuthManager API [iceberg]

2024-12-10 Thread via GitHub
nastra commented on code in PR #10753: URL: https://github.com/apache/iceberg/pull/10753#discussion_r1878738170 ## core/src/main/java/org/apache/iceberg/rest/auth/AuthSession.java: ## @@ -0,0 +1,53 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more

Re: [PR] Core, Flink, Spark: Drop deprecated APIs scheduled for removal in 1.8.0 [iceberg]

2024-12-10 Thread via GitHub
nastra merged PR #11721: URL: https://github.com/apache/iceberg/pull/11721 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] [Views] Update view spec with table identifier requirements [iceberg]

2024-12-10 Thread via GitHub
JanKaul commented on code in PR #11365: URL: https://github.com/apache/iceberg/pull/11365#discussion_r1878651163 ## format/view-spec.md: ## @@ -97,7 +97,10 @@ Summary is a string to string map of metadata about a view version. Common metad View definitions can be represented

Re: [PR] WIP proposal for fixing issue #11742 [iceberg]

2024-12-10 Thread via GitHub
jkolash commented on PR #11743: URL: https://github.com/apache/iceberg/pull/11743#issuecomment-2532509896 Closing this as I now see I can just set the spark.sql.catalog.default_iceberg property will add more detail to the issue -- This is an automated message from the Apache Git Service.

Re: [I] org.apache.iceberg.spark.source.IcebergSource.setupDefaultSparkCatalogs shouldn't default to hive catalog [iceberg]

2024-12-10 Thread via GitHub
jkolash closed issue #11742: org.apache.iceberg.spark.source.IcebergSource.setupDefaultSparkCatalogs shouldn't default to hive catalog URL: https://github.com/apache/iceberg/issues/11742 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [I] org.apache.iceberg.spark.source.IcebergSource.setupDefaultSparkCatalogs shouldn't default to hive catalog [iceberg]

2024-12-10 Thread via GitHub
jkolash commented on issue #11742: URL: https://github.com/apache/iceberg/issues/11742#issuecomment-2532512695 ok setting spark.sql.catalog.default_iceberg=org.apache.iceberg.spark.SparkCatalog will allow you to override the default values. -- This is an automated message f

Re: [PR] Bump Spark 3.5.4 RC1 [iceberg]

2024-12-10 Thread via GitHub
LuciferYang commented on PR #11731: URL: https://github.com/apache/iceberg/pull/11731#issuecomment-2532512457 > Please check for some discussion there [apache/spark#49131 (comment)](https://github.com/apache/spark/pull/49131#issuecomment-2532341673) Yes, I tested it locally and found

Re: [PR] WIP proposal for fixing issue #11742 [iceberg]

2024-12-10 Thread via GitHub
jkolash closed pull request #11743: WIP proposal for fixing issue #11742 URL: https://github.com/apache/iceberg/pull/11743 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

Re: [PR] [Views] Update view spec with table identifier requirements [iceberg]

2024-12-10 Thread via GitHub
wmoustafa commented on code in PR #11365: URL: https://github.com/apache/iceberg/pull/11365#discussion_r1878540920 ## format/view-spec.md: ## @@ -97,7 +97,10 @@ Summary is a string to string map of metadata about a view version. Common metad View definitions can be represent

Re: [PR] Add basic CI to build iceberg and example [iceberg-cpp]

2024-12-10 Thread via GitHub
raulcd commented on code in PR #7: URL: https://github.com/apache/iceberg-cpp/pull/7#discussion_r1878524143 ## .github/workflows/test.yml: ## @@ -0,0 +1,95 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE

Re: [PR] Bump Spark 3.5.4 RC1 [iceberg]

2024-12-10 Thread via GitHub
viirya commented on PR #11731: URL: https://github.com/apache/iceberg/pull/11731#issuecomment-2532346946 Please check for some discussion there https://github.com/apache/spark/pull/49131#issuecomment-2532341673 -- This is an automated message from the Apache Git Service. To respond to the

Re: [I] DataScan doesn't respect casesensitive argument [iceberg-python]

2024-12-10 Thread via GitHub
jiakai-li commented on issue #1421: URL: https://github.com/apache/iceberg-python/issues/1421#issuecomment-2532341957 Can I work on this issue? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[I] DataScan doesn't respect casesensitive argument [iceberg-python]

2024-12-10 Thread via GitHub
jiakai-li opened a new issue, #1421: URL: https://github.com/apache/iceberg-python/issues/1421 ### Apache Iceberg version None ### Please describe the bug 🐞 When I was learning the code, I noticed currently the `DataScan` class doesn't respect `case_sensitive` when it pl

Re: [PR] AWS: adding s3 tables catalog [iceberg]

2024-12-10 Thread via GitHub
stubz151 closed pull request #11739: AWS: adding s3 tables catalog URL: https://github.com/apache/iceberg/pull/11739 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscr

Re: [I] org.apache.iceberg.spark.source.IcebergSource.setupDefaultSparkCatalogs shouldn't default to hive catalog [iceberg]

2024-12-10 Thread via GitHub
jkolash commented on issue #11742: URL: https://github.com/apache/iceberg/issues/11742#issuecomment-2532329867 This is probably the simplest fix https://github.com/apache/iceberg/pull/11743 allow users to set these properties directly a bigger fix may be to use another catal

Re: [PR] [Views] Update view spec with table identifier requirements [iceberg]

2024-12-10 Thread via GitHub
bennychow commented on code in PR #11365: URL: https://github.com/apache/iceberg/pull/11365#discussion_r1878496657 ## format/view-spec.md: ## @@ -97,7 +97,10 @@ Summary is a string to string map of metadata about a view version. Common metad View definitions can be represent

Re: [PR] Bump Spark 3.5.4 RC1 [iceberg]

2024-12-10 Thread via GitHub
viirya commented on PR #11731: URL: https://github.com/apache/iceberg/pull/11731#issuecomment-2532312814 Does Iceberg implement ColumnVector interface which is similar to writable column vector or constant column vector? -- This is an automated message from the Apache Git Service. To resp

[I] org.apache.iceberg.spark.source.IcebergSource.setupDefaultSparkCatalogs shouldn't default to hive catalog [iceberg]

2024-12-10 Thread via GitHub
jkolash opened a new issue, #11742: URL: https://github.com/apache/iceberg/issues/11742 ### Feature Request / Improvement https://github.com/apache/iceberg/blob/main/spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/source/IcebergSource.java#L239 It should default to anot

Re: [PR] Core: Add Variant implementation to read serialized objects [iceberg]

2024-12-10 Thread via GitHub
aihuaxu commented on code in PR #11415: URL: https://github.com/apache/iceberg/pull/11415#discussion_r1878469751 ## core/src/main/java/org/apache/iceberg/variants/PrimitiveWrapper.java: ## @@ -0,0 +1,206 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * o

Re: [PR] Core: Add Variant implementation to read serialized objects [iceberg]

2024-12-10 Thread via GitHub
aihuaxu commented on code in PR #11415: URL: https://github.com/apache/iceberg/pull/11415#discussion_r1878440484 ## core/src/main/java/org/apache/iceberg/variants/ShreddedObject.java: ## @@ -0,0 +1,211 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or

[I] Rest Catalog: spark catalog api fails working rest based catalog is used [iceberg]

2024-12-10 Thread via GitHub
sunny1154 opened a new issue, #11741: URL: https://github.com/apache/iceberg/issues/11741 ### Apache Iceberg version 1.5.0 ### Query engine Spark ### Please describe the bug 🐞 Hi, I am observing issues when working with rest based catalog. my

Re: [PR] Spark : Derive Stats From Manifest on the Fly [iceberg]

2024-12-10 Thread via GitHub
saitharun15 commented on PR #11615: URL: https://github.com/apache/iceberg/pull/11615#issuecomment-2532200924 Hi @RussellSpitzer ,@huaxingao can u please review the pr once, Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

Re: [PR] Add basic CI to build iceberg and example [iceberg-cpp]

2024-12-10 Thread via GitHub
pitrou commented on code in PR #7: URL: https://github.com/apache/iceberg-cpp/pull/7#discussion_r1878396289 ## .github/workflows/test.yml: ## @@ -0,0 +1,95 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE

Re: [PR] Add basic CI to build iceberg and example [iceberg-cpp]

2024-12-10 Thread via GitHub
raulcd commented on code in PR #7: URL: https://github.com/apache/iceberg-cpp/pull/7#discussion_r1878388213 ## .github/workflows/test.yml: ## @@ -0,0 +1,95 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE

Re: [PR] Add basic CI to build iceberg and example [iceberg-cpp]

2024-12-10 Thread via GitHub
gaborkaszab commented on code in PR #7: URL: https://github.com/apache/iceberg-cpp/pull/7#discussion_r1878384745 ## .github/workflows/test.yml: ## @@ -0,0 +1,95 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the N

Re: [PR] Bump Spark 3.5.4 RC1 [iceberg]

2024-12-10 Thread via GitHub
pan3793 commented on PR #11731: URL: https://github.com/apache/iceberg/pull/11731#issuecomment-2532122346 update: the failure is related to [SPARK-50235](https://github.com/apache/spark/pull/48767), and the test passed on Spark 3.5.4 RC0 with reverting that patch -- This is an automated

Re: [PR] Core: Fix loading a table in CachingCatalog with metadata table name [iceberg]

2024-12-10 Thread via GitHub
gaborkaszab commented on code in PR #11738: URL: https://github.com/apache/iceberg/pull/11738#discussion_r1878337162 ## core/src/main/java/org/apache/iceberg/CachingCatalog.java: ## @@ -144,14 +144,16 @@ public Table loadTable(TableIdentifier ident) { return cached;

Re: [I] Bug in `PartialEq` for `Struct` [iceberg-rust]

2024-12-10 Thread via GitHub
sungwy commented on issue #706: URL: https://github.com/apache/iceberg-rust/issues/706#issuecomment-2531897325 Hi @Sl1mb0 - Thank you for raising this issue! I took a look at this issue in order to resolve all issues blocking the 0.4.0 release. I put up the PR #772 to replicate your i

Re: [PR] Core: Fix loading a table in CachingCatalog with metadata table name [iceberg]

2024-12-10 Thread via GitHub
manuzhang commented on code in PR #11738: URL: https://github.com/apache/iceberg/pull/11738#discussion_r1878295947 ## core/src/main/java/org/apache/iceberg/CachingCatalog.java: ## @@ -144,14 +144,16 @@ public Table loadTable(TableIdentifier ident) { return cached; }

Re: [I] Make field-id of name-mapping optional [iceberg-python]

2024-12-10 Thread via GitHub
Fokko commented on issue #1420: URL: https://github.com/apache/iceberg-python/issues/1420#issuecomment-2531944719 @barronw certainly, thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] GCP: Implement SupportsRecoveryOperations for GCSFileIO [iceberg]

2024-12-10 Thread via GitHub
amogh-jahagirdar commented on code in PR #11565: URL: https://github.com/apache/iceberg/pull/11565#discussion_r1878222555 ## gcp/src/main/java/org/apache/iceberg/gcp/gcs/GCSFileIO.java: ## @@ -242,4 +248,106 @@ private void internalDeleteFiles(Stream blobIdsToDelete) { Str

Re: [PR] GCP: Implement SupportsRecoveryOperations for GCSFileIO [iceberg]

2024-12-10 Thread via GitHub
amogh-jahagirdar commented on code in PR #11565: URL: https://github.com/apache/iceberg/pull/11565#discussion_r1878222555 ## gcp/src/main/java/org/apache/iceberg/gcp/gcs/GCSFileIO.java: ## @@ -242,4 +248,106 @@ private void internalDeleteFiles(Stream blobIdsToDelete) { Str

Re: [PR] GCP: Implement SupportsRecoveryOperations for GCSFileIO [iceberg]

2024-12-10 Thread via GitHub
amogh-jahagirdar commented on code in PR #11565: URL: https://github.com/apache/iceberg/pull/11565#discussion_r1878209991 ## gcp/src/main/java/org/apache/iceberg/gcp/gcs/GCSFileIO.java: ## @@ -242,4 +248,106 @@ private void internalDeleteFiles(Stream blobIdsToDelete) { Str

Re: [PR] REST: AuthManager API [iceberg]

2024-12-10 Thread via GitHub
adutra commented on code in PR #10753: URL: https://github.com/apache/iceberg/pull/10753#discussion_r1878163433 ## core/src/main/java/org/apache/iceberg/rest/auth/AuthSession.java: ## @@ -0,0 +1,53 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more

Re: [PR] Add basic CI to build iceberg and example [iceberg-cpp]

2024-12-10 Thread via GitHub
raulcd commented on code in PR #7: URL: https://github.com/apache/iceberg-cpp/pull/7#discussion_r1877940138 ## .github/workflows/test.yml: ## @@ -0,0 +1,95 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE

Re: [I] Compatibility Issue with pydantic and annotated-types in pyiceberg 0.8.1 [iceberg-python]

2024-12-10 Thread via GitHub
Fokko commented on issue #1418: URL: https://github.com/apache/iceberg-python/issues/1418#issuecomment-2531608924 @djouallah do you know which version of Pydantic you're using? You can easily check it using: ``` Python 3.10.14 (main, Mar 19 2024, 21:46:16) [Clang 15.0.0 (clang-15

Re: [PR] Bump mkdocs-material from 9.5.47 to 9.5.48 [iceberg-python]

2024-12-10 Thread via GitHub
Fokko merged PR #1419: URL: https://github.com/apache/iceberg-python/pull/1419 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

  1   2   >