Re: [PR] Remove all usages of deprecated AssertHelpers [iceberg]

2024-06-16 Thread via GitHub
nastra commented on code in PR #10500: URL: https://github.com/apache/iceberg/pull/10500#discussion_r1642276958 ## spark/v3.3/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestAddFilesProcedure.java: ## @@ -688,23 +688,23 @@ public void invalidDataImport() {

Re: [PR] Remove all usages of deprecated AssertHelpers [iceberg]

2024-06-16 Thread via GitHub
nastra commented on code in PR #10500: URL: https://github.com/apache/iceberg/pull/10500#discussion_r1642269689 ## spark/v3.3/spark/src/test/java/org/apache/iceberg/spark/actions/TestRewriteDataFilesAction.java: ## @@ -643,10 +642,9 @@ public void testSingleCommitWithRewriteFail

Re: [PR] feat: make BoundPredicate,Datum serializable [iceberg-rust]

2024-06-16 Thread via GitHub
sdd commented on code in PR #406: URL: https://github.com/apache/iceberg-rust/pull/406#discussion_r1642258305 ## crates/iceberg/src/spec/values.rs: ## @@ -2334,9 +2453,20 @@ mod _serde { }, RawLiteralEnum::Float(v) => match ty {

Re: [PR] Remove deprecated APIs scheduled for removal in 1.6.0 [iceberg]

2024-06-16 Thread via GitHub
nastra merged PR #10501: URL: https://github.com/apache/iceberg/pull/10501 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Remove deprecated APIs scheduled for removal in 1.6.0 [iceberg]

2024-06-16 Thread via GitHub
nastra commented on code in PR #10501: URL: https://github.com/apache/iceberg/pull/10501#discussion_r1642245973 ## .palantir/revapi.yml: ## @@ -1024,6 +1024,38 @@ acceptedBreaks: old: "class org.apache.iceberg.types.Types.NestedField" new: "class org.apache.iceberg

Re: [PR] Remove deprecated APIs scheduled for removal in 1.6.0 [iceberg]

2024-06-16 Thread via GitHub
nastra commented on code in PR #10501: URL: https://github.com/apache/iceberg/pull/10501#discussion_r1642245973 ## .palantir/revapi.yml: ## @@ -1024,6 +1024,38 @@ acceptedBreaks: old: "class org.apache.iceberg.types.Types.NestedField" new: "class org.apache.iceberg

Re: [PR] feat: make BoundPredicate,Datum serializable [iceberg-rust]

2024-06-16 Thread via GitHub
sdd commented on code in PR #406: URL: https://github.com/apache/iceberg-rust/pull/406#discussion_r1642245641 ## crates/iceberg/src/spec/values.rs: ## @@ -2334,9 +2453,20 @@ mod _serde { }, RawLiteralEnum::Float(v) => match ty {

Re: [PR] Remove deprecated APIs scheduled for removal in 1.6.0 [iceberg]

2024-06-16 Thread via GitHub
nastra commented on code in PR #10501: URL: https://github.com/apache/iceberg/pull/10501#discussion_r1642236950 ## .palantir/revapi.yml: ## @@ -1024,6 +1024,38 @@ acceptedBreaks: old: "class org.apache.iceberg.types.Types.NestedField" new: "class org.apache.iceberg

Re: [PR] feat: make BoundPredicate,Datum serializable [iceberg-rust]

2024-06-16 Thread via GitHub
sdd commented on code in PR #406: URL: https://github.com/apache/iceberg-rust/pull/406#discussion_r1642234401 ## crates/iceberg/src/spec/values.rs: ## @@ -105,6 +108,115 @@ pub struct Datum { literal: PrimitiveLiteral, } +impl Serialize for Datum { +fn serialize( +

Re: [I] "ICEBERG" vs. "iceberg" for "table_type" parameter [iceberg]

2024-06-16 Thread via GitHub
github-actions[bot] commented on issue #2751: URL: https://github.com/apache/iceberg/issues/2751#issuecomment-2171951809 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] Support hive metastore client retries [iceberg]

2024-06-16 Thread via GitHub
github-actions[bot] commented on issue #2741: URL: https://github.com/apache/iceberg/issues/2741#issuecomment-2171951800 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] Getting - java.lang.IllegalArgumentException: Can't zip RDDs with unequal numbers of partitions: List(1, 0) on MERGE INTO [iceberg]

2024-06-16 Thread via GitHub
github-actions[bot] commented on issue #2734: URL: https://github.com/apache/iceberg/issues/2734#issuecomment-2171951788 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] Refactoring table and format doc contents [iceberg]

2024-06-16 Thread via GitHub
github-actions[bot] commented on issue #2728: URL: https://github.com/apache/iceberg/issues/2728#issuecomment-2171951773 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] lost table columns while making iceberg table accessible via hive by adding table property [iceberg]

2024-06-16 Thread via GitHub
github-actions[bot] commented on issue #2725: URL: https://github.com/apache/iceberg/issues/2725#issuecomment-2171951762 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] lost table columns while making iceberg table accessible via hive by adding table property [iceberg]

2024-06-16 Thread via GitHub
github-actions[bot] closed issue #2725: lost table columns while making iceberg table accessible via hive by adding table property URL: https://github.com/apache/iceberg/issues/2725 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [I] Spark SQL extension syntax for identifier field update [iceberg]

2024-06-16 Thread via GitHub
github-actions[bot] commented on issue #2724: URL: https://github.com/apache/iceberg/issues/2724#issuecomment-2171951747 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Spark SQL extension syntax for identifier field update [iceberg]

2024-06-16 Thread via GitHub
github-actions[bot] closed issue #2724: Spark SQL extension syntax for identifier field update URL: https://github.com/apache/iceberg/issues/2724 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] Kafka Connect: Add kerberos authentication option [iceberg]

2024-06-16 Thread via GitHub
bryanck commented on PR #10173: URL: https://github.com/apache/iceberg/pull/10173#issuecomment-2171763317 Thanks for the submission @Dawnpool , one question I have is if we can use the existing facility for setting Hadoop properties, i.e. via the `iceberg.hadoop.*` config? -- This is an

Re: [PR] Kafka Connect: Commit coordination [iceberg]

2024-06-16 Thread via GitHub
bryanck commented on PR #10351: URL: https://github.com/apache/iceberg/pull/10351#issuecomment-2171758810 @ajantha-bhat @fqaiser94 Thanks for the reviews! I think I have addressed comments in case you want to take another look. @Fokko @nastra If either of you have time to take a look, that

Re: [PR] Kafka Connect: Commit coordination [iceberg]

2024-06-16 Thread via GitHub
bryanck commented on code in PR #10351: URL: https://github.com/apache/iceberg/pull/10351#discussion_r1641913835 ## kafka-connect/kafka-connect/src/main/java/org/apache/iceberg/connect/IcebergSinkTask.java: ## @@ -0,0 +1,103 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] Kafka Connect: Commit coordination [iceberg]

2024-06-16 Thread via GitHub
bryanck commented on code in PR #10351: URL: https://github.com/apache/iceberg/pull/10351#discussion_r1641913080 ## kafka-connect/kafka-connect/src/main/java/org/apache/iceberg/connect/IcebergSinkTask.java: ## @@ -0,0 +1,103 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] Kafka Connect: Commit coordination [iceberg]

2024-06-16 Thread via GitHub
bryanck commented on code in PR #10351: URL: https://github.com/apache/iceberg/pull/10351#discussion_r1641910518 ## kafka-connect/kafka-connect/src/main/java/org/apache/iceberg/connect/data/SinkWriter.java: ## @@ -0,0 +1,140 @@ +/* + * Licensed to the Apache Software Foundation

Re: [I] Hive 4.0.0 can't read Iceberg table while inserting data works fine [iceberg]

2024-06-16 Thread via GitHub
nadialeiden commented on issue #10506: URL: https://github.com/apache/iceberg/issues/10506#issuecomment-2171738137 @pvary I see, thank you -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

Re: [PR] feat: make BoundPredicate,Datum serializable [iceberg-rust]

2024-06-16 Thread via GitHub
ZENOTME commented on code in PR #406: URL: https://github.com/apache/iceberg-rust/pull/406#discussion_r1641901794 ## crates/iceberg/src/spec/values.rs: ## @@ -105,6 +108,115 @@ pub struct Datum { literal: PrimitiveLiteral, } +impl Serialize for Datum { +fn serialize(

Re: [PR] feat: make BoundPredicate,Datum serializable [iceberg-rust]

2024-06-16 Thread via GitHub
ZENOTME commented on code in PR #406: URL: https://github.com/apache/iceberg-rust/pull/406#discussion_r1641888209 ## crates/iceberg/src/spec/values.rs: ## @@ -105,6 +108,115 @@ pub struct Datum { literal: PrimitiveLiteral, } +impl Serialize for Datum { +fn serialize(

Re: [PR] feat: make BoundPredicate,Datum serializable [iceberg-rust]

2024-06-16 Thread via GitHub
liurenjie1024 commented on code in PR #406: URL: https://github.com/apache/iceberg-rust/pull/406#discussion_r1641882147 ## crates/iceberg/src/spec/values.rs: ## @@ -105,6 +108,115 @@ pub struct Datum { literal: PrimitiveLiteral, } +impl Serialize for Datum { +fn seri

[PR] Support Remove Branch or Tag APIs [iceberg-python]

2024-06-16 Thread via GitHub
chinmay-bhat opened a new pull request, #822: URL: https://github.com/apache/iceberg-python/pull/822 Support `remove_branch`and `remove_tag` snapshot management APIs. See more [here](https://iceberg.apache.org/docs/nightly/java-api-quickstart/?h=remove+tag#removing-branches-and-tags).

Re: [PR] Concurrent table scans [iceberg-rust]

2024-06-16 Thread via GitHub
liurenjie1024 commented on code in PR #373: URL: https://github.com/apache/iceberg-rust/pull/373#discussion_r1641879767 ## crates/iceberg/src/scan.rs: ## @@ -389,12 +333,158 @@ impl FileScanStreamContext { file_io, bound_filter, case_sensit

Re: [PR] feat: make BoundPredicate,Datum serializable [iceberg-rust]

2024-06-16 Thread via GitHub
ZENOTME commented on PR #406: URL: https://github.com/apache/iceberg-rust/pull/406#issuecomment-2171685472 cc @liurenjie1024 @sdd -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

Re: [PR] Enhancement: refine the reader interface [iceberg-rust]

2024-06-16 Thread via GitHub
ZENOTME commented on PR #401: URL: https://github.com/apache/iceberg-rust/pull/401#issuecomment-2171684939 > > Can we at least split the code that makes `Predicate` serializable out of this PR into it's own PR? I think that is pretty uncontroversial and very useful on its own. > > I

Re: [PR] Enhancement: refine the reader interface [iceberg-rust]

2024-06-16 Thread via GitHub
liurenjie1024 commented on PR #401: URL: https://github.com/apache/iceberg-rust/pull/401#issuecomment-2171676598 > Can we at least split the code that makes `Predicate` serializable out of this PR into it's own PR? I think that is pretty uncontroversial and very useful on its own. I

Re: [PR] Enhancement: refine the reader interface [iceberg-rust]

2024-06-16 Thread via GitHub
liurenjie1024 commented on code in PR #401: URL: https://github.com/apache/iceberg-rust/pull/401#discussion_r1641871941 ## crates/iceberg/src/arrow/reader.rs: ## @@ -74,38 +70,22 @@ impl ArrowReaderBuilder { self } -/// Sets the desired column projection with

Re: [PR] Enhancement: refine the reader interface [iceberg-rust]

2024-06-16 Thread via GitHub
sdd commented on PR #401: URL: https://github.com/apache/iceberg-rust/pull/401#issuecomment-2171407800 Can we at least split the code that makes `Predicate` serializable out of this PR into it's own PR? I think that is pretty uncontroversial and very useful on its own. -- This is an auto

Re: [PR] build(deps): bump github.com/aws/aws-sdk-go-v2/service/glue from 1.73.1 to 1.84.1 [iceberg-go]

2024-06-16 Thread via GitHub
dependabot[bot] closed pull request #99: build(deps): bump github.com/aws/aws-sdk-go-v2/service/glue from 1.73.1 to 1.84.1 URL: https://github.com/apache/iceberg-go/pull/99 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] build(deps): bump github.com/aws/aws-sdk-go-v2/service/glue from 1.73.1 to 1.84.1 [iceberg-go]

2024-06-16 Thread via GitHub
dependabot[bot] commented on PR #99: URL: https://github.com/apache/iceberg-go/pull/99#issuecomment-2171241261 Superseded by #100. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

[PR] build(deps): bump github.com/aws/aws-sdk-go-v2/service/glue from 1.73.1 to 1.85.0 [iceberg-go]

2024-06-16 Thread via GitHub
dependabot[bot] opened a new pull request, #100: URL: https://github.com/apache/iceberg-go/pull/100 Bumps [github.com/aws/aws-sdk-go-v2/service/glue](https://github.com/aws/aws-sdk-go-v2) from 1.73.1 to 1.85.0. Changelog Sourced from https://github.com/aws/aws-sdk-go-v2/blob/servi

Re: [PR] Remove deprecated APIs scheduled for removal in 1.6.0 [iceberg]

2024-06-16 Thread via GitHub
Fokko commented on code in PR #10501: URL: https://github.com/apache/iceberg/pull/10501#discussion_r1641671117 ## .palantir/revapi.yml: ## @@ -1024,6 +1024,38 @@ acceptedBreaks: old: "class org.apache.iceberg.types.Types.NestedField" new: "class org.apache.iceberg.

Re: [PR] Build: Bump datamodel-code-generator from 0.25.6 to 0.25.7 [iceberg]

2024-06-16 Thread via GitHub
Fokko merged PR #10507: URL: https://github.com/apache/iceberg/pull/10507 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] Build: Bump software.amazon.awssdk:bom from 2.25.69 to 2.26.3 [iceberg]

2024-06-16 Thread via GitHub
Fokko merged PR #10505: URL: https://github.com/apache/iceberg/pull/10505 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] Build: Bump io.netty:netty-buffer from 4.1.110.Final to 4.1.111.Final [iceberg]

2024-06-16 Thread via GitHub
Fokko merged PR #10504: URL: https://github.com/apache/iceberg/pull/10504 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] Build: Bump org.springframework:spring-web from 5.3.36 to 5.3.37 [iceberg]

2024-06-16 Thread via GitHub
Fokko merged PR #10503: URL: https://github.com/apache/iceberg/pull/10503 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] Build: Update NOTICE to include copyright for 2024 [iceberg]

2024-06-16 Thread via GitHub
caicancai commented on PR #10471: URL: https://github.com/apache/iceberg/pull/10471#issuecomment-2171146052 > > > While we are here, let's do the other ones, like azure-bundle NOTICE as well? > > > > > > Sorry, I don't seem to understand what you mean. Is there anything you need

Re: [PR] Create rollback and set snapshot APIs [iceberg-python]

2024-06-16 Thread via GitHub
chinmay-bhat commented on code in PR #758: URL: https://github.com/apache/iceberg-python/pull/758#discussion_r1641646010 ## pyiceberg/table/__init__.py: ## @@ -2010,6 +2016,84 @@ def create_branch( self._requirements += requirement return self +def rollba

Re: [PR] Create rollback and set snapshot APIs [iceberg-python]

2024-06-16 Thread via GitHub
chinmay-bhat commented on code in PR #758: URL: https://github.com/apache/iceberg-python/pull/758#discussion_r1641646010 ## pyiceberg/table/__init__.py: ## @@ -2010,6 +2016,84 @@ def create_branch( self._requirements += requirement return self +def rollba

Re: [PR] Create rollback and set snapshot APIs [iceberg-python]

2024-06-16 Thread via GitHub
chinmay-bhat commented on code in PR #758: URL: https://github.com/apache/iceberg-python/pull/758#discussion_r1641646010 ## pyiceberg/table/__init__.py: ## @@ -2010,6 +2016,84 @@ def create_branch( self._requirements += requirement return self +def rollba

Re: [PR] Build: Update NOTICE to include copyright for 2024 [iceberg]

2024-06-16 Thread via GitHub
szehon-ho commented on PR #10471: URL: https://github.com/apache/iceberg/pull/10471#issuecomment-2171122177 > > While we are here, let's do the other ones, like azure-bundle NOTICE as well? > > Sorry, I don't seem to understand what you mean. Is there anything you need me to do?

Re: [PR] Build: Update NOTICE to include copyright for 2024 [iceberg]

2024-06-16 Thread via GitHub
caicancai commented on PR #10471: URL: https://github.com/apache/iceberg/pull/10471#issuecomment-2171118437 > While we are here, let's do the other ones, like azure-bundle NOTICE as well? Sorry, I don't seem to understand what you mean. Is there anything you need me to do? -- This

Re: [PR] Build: Update NOTICE to include copyright for 2024 [iceberg]

2024-06-16 Thread via GitHub
caicancai commented on PR #10471: URL: https://github.com/apache/iceberg/pull/10471#issuecomment-2171118114 > For the record, here's the details about `NOTICE` content: https://www.apache.org/legal/src-headers.html > > IImho, we should have the `NOTICE` only at root level (no need to