Re: [PR] Kafka Connect: Add mechanisms for routing records by topic name [iceberg]

2025-05-30 Thread via GitHub
igorvoltaic commented on PR #11623: URL: https://github.com/apache/iceberg/pull/11623#issuecomment-2924377338 Somehow neither of suggestions gets merged or left with feedback -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

Re: [PR] Reduce code duplication in VectorizedParquetDefinitionLevelReader [iceberg]

2025-05-30 Thread via GitHub
wypoon commented on PR #11661: URL: https://github.com/apache/iceberg/pull/11661#issuecomment-2924369376 `TestRewriteDataFilesAction.testParallelPartialProgressWithMaxFailedCommitsLargerThanTotalFileGroup` seems flaky. I've encountered intermittent failure in this test before. It is unrela

Re: [I] Spark Streaming connector read initial snapshot of iceberg table [iceberg]

2025-05-30 Thread via GitHub
JeonDaehong commented on issue #13188: URL: https://github.com/apache/iceberg/issues/13188#issuecomment-2924248294 > > In our use case, we want the streaming query to process all existing data in the table(regardless of whether there is any historical delete or overwrite) > > present

Re: [PR] Docs: Add docs for Spark SQL Iceberg transform functions (#13156) [iceberg]

2025-05-30 Thread via GitHub
VerneZhong commented on PR #13194: URL: https://github.com/apache/iceberg/pull/13194#issuecomment-2923815101 @manuzhang Added missing license header to fix CI failure. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [I] V3 Tracking issue [iceberg-python]

2025-05-30 Thread via GitHub
stevie9868 commented on issue #1818: URL: https://github.com/apache/iceberg-python/issues/1818#issuecomment-2923788231 @Fokko I am interested in Deletion vectors work if no one is working on it. Also, I see that read for equality deletes are not implemented yet and write for

Re: [PR] Docs: Add docs for Spark SQL Iceberg transform functions (#13156) [iceberg]

2025-05-30 Thread via GitHub
VerneZhong commented on PR #13194: URL: https://github.com/apache/iceberg/pull/13194#issuecomment-2923764565 @manuzhang Hi, this PR adds documentation for Iceberg SQL functions. Please help review when you have time. Also, the workflow is pending your approval. Thanks! -- This is an auto

Re: [PR] Kafka Connect: Add mechanisms for routing records by topic name [iceberg]

2025-05-30 Thread via GitHub
github-actions[bot] commented on PR #11623: URL: https://github.com/apache/iceberg/pull/11623#issuecomment-2923770415 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

[PR] Docs: Add docs for Spark SQL Iceberg transform functions (#13156) [iceberg]

2025-05-30 Thread via GitHub
VerneZhong opened a new pull request, #13194: URL: https://github.com/apache/iceberg/pull/13194 This PR adds documentation for the following Iceberg-related SQL functions: • system.iceberg_version • system.bucket • system.years • system.

Re: [PR] Docs: Add docs for Spark SQL Iceberg transform functions (#13156) [iceberg]

2025-05-30 Thread via GitHub
VerneZhong closed pull request #13194: Docs: Add docs for Spark SQL Iceberg transform functions (#13156) URL: https://github.com/apache/iceberg/pull/13194 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

Re: [PR] AWS: Fix DynamoDB and Glue integration test failures [iceberg]

2025-05-30 Thread via GitHub
yyanyy commented on code in PR #12718: URL: https://github.com/apache/iceberg/pull/12718#discussion_r2116858171 ## aws/src/integration/java/org/apache/iceberg/aws/glue/TestGlueCatalogTable.java: ## @@ -457,16 +507,18 @@ public void testCommitTableSkipArchive() { String tabl

[PR] AWS: Support StaticCredentialsProvider in DefaultAwsClientFactory [iceberg]

2025-05-30 Thread via GitHub
xiaoxuandev opened a new pull request, #12695: URL: https://github.com/apache/iceberg/pull/12695 This change adds support for `StaticCredentialsProvider` in `DefaultAwsClientFactory`. Fixes #10614. Using the following properties to use `StaticCredentialsProvider`: ``` AwsClient

Re: [PR] Spark: clean up obsoleted/unused writer classes [iceberg]

2025-05-30 Thread via GitHub
huaxingao commented on PR #13193: URL: https://github.com/apache/iceberg/pull/13193#issuecomment-2923669824 Thanks @stevenzwu for the PR! Do we need to delete these two classes from Spark3.4 and Spark3.5 too? -- This is an automated message from the Apache Git Service. To respond to the m

Re: [PR] Spark 3.5: Update to Spark 3.5.6 [iceberg]

2025-05-30 Thread via GitHub
huaxingao commented on PR #13142: URL: https://github.com/apache/iceberg/pull/13142#issuecomment-2923665103 Merged. Thanks @drexler-sky for the PR! Thanks @ajantha-bhat for reviewing! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to G

Re: [PR] Spark 3.5: Update to Spark 3.5.6 [iceberg]

2025-05-30 Thread via GitHub
huaxingao merged PR #13142: URL: https://github.com/apache/iceberg/pull/13142 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

[PR] testing(catalog/glue): Verify glue catalog snapshot management is supported [iceberg-go]

2025-05-30 Thread via GitHub
lliangyu-lin opened a new pull request, #448: URL: https://github.com/apache/iceberg-go/pull/448 ### Description * With commit table implemented in glue catalog https://github.com/apache/iceberg-go/pull/403, snapshot management is supported. * Add integration test to verify and showc

Re: [PR] AWS: Support StaticCredentialsProvider in DefaultAwsClientFactory [iceberg]

2025-05-30 Thread via GitHub
yyanyy commented on PR #12695: URL: https://github.com/apache/iceberg/pull/12695#issuecomment-2923579883 reopening since the issue is still relevant; I do agree that this feature is not a best practice in production, but looks like there are two valid use cases for development: - integra

Re: [PR] AWS: Support StaticCredentialsProvider in DefaultAwsClientFactory [iceberg]

2025-05-30 Thread via GitHub
yyanyy commented on PR #12695: URL: https://github.com/apache/iceberg/pull/12695#issuecomment-2923580299 reopening since the issue is still relevant; I do agree that this feature is not a best practice in production, but looks like there are two valid use cases for development: integ

[PR] Spark: clean up obsoleted/unused writer classes [iceberg]

2025-05-30 Thread via GitHub
stevenzwu opened a new pull request, #13193: URL: https://github.com/apache/iceberg/pull/13193 while poking around the writer classes, I found those two classes are not used at all. So clean them up from the 4.0 module. -- This is an automated message from the Apache Git Service. To respo

Re: [PR] Core: Fix numeric overflow of timestamp nano literal [iceberg]

2025-05-30 Thread via GitHub
stevenzwu commented on code in PR #11775: URL: https://github.com/apache/iceberg/pull/11775#discussion_r2116626909 ## api/src/main/java/org/apache/iceberg/expressions/Literals.java: ## @@ -300,8 +300,7 @@ public Literal to(Type type) { case TIMESTAMP: return

Re: [PR] Add `snapshot-loading-mode` option to RESTCatalog [iceberg-python]

2025-05-30 Thread via GitHub
corleyma commented on code in PR #1998: URL: https://github.com/apache/iceberg-python/pull/1998#discussion_r2116504450 ## pyiceberg/catalog/rest/__init__.py: ## @@ -678,7 +679,16 @@ def list_tables(self, namespace: Union[str, Identifier]) -> List[Identifier]: @retry(**_R

Re: [PR] Add `snapshot-loading-mode` option to RESTCatalog [iceberg-python]

2025-05-30 Thread via GitHub
corleyma commented on code in PR #1998: URL: https://github.com/apache/iceberg-python/pull/1998#discussion_r2116504450 ## pyiceberg/catalog/rest/__init__.py: ## @@ -678,7 +679,16 @@ def list_tables(self, namespace: Union[str, Identifier]) -> List[Identifier]: @retry(**_R

Re: [PR] fix(iceberg): add_files correctly check duplicates [iceberg-rust]

2025-05-30 Thread via GitHub
Erigara commented on code in PR #1395: URL: https://github.com/apache/iceberg-rust/pull/1395#discussion_r2116445984 ## crates/iceberg/src/transaction/append.rs: ## @@ -379,6 +364,7 @@ mod tests { // Attempt to add the existing Parquet files with fast append.

Re: [PR] release: improve release.sh script [iceberg-go]

2025-05-30 Thread via GitHub
zeroshade commented on PR #447: URL: https://github.com/apache/iceberg-go/pull/447#issuecomment-2922946828 Ah, okay I'll try that. I'll have to change the release_rc.sh scripts to rename the files and remove the -rc before uploading them to svn. -- This is an automated message from the Ap

Re: [PR] fix(iceberg): add_files correctly check duplicates [iceberg-rust]

2025-05-30 Thread via GitHub
Erigara commented on PR #1395: URL: https://github.com/apache/iceberg-rust/pull/1395#issuecomment-2923146110 I changed my initial implementation so that DELETED manifest entries are skipped. Which is in line with pyiceberg behavior: - https://github.com/apache/iceberg-python/blob/3070

Re: [PR] Introduce MetricsMaxInferredColumnDefaultsStrategy [iceberg]

2025-05-30 Thread via GitHub
jkolash commented on PR #13039: URL: https://github.com/apache/iceberg/pull/13039#issuecomment-2923131118 Thanks, I will simplify this PR to just include the breadth strategy and without a new property. -- This is an automated message from the Apache Git Service. To respond to the message

Re: [PR] Introduce MetricsMaxInferredColumnDefaultsStrategy [iceberg]

2025-05-30 Thread via GitHub
rdblue commented on PR #13039: URL: https://github.com/apache/iceberg/pull/13039#issuecomment-2923078406 I brought this up at the community sync to gauge what other people thought about changing the behavior here without introducing a "strategy" option and the response was positive. Our rat

[PR] Fixed OAuth2Util [iceberg]

2025-05-30 Thread via GitHub
nika-qubit opened a new pull request, #13192: URL: https://github.com/apache/iceberg/pull/13192 Fixed an unintentional crash with IllegalArgumentException when attempting to extract expiration time from arbitrary Bearer token that looks like jwt in OAuth2Util. Without the change, for

Re: [PR] [CORE][REST]: Add context aware response parsing [iceberg]

2025-05-30 Thread via GitHub
singhpk234 commented on PR #13191: URL: https://github.com/apache/iceberg/pull/13191#issuecomment-2922946600 cc @danielcweeks @amogh-jahagirdar @rdblue -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[PR] [CORE][REST]: Add context aware response parsing [iceberg]

2025-05-30 Thread via GitHub
singhpk234 opened a new pull request, #13191: URL: https://github.com/apache/iceberg/pull/13191 ### About the change Presently the REST parsers are stateless aka static, this makes hard to inject a state which can help the parser to construct the response object if it needs some pri

Re: [PR] release: improve release.sh script [iceberg-go]

2025-05-30 Thread via GitHub
kevinjqliu commented on PR #447: URL: https://github.com/apache/iceberg-go/pull/447#issuecomment-2922886348 okay i see what youre trying to do now.. instead of going through the hoops of renaming the files, could we just name the folder with the rc but not the files? ``` a

[PR] fix(iceberg): add_files correctly check duplicates [iceberg-rust]

2025-05-30 Thread via GitHub
Erigara opened a new pull request, #1395: URL: https://github.com/apache/iceberg-rust/pull/1395 ## Which issue does this PR close? - Closes #1394. ## What changes are included in this PR? - compare duplicates by loading manifest files and taking `file_path

Re: [PR] [REST] Add option to configure TLS settings in REST client [iceberg]

2025-05-30 Thread via GitHub
bryanck commented on code in PR #13190: URL: https://github.com/apache/iceberg/pull/13190#discussion_r2116266235 ## core/src/main/java/org/apache/iceberg/rest/auth/TLSConfigurer.java: ## @@ -0,0 +1,38 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or m

Re: [PR] Core: Make pageToken query parameter optional [iceberg]

2025-05-30 Thread via GitHub
kevinjqliu commented on PR #13129: URL: https://github.com/apache/iceberg/pull/13129#issuecomment-2922892369 @elphastori gentle ping, could you follow up on the comments? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and us

Re: [PR] Fix Issue #13064 [iceberg]

2025-05-30 Thread via GitHub
Bhargavkonidena commented on PR #13113: URL: https://github.com/apache/iceberg/pull/13113#issuecomment-2922858972 @manuzhang can you please check above screenshot. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

Re: [I] Spark Streaming connector read initial snapshot of iceberg table [iceberg]

2025-05-30 Thread via GitHub
singhpk234 commented on issue #13188: URL: https://github.com/apache/iceberg/issues/13188#issuecomment-2922849717 > In our use case, we want the streaming query to process all existing data in the table(regardless of whether there is any historical delete or overwrite) presently we ju

Re: [PR] Flink: Support compact in iceberg sink v2 [iceberg]

2025-05-30 Thread via GitHub
Guosmilesmile commented on code in PR #12979: URL: https://github.com/apache/iceberg/pull/12979#discussion_r2115917851 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/sink/CommittableToTableChangeConverter.java: ## @@ -0,0 +1,222 @@ +/* + * Licensed to the Apache Sof

Re: [PR] [REST] Add option to configure TLS settings in REST client [iceberg]

2025-05-30 Thread via GitHub
singhpk234 commented on code in PR #13190: URL: https://github.com/apache/iceberg/pull/13190#discussion_r2116220639 ## core/src/main/java/org/apache/iceberg/rest/auth/TLSConfigurer.java: ## @@ -0,0 +1,38 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * o

Re: [I] ClassCastException in RowDataWrapper.java [iceberg]

2025-05-30 Thread via GitHub
mxm commented on issue #13170: URL: https://github.com/apache/iceberg/issues/13170#issuecomment-2922829484 @tharvey5 Could you share more debug info? For example, the query used and the table types? -- This is an automated message from the Apache Git Service. To respond to the message, pl

[I] bug(iceberg): `add_files` doesn't actually check duplicated files [iceberg-rust]

2025-05-30 Thread via GitHub
Erigara opened a new issue, #1394: URL: https://github.com/apache/iceberg-rust/issues/1394 ### Apache Iceberg Rust version None ### Describe the bug I've noticed that `add_files` with `check_duplicate` doesn't find duplicated files. The problem is that in the impl

Re: [PR] Flink: Dynamic Iceberg Sink: Add table update code for schema comparison and evolution [iceberg]

2025-05-30 Thread via GitHub
mxm commented on code in PR #13032: URL: https://github.com/apache/iceberg/pull/13032#discussion_r2116144575 ## flink/v2.0/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/CompareSchemasVisitor.java: ## @@ -0,0 +1,254 @@ +/* + * Licensed to the Apache Software Foundatio

Re: [PR] Flink: Dynamic Iceberg Sink: Add table update code for schema comparison and evolution [iceberg]

2025-05-30 Thread via GitHub
mxm commented on code in PR #13032: URL: https://github.com/apache/iceberg/pull/13032#discussion_r2116143319 ## flink/v2.0/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/TableUpdater.java: ## @@ -0,0 +1,205 @@ +/* + * Licensed to the Apache Software Foundation (ASF) u

Re: [PR] Flink: Dynamic Iceberg Sink: Add table update code for schema comparison and evolution [iceberg]

2025-05-30 Thread via GitHub
mxm commented on code in PR #13032: URL: https://github.com/apache/iceberg/pull/13032#discussion_r2116144212 ## flink/v2.0/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/EvolveSchemaVisitor.java: ## @@ -0,0 +1,184 @@ +/* + * Licensed to the Apache Software Foundation

[PR] [REST] Add option to configure TLS settings in REST client [iceberg]

2025-05-30 Thread via GitHub
bryanck opened a new pull request, #13190: URL: https://github.com/apache/iceberg/pull/13190 This PR adds an option to the REST client to configure TLS settings via a pluggable configurer class. Java supports setting some TLS parameters via System properties, but doing so will affect all co

Re: [PR] Flink: Support compact in iceberg sink v2 [iceberg]

2025-05-30 Thread via GitHub
pvary commented on code in PR #12979: URL: https://github.com/apache/iceberg/pull/12979#discussion_r2115595974 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/TableChange.java: ## @@ -76,7 +82,11 @@ private TableChange( this.eqDele

Re: [PR] chore(deps): bump cpp-linter/cpp-linter-action from 2.13.3 to 2.15.0 [iceberg-cpp]

2025-05-30 Thread via GitHub
wgtmac commented on PR #114: URL: https://github.com/apache/iceberg-cpp/pull/114#issuecomment-2922559432 I think the Infra team has changed the rule to use Github actions. Let me investigate what is the best way for this. -- This is an automated message from the Apache Git Service. To res

Re: [PR] chore(deps): bump cpp-linter/cpp-linter-action from 2.13.3 to 2.15.0 [iceberg-cpp]

2025-05-30 Thread via GitHub
zhjwpku commented on PR #114: URL: https://github.com/apache/iceberg-cpp/pull/114#issuecomment-2922546780 @Fokko @Xuanwo Could you add a comment to prevent the dependabot from creating more of this? Or maybe just ignore this major version, as we might want to upgrade it in the future by re

Re: [PR] chore(deps): bump cpp-linter/cpp-linter-action from 2.13.3 to 2.15.0 [iceberg-cpp]

2025-05-30 Thread via GitHub
dependabot[bot] commented on PR #114: URL: https://github.com/apache/iceberg-cpp/pull/114#issuecomment-2922532054 Sorry, only users with push access can use that command. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] chore(deps): bump cpp-linter/cpp-linter-action from 2.13.3 to 2.15.0 [iceberg-cpp]

2025-05-30 Thread via GitHub
zhjwpku commented on PR #114: URL: https://github.com/apache/iceberg-cpp/pull/114#issuecomment-2922531934 @dependabot ignore this dependency Apache infra restricts the use of cpp-linter/cpp-linter-action@v2.13.3 in apache/iceberg-cpp, so please stop creating more for this dependency. --

Re: [PR] Spec: Add DV information in overview [iceberg]

2025-05-30 Thread via GitHub
ajantha-bhat commented on code in PR #13189: URL: https://github.com/apache/iceberg/pull/13189#discussion_r2115760706 ## format/spec.md: ## @@ -101,10 +101,10 @@ Inheriting the sequence number from manifest metadata allows writing a new manif Row-level deletes are stored in

Re: [PR] Flink: Support compact in iceberg sink v2 [iceberg]

2025-05-30 Thread via GitHub
Guosmilesmile commented on code in PR #12979: URL: https://github.com/apache/iceberg/pull/12979#discussion_r2115920374 ## flink/v1.20/flink/src/test/java/org/apache/iceberg/flink/TestFlinkTableSinkCompaction.java: ## @@ -0,0 +1,182 @@ +/* + * Licensed to the Apache Software Foun

Re: [PR] Flink: Support compact in iceberg sink v2 [iceberg]

2025-05-30 Thread via GitHub
Guosmilesmile commented on code in PR #12979: URL: https://github.com/apache/iceberg/pull/12979#discussion_r2115917406 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/TriggerEvaluator.java: ## @@ -100,6 +100,22 @@ public Builder timeout(Duration

Re: [PR] Hive: Throw exception for when listing a non-existing namespace [iceberg]

2025-05-30 Thread via GitHub
jmelinav commented on code in PR #13130: URL: https://github.com/apache/iceberg/pull/13130#discussion_r2115914352 ## hive-metastore/src/test/java/org/apache/iceberg/hive/TestHiveCatalog.java: ## @@ -1209,11 +1208,4 @@ public void testDatabaseLocationWithSlashInWarehouseDir() {

Re: [PR] Flink: Support compact in iceberg sink v2 [iceberg]

2025-05-30 Thread via GitHub
Guosmilesmile commented on code in PR #12979: URL: https://github.com/apache/iceberg/pull/12979#discussion_r2115921971 ## flink/v1.20/flink/src/test/java/org/apache/iceberg/flink/sink/TestCommittableToTableChangeConverter.java: ## @@ -0,0 +1,338 @@ +/* + * Licensed to the Apache

Re: [PR] Flink: Migrate Flink TableSchema for IcebergSource [iceberg]

2025-05-30 Thread via GitHub
mxm commented on code in PR #13072: URL: https://github.com/apache/iceberg/pull/13072#discussion_r2115922995 ## flink/v2.0/flink/src/main/java/org/apache/iceberg/flink/FlinkSchemaUtil.java: ## @@ -192,7 +222,9 @@ public static Type convert(LogicalType flinkType) { * * @p

Re: [PR] Flink: Support compact in iceberg sink v2 [iceberg]

2025-05-30 Thread via GitHub
Guosmilesmile commented on code in PR #12979: URL: https://github.com/apache/iceberg/pull/12979#discussion_r2115921453 ## flink/v1.20/flink/src/test/java/org/apache/iceberg/flink/maintenance/api/TestRewriteDataFilesConfig.java: ## @@ -0,0 +1,147 @@ +/* + * Licensed to the Apache

Re: [PR] Hive: Throw exception for when listing a non-existing namespace [iceberg]

2025-05-30 Thread via GitHub
jmelinav commented on code in PR #13130: URL: https://github.com/apache/iceberg/pull/13130#discussion_r2115914352 ## hive-metastore/src/test/java/org/apache/iceberg/hive/TestHiveCatalog.java: ## @@ -1209,11 +1208,4 @@ public void testDatabaseLocationWithSlashInWarehouseDir() {

Re: [PR] Flink: Support compact in iceberg sink v2 [iceberg]

2025-05-30 Thread via GitHub
Guosmilesmile commented on code in PR #12979: URL: https://github.com/apache/iceberg/pull/12979#discussion_r2115915255 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/maintenance/api/RewriteDataFilesConfig.java: ## @@ -0,0 +1,117 @@ +/* + * Licensed to the Apache Sof

Re: [PR] Flink: Support compact in iceberg sink v2 [iceberg]

2025-05-30 Thread via GitHub
Guosmilesmile commented on code in PR #12979: URL: https://github.com/apache/iceberg/pull/12979#discussion_r2115916429 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/TableChange.java: ## @@ -76,7 +82,11 @@ private TableChange( thi

Re: [PR] Flink: Support compact in iceberg sink v2 [iceberg]

2025-05-30 Thread via GitHub
Guosmilesmile commented on code in PR #12979: URL: https://github.com/apache/iceberg/pull/12979#discussion_r2115912010 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/FlinkConfParser.java: ## @@ -29,13 +29,13 @@ import org.apache.iceberg.relocated.com.google.common.

Re: [I] S3 client for storage path not available [iceberg]

2025-05-30 Thread via GitHub
and124578963 commented on issue #13174: URL: https://github.com/apache/iceberg/issues/13174#issuecomment-2921827619 Hey @vanshb03, Typically, I use Iceberg 1.7.2, and my configuration with s3a:// works well. I found the following catalog-related properties in spark-defaults.conf

Re: [PR] feat: implement avro file reader [iceberg-cpp]

2025-05-30 Thread via GitHub
lidavidm commented on code in PR #113: URL: https://github.com/apache/iceberg-cpp/pull/113#discussion_r2115790873 ## src/iceberg/expected.h: ## @@ -122,7 +122,7 @@ inline constexpr bool is_error_type_valid_v = is_error_type_valid::value; } // namespace expected_detail temp

Re: [PR] Spec: Add DV information in overview [iceberg]

2025-05-30 Thread via GitHub
ajantha-bhat commented on PR #13189: URL: https://github.com/apache/iceberg/pull/13189#issuecomment-2922149448 @rdblue, @aokolnychyi, @RussellSpitzer: Let me know what you guys think on this. While scanning the spec for v3 completion, I felt this section needs an update. -- This is a

[PR] Spec: Add DV information in overview [iceberg]

2025-05-30 Thread via GitHub
ajantha-bhat opened a new pull request, #13189: URL: https://github.com/apache/iceberg/pull/13189 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscrib

Re: [PR] Flink: Support compact in iceberg sink v2 [iceberg]

2025-05-30 Thread via GitHub
pvary commented on code in PR #12979: URL: https://github.com/apache/iceberg/pull/12979#discussion_r2115611012 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/maintenance/api/RewriteDataFilesConfig.java: ## @@ -0,0 +1,117 @@ +/* + * Licensed to the Apache Software Fo

Re: [PR] Flink: Support compact in iceberg sink v2 [iceberg]

2025-05-30 Thread via GitHub
pvary commented on code in PR #12979: URL: https://github.com/apache/iceberg/pull/12979#discussion_r2115605286 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/maintenance/api/FlinkMaintenanceConfig.java: ## @@ -0,0 +1,89 @@ +/* + * Licensed to the Apache Software Fou

Re: [PR] Flink: Support compact in iceberg sink v2 [iceberg]

2025-05-30 Thread via GitHub
pvary commented on code in PR #12979: URL: https://github.com/apache/iceberg/pull/12979#discussion_r2115600785 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/TriggerEvaluator.java: ## @@ -100,6 +100,22 @@ public Builder timeout(Duration newTimeo

Re: [I] S3 client for storage path not available [iceberg]

2025-05-30 Thread via GitHub
hemanthboyina commented on issue #13174: URL: https://github.com/apache/iceberg/issues/13174#issuecomment-2921921245 s3a:// might not be the problem over here. As by default for ROOT_PREFIX="s3" we have a PrefixedS3Client added in Map, even though your storage path doesn't match, the defaul

Re: [PR] Spark: Iceberg_Spark Add delegation token for HiveCatalog [iceberg]

2025-05-30 Thread via GitHub
pvary commented on code in PR #13187: URL: https://github.com/apache/iceberg/pull/13187#discussion_r2115570053 ## spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/security/IcebergHiveConnectorDelegationTokenProvider.java: ## @@ -0,0 +1,237 @@ +/* + * Licensed to the Apach

Re: [PR] Spark: Iceberg_Spark Add delegation token for HiveCatalog [iceberg]

2025-05-30 Thread via GitHub
pvary commented on code in PR #13187: URL: https://github.com/apache/iceberg/pull/13187#discussion_r2115567983 ## spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/security/IcebergHiveConnectorDelegationTokenProvider.java: ## @@ -0,0 +1,237 @@ +/* + * Licensed to the Apach

Re: [PR] feat: implement avro file reader [iceberg-cpp]

2025-05-30 Thread via GitHub
wgtmac commented on code in PR #113: URL: https://github.com/apache/iceberg-cpp/pull/113#discussion_r2115528845 ## src/iceberg/expected.h: ## @@ -122,7 +122,7 @@ inline constexpr bool is_error_type_valid_v = is_error_type_valid::value; } // namespace expected_detail templa

Re: [PR] Core: introduce shared authentication refresh executor [iceberg]

2025-05-30 Thread via GitHub
adutra commented on PR #12563: URL: https://github.com/apache/iceberg/pull/12563#issuecomment-2921638232 Pinging @nastra and @danielcweeks again: do you think we can consider this one for 1.10? -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] Core/REST: generify AuthSessionCache [iceberg]

2025-05-30 Thread via GitHub
adutra commented on PR #12562: URL: https://github.com/apache/iceberg/pull/12562#issuecomment-2921639572 Pinging @nastra and @danielcweeks again: do you think we can consider this one for 1.10? -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] feat: implement avro file reader [iceberg-cpp]

2025-05-30 Thread via GitHub
wgtmac commented on code in PR #113: URL: https://github.com/apache/iceberg-cpp/pull/113#discussion_r2115392858 ## src/iceberg/avro/avro_reader.cc: ## @@ -0,0 +1,222 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements.

Re: [I] Spark: add IcebergHiveConnectorDelegationTokenProvider for HiveCatalog [iceberg]

2025-05-30 Thread via GitHub
pan3793 commented on issue #13116: URL: https://github.com/apache/iceberg/issues/13116#issuecomment-2921607470 > but kyuubi can only fetch the delegation token of one HiveMetaStore. Incorrect, it supports an arbitrary number of HMS -- This is an automated message from the Apache Git

Re: [PR] feat: implement avro file reader [iceberg-cpp]

2025-05-30 Thread via GitHub
mapleFU commented on code in PR #113: URL: https://github.com/apache/iceberg-cpp/pull/113#discussion_r2115398710 ## src/iceberg/avro/avro_reader.cc: ## @@ -0,0 +1,222 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements

Re: [PR] feat: implement avro file reader [iceberg-cpp]

2025-05-30 Thread via GitHub
wgtmac commented on code in PR #113: URL: https://github.com/apache/iceberg-cpp/pull/113#discussion_r2115396425 ## src/iceberg/avro/avro_reader.cc: ## @@ -0,0 +1,222 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements.

Re: [PR] feat: implement avro file reader [iceberg-cpp]

2025-05-30 Thread via GitHub
wgtmac commented on code in PR #113: URL: https://github.com/apache/iceberg-cpp/pull/113#discussion_r2115395582 ## src/iceberg/avro/avro_reader.cc: ## @@ -0,0 +1,222 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements.

Re: [PR] feat: implement avro file reader [iceberg-cpp]

2025-05-30 Thread via GitHub
wgtmac commented on code in PR #113: URL: https://github.com/apache/iceberg-cpp/pull/113#discussion_r2115393660 ## src/iceberg/avro/avro_data_util.cc: ## @@ -0,0 +1,32 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreement

Re: [PR] feat: implement avro file reader [iceberg-cpp]

2025-05-30 Thread via GitHub
mapleFU commented on code in PR #113: URL: https://github.com/apache/iceberg-cpp/pull/113#discussion_r2115383164 ## src/iceberg/avro/avro_reader.cc: ## @@ -0,0 +1,222 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements

[PR] chore(deps): bump cpp-linter/cpp-linter-action from 2.13.3 to 2.15.0 [iceberg-cpp]

2025-05-30 Thread via GitHub
dependabot[bot] opened a new pull request, #114: URL: https://github.com/apache/iceberg-cpp/pull/114 Bumps [cpp-linter/cpp-linter-action](https://github.com/cpp-linter/cpp-linter-action) from 2.13.3 to 2.15.0. Release notes Sourced from https://github.com/cpp-linter/cpp-linter-act

Re: [I] Spark Streaming connector read initial snapshot of iceberg table [iceberg]

2025-05-30 Thread via GitHub
JeonDaehong commented on issue #13188: URL: https://github.com/apache/iceberg/issues/13188#issuecomment-2921535443 Hello! Are you looking for a feature improvement? I'm very interested in Apache Iceberg and would love to contribute by helping address issues like this.:D -- This is an aut

Re: [I] Spark: add IcebergHiveConnectorDelegationTokenProvider for HiveCatalog [iceberg]

2025-05-30 Thread via GitHub
zhangwl9 commented on issue #13116: URL: https://github.com/apache/iceberg/issues/13116#issuecomment-2921556739 > Glad to see that the community is considering multiple HMS cases, actually, a similar work has been done in Kyuubi Spark Hive Connector - a Hive connector based on Spark DSv2 AP

Re: [PR] feat: implement avro file reader [iceberg-cpp]

2025-05-30 Thread via GitHub
lidavidm commented on code in PR #113: URL: https://github.com/apache/iceberg-cpp/pull/113#discussion_r2115359723 ## src/iceberg/avro/avro_reader.cc: ## @@ -0,0 +1,222 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreement

[I] Spark Streaming connector read initial snapshot of iceberg table [iceberg]

2025-05-30 Thread via GitHub
chaoqin-li1123 opened a new issue, #13188: URL: https://github.com/apache/iceberg/issues/13188 ### Feature Request / Improvement The spark structured streaming connector do not scan all files in initial table snapshot. In our use case, we want the streaming query to process all

[PR] Iceberg_Spark Add delegation token for HiveCatalog [iceberg]

2025-05-30 Thread via GitHub
zhangwl9 opened a new pull request, #13187: URL: https://github.com/apache/iceberg/pull/13187 Implementing Delegation Token Fetching for HiveCatalog in the Iceberg_Spark Module close #13116 -- This is an automated message from the Apache Git Service. To respond to the message, please

[PR] Azure: KeyManagementClient implementation for Azure Key Vault [iceberg]

2025-05-30 Thread via GitHub
nandorKollar opened a new pull request, #13186: URL: https://github.com/apache/iceberg/pull/13186 KeyManagement client implementation for connecting to Azure Key Vault. Unlike other key managers (like KMS on AWS), it seems that Key Vault doesn't support key generation, therefore `supportsKe

Re: [I] Spark, Avro and Iceberg Timestamp, Timestamp NTZ definition need more clarifications [iceberg]

2025-05-30 Thread via GitHub
anesterenok commented on issue #12751: URL: https://github.com/apache/iceberg/issues/12751#issuecomment-2921446275 Oh, and since there's `timestamp_ns` now, the same goes for it: should be `local-timestamp-nanos`, not `timestamp-nanos`. -- This is an automated message from the Apache Git

Re: [I] Spark, Avro and Iceberg Timestamp, Timestamp NTZ definition need more clarifications [iceberg]

2025-05-30 Thread via GitHub
anesterenok commented on issue #12751: URL: https://github.com/apache/iceberg/issues/12751#issuecomment-2921427738 > we just need to change the written logical type. Yes, I guess that's it. -- This is an automated message from the Apache Git Service. To respond to the message, plea