[PR] Core: Use `isFile` for hadoop table metadata file operations [iceberg]

IzzelAliz opened a new pull request, #11690: URL: https://github.com/apache/iceberg/pull/11690 The s3a filesystem implementation of `exists` calls are headobject and listobjects, where listobjects do not scale well in some self hosted s3 implementations (minio). This PR replace `exis

Re: [PR] Core, Spark: Refactor RewriteFileGroup planner to core [iceberg]

pvary commented on code in PR #11513: URL: https://github.com/apache/iceberg/pull/11513#discussion_r1867601536 ## core/src/main/java/org/apache/iceberg/actions/RewritePositionDeletesGroupPlanner.java: ## @@ -0,0 +1,238 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] Fix when reading struct-type data without an id in iceberg-parquet [iceberg]

Fokko commented on code in PR #11378: URL: https://github.com/apache/iceberg/pull/11378#discussion_r1867603193 ## parquet/src/main/java/org/apache/iceberg/parquet/ParquetWriter.java: ## @@ -49,7 +49,7 @@ class ParquetWriter implements FileAppender, Closeable { private final l

Re: [I] How do I find if there is residual in the table scan/plan files? [iceberg-python]

tusharchou commented on issue #785: URL: https://github.com/apache/iceberg-python/issues/785#issuecomment-2514435875 @Fokko I am picking this up in #1223 #1388 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

Re: [PR] Encryption integration and test [iceberg]

ggershinsky commented on PR #5544: URL: https://github.com/apache/iceberg/pull/5544#issuecomment-2514437399 Thanks @nastra . This PR is heavily dependent on #7770 , so can be properly rebased after the latter is merged. Both PRs are active, awaiting a progress in the 7770. -- This is an

Re: [PR] Fix when reading struct-type data without an id in iceberg-parquet [iceberg]

Fokko commented on code in PR #11378: URL: https://github.com/apache/iceberg/pull/11378#discussion_r1867655469 ## parquet/src/main/java/org/apache/iceberg/data/parquet/BaseParquetReaders.java: ## @@ -291,6 +291,13 @@ public ParquetValueReader struct( return createStructRe

Re: [I] How do I find if there is residual in the table scan/plan files? [iceberg-python]

Fokko commented on issue #785: URL: https://github.com/apache/iceberg-python/issues/785#issuecomment-2514456361 @tusharchou Thank you, I've removed the stale label 🙌 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] Core: Generalize Util::blockLocations [iceberg]

okumin commented on code in PR #11053: URL: https://github.com/apache/iceberg/pull/11053#discussion_r1867658087 ## .palantir/revapi.yml: ## @@ -1100,6 +1100,12 @@ acceptedBreaks: - code: "java.class.removed" old: "enum org.apache.iceberg.BaseMetastoreTableOperations.

[PR] Ignore partition fields that reference a dropped source-id [iceberg-python]

Fokko opened a new pull request, #1393: URL: https://github.com/apache/iceberg-python/pull/1393 Let's first get consensus on the Java side of things: https://github.com/apache/iceberg/pull/11604 -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [I] Why shouldn't we return an `UnboundPartitionSpec` instead? [iceberg-rust]

Fokko commented on issue #694: URL: https://github.com/apache/iceberg-rust/issues/694#issuecomment-2514536378 I've created https://github.com/apache/iceberg/pull/11604 to reflect my idea on the Java side. When you have `PartitionSpec` that references a field that has been dropped, then we c

Re: [PR] Core, Spark: Refactor RewriteFileGroup planner to core [iceberg]

pvary commented on code in PR #11513: URL: https://github.com/apache/iceberg/pull/11513#discussion_r1867596164 ## core/src/main/java/org/apache/iceberg/actions/RewritePositionDeletePlan.java: ## @@ -0,0 +1,41 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

Re: [PR] Add basic CMake support for the iceberg library [iceberg-cpp]

gaborkaszab commented on code in PR #3: URL: https://github.com/apache/iceberg-cpp/pull/3#discussion_r1866085625 ## src/demo.cc: ## @@ -0,0 +1,26 @@ +/* Review Comment: Seems great! -- This is an automated message from the Apache Git Service. To respond to the message, p

Re: [PR] Core, Spark: Refactor RewriteFileGroup planner to core [iceberg]

pvary commented on code in PR #11513: URL: https://github.com/apache/iceberg/pull/11513#discussion_r1867620928 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/SparkShufflingDataRewritePlanner.java: ## @@ -0,0 +1,84 @@ +/* + * Licensed to the Apache Software Fo

Re: [PR] Fix when reading struct-type data without an id in iceberg-parquet [iceberg]

joyCurry30 commented on code in PR #11378: URL: https://github.com/apache/iceberg/pull/11378#discussion_r1867625158 ## parquet/src/main/java/org/apache/iceberg/parquet/ParquetWriter.java: ## @@ -49,7 +49,7 @@ class ParquetWriter implements FileAppender, Closeable { private fi

Re: [PR] Core, Spark: Refactor RewriteFileGroup planner to core [iceberg]

pvary commented on code in PR #11513: URL: https://github.com/apache/iceberg/pull/11513#discussion_r1867596164 ## core/src/main/java/org/apache/iceberg/actions/RewritePositionDeletePlan.java: ## @@ -0,0 +1,41 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

[PR] Add missing license headers [iceberg-python]

manuzhang opened a new pull request, #1396: URL: https://github.com/apache/iceberg-python/pull/1396 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscr

Re: [PR] Add basic CMake support for the iceberg library [iceberg-cpp]

wgtmac commented on code in PR #3: URL: https://github.com/apache/iceberg-cpp/pull/3#discussion_r1867887143 ## api/CMakeLists.txt: ## @@ -0,0 +1,22 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +#

Re: [PR] Flink: Maintenance - RewriteDataFiles [iceberg]

pvary commented on code in PR #11497: URL: https://github.com/apache/iceberg/pull/11497#discussion_r1867878070 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/DataFileRewriteCommitter.java: ## @@ -0,0 +1,304 @@ +/* + * Licensed to the Apache Soft

Re: [PR] Add basic CMake support for the iceberg library [iceberg-cpp]

wgtmac commented on code in PR #3: URL: https://github.com/apache/iceberg-cpp/pull/3#discussion_r1867907240 ## src/core/demo_table.h: ## @@ -0,0 +1,34 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTIC

Re: [PR] Add basic CMake support for the iceberg library [iceberg-cpp]

wgtmac commented on code in PR #3: URL: https://github.com/apache/iceberg-cpp/pull/3#discussion_r1867907240 ## src/core/demo_table.h: ## @@ -0,0 +1,34 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTIC

Re: [PR] Add basic CMake support for the iceberg library [iceberg-cpp]

wgtmac commented on code in PR #3: URL: https://github.com/apache/iceberg-cpp/pull/3#discussion_r1867907240 ## src/core/demo_table.h: ## @@ -0,0 +1,34 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTIC

Re: [PR] Add basic CMake support for the iceberg library [iceberg-cpp]

wgtmac commented on code in PR #3: URL: https://github.com/apache/iceberg-cpp/pull/3#discussion_r1867914158 ## cmake_modules/BuildUtils.cmake: ## @@ -0,0 +1,212 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the N

Re: [PR] Add basic CMake support for the iceberg library [iceberg-cpp]

wgtmac commented on code in PR #3: URL: https://github.com/apache/iceberg-cpp/pull/3#discussion_r1867893586 ## CMakeLists.txt: ## @@ -0,0 +1,63 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# dis

Re: [PR] Add basic CMake support for the iceberg library [iceberg-cpp]

wgtmac commented on code in PR #3: URL: https://github.com/apache/iceberg-cpp/pull/3#discussion_r1867897791 ## README.md: ## @@ -21,6 +21,11 @@ C++ implementation of [Apache Iceberg™](https://iceberg.apache.org/). +## Requirements Review Comment: Good suggestion! I'll a

Re: [PR] Add basic CMake support for the iceberg library [iceberg-cpp]

wgtmac commented on code in PR #3: URL: https://github.com/apache/iceberg-cpp/pull/3#discussion_r1867920857 ## src/core/icebergConfig.cmake.in: ## @@ -0,0 +1,22 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the N

Re: [PR] Add basic CMake support for the iceberg library [iceberg-cpp]

wgtmac commented on code in PR #3: URL: https://github.com/apache/iceberg-cpp/pull/3#discussion_r1867948819 ## cmake_modules/BuildUtils.cmake: ## @@ -0,0 +1,212 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the N

Re: [PR] Spark: Remove extra columns for ColumnBatch [iceberg]

huaxingao commented on PR #11551: URL: https://github.com/apache/iceberg/pull/11551#issuecomment-2512236843 @flyrain I will have a follow-up PR. Thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

Re: [PR] Create publish-docker.yml [iceberg]

sungwy commented on code in PR #11632: URL: https://github.com/apache/iceberg/pull/11632#discussion_r1865927839 ## .github/workflows/publish-docker.yml: ## @@ -0,0 +1,51 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements

Re: [PR] chore: Align argument name with doc comment [iceberg-rust]

c-thiel commented on PR #750: URL: https://github.com/apache/iceberg-rust/pull/750#issuecomment-2513854888 @Fokko LGTM, thanks @SergeiPatiakin! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] chore: Align argument name with doc comment [iceberg-rust]

Fokko merged PR #750: URL: https://github.com/apache/iceberg-rust/pull/750 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [I] Build Failure When Using JDK 1.8 with Iceberg 1.4.x [iceberg]

nastra commented on issue #11689: URL: https://github.com/apache/iceberg/issues/11689#issuecomment-2513947002 I do recall that this was potentially an issue with the Shadow plugin we were using at the time when 1.4.x was released (https://github.com/GradleUp/shadow/issues/894). We're not go

Re: [PR] Azure: Support vended credentials refresh in ADLSFileIO. [iceberg]

nastra commented on PR #11577: URL: https://github.com/apache/iceberg/pull/11577#issuecomment-2514171444 @ChaladiMohanVamsi thanks for working on this. Do you have a way of actually testing this PR with an ADLS environment and see whether the refreshes work? -- This is an automated messag

Re: [PR] API: Support removeUnusedSpecs in ExpireSnapshots [iceberg]

amogh-jahagirdar commented on code in PR #10755: URL: https://github.com/apache/iceberg/pull/10755#discussion_r1866315984 ## api/src/main/java/org/apache/iceberg/ExpireSnapshots.java: ## @@ -118,4 +118,16 @@ public interface ExpireSnapshots extends PendingUpdate> { * @retur

Re: [PR] Azure: Support vended credentials refresh in ADLSFileIO. [iceberg]

nastra commented on code in PR #11577: URL: https://github.com/apache/iceberg/pull/11577#discussion_r1867465906 ## azure/src/test/java/org/apache/iceberg/azure/adlsv2/VendedAzureSasCredentialProviderTest.java: ## @@ -0,0 +1,323 @@ +/* + * Licensed to the Apache Software Foundati

Re: [PR] Add basic CMake support for the iceberg library [iceberg-cpp]

wgtmac commented on PR #3: URL: https://github.com/apache/iceberg-cpp/pull/3#issuecomment-2513818638 I think I have addressed all comments. The `BuildUtils.cmake` is borrowed from Apache Arrow (a comment has been added to reflect this). Let me know what do you think. Thanks! @Fokko @gaborka

Re: [PR] Added force virtual addressing configuration for S3, oss and r2 protocol to use PyArrowFileIO [iceberg-python]

helmiazizm commented on code in PR #1392: URL: https://github.com/apache/iceberg-python/pull/1392#discussion_r1866939185 ## pyiceberg/io/pyarrow.py: ## @@ -350,7 +351,7 @@ def parse_location(location: str) -> Tuple[str, str, str]: return uri.scheme, uri.netloc, f"{u

Re: [PR] Added force virtual addressing configuration for S3, oss and r2 protocol to use PyArrowFileIO [iceberg-python]

helmiazizm commented on code in PR #1392: URL: https://github.com/apache/iceberg-python/pull/1392#discussion_r1867233125 ## pyiceberg/io/pyarrow.py: ## @@ -350,7 +351,7 @@ def parse_location(location: str) -> Tuple[str, str, str]: return uri.scheme, uri.netloc, f"{u

[I] Add `curl` to the `iceberg-rest-fixture` docker container [iceberg]

Fokko opened a new issue, #11688: URL: https://github.com/apache/iceberg/issues/11688 ### Feature Request / Improvement While checking the integration tests on Iceberg-rust (https://github.com/apache/iceberg-rust/pull/748 specifically), it might be nice to add curl to the docker imag

Re: [PR] Azure: Support vended credentials refresh in ADLSFileIO. [iceberg]

nastra commented on code in PR #11577: URL: https://github.com/apache/iceberg/pull/11577#discussion_r1867398362 ## azure/src/main/java/org/apache/iceberg/azure/AzureProperties.java: ## @@ -25,24 +25,40 @@ import java.util.Collections; import java.util.Map; import java.util.Op

Re: [PR] Azure: Support vended credentials refresh in ADLSFileIO. [iceberg]

nastra commented on code in PR #11577: URL: https://github.com/apache/iceberg/pull/11577#discussion_r1867399820 ## azure/src/main/java/org/apache/iceberg/azure/AzureProperties.java: ## @@ -25,24 +25,40 @@ import java.util.Collections; import java.util.Map; import java.util.Op

Re: [PR] Azure: Support vended credentials refresh in ADLSFileIO. [iceberg]

nastra commented on code in PR #11577: URL: https://github.com/apache/iceberg/pull/11577#discussion_r1867400944 ## azure/src/main/java/org/apache/iceberg/azure/AzureProperties.java: ## @@ -25,24 +25,40 @@ import java.util.Collections; import java.util.Map; import java.util.Op

Re: [I] Unsupported class file major version 65 when building Iceberg 1.4.x with JDK 1.8 [iceberg]

nastra closed issue #11689: Unsupported class file major version 65 when building Iceberg 1.4.x with JDK 1.8 URL: https://github.com/apache/iceberg/issues/11689 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

Re: [I] Unsupported class file major version 65 when building Iceberg 1.4.x with JDK 1.8 [iceberg]

nastra commented on issue #11689: URL: https://github.com/apache/iceberg/issues/11689#issuecomment-2514053884 thanks for confirming @JackeeH. I'll close this then since we have a documented workaround for this -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] Azure: Support vended credentials refresh in ADLSFileIO. [iceberg]

nastra commented on code in PR #11577: URL: https://github.com/apache/iceberg/pull/11577#discussion_r1867402766 ## azure/src/main/java/org/apache/iceberg/azure/AzureProperties.java: ## @@ -25,24 +25,40 @@ import java.util.Collections; import java.util.Map; import java.util.Op

Re: [PR] Hive: Optimize tableExists API in hive catalog [iceberg]

gaborkaszab commented on code in PR #11597: URL: https://github.com/apache/iceberg/pull/11597#discussion_r1867329974 ## hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java: ## @@ -412,6 +412,34 @@ private void validateTableIsIcebergTableOrView( } } +

Re: [PR] Azure: Support vended credentials refresh in ADLSFileIO. [iceberg]

nastra commented on code in PR #11577: URL: https://github.com/apache/iceberg/pull/11577#discussion_r1867409678 ## azure/src/main/java/org/apache/iceberg/azure/AzureProperties.java: ## @@ -67,6 +83,16 @@ public AzureProperties(Map properties) { if (properties.containsKey(AD

Re: [PR] Azure: Support vended credentials refresh in ADLSFileIO. [iceberg]

nastra commented on code in PR #11577: URL: https://github.com/apache/iceberg/pull/11577#discussion_r1867416416 ## azure/src/main/java/org/apache/iceberg/azure/adlsv2/VendedAzureSasCredentialProvider.java: ## @@ -0,0 +1,197 @@ +/* + * Licensed to the Apache Software Foundation (

Re: [PR] Azure: Support vended credentials refresh in ADLSFileIO. [iceberg]

nastra commented on code in PR #11577: URL: https://github.com/apache/iceberg/pull/11577#discussion_r1867428213 ## azure/src/main/java/org/apache/iceberg/azure/adlsv2/VendedAzureSasCredentialProvider.java: ## @@ -0,0 +1,197 @@ +/* + * Licensed to the Apache Software Foundation (

Re: [PR] Azure: Support vended credentials refresh in ADLSFileIO. [iceberg]

nastra commented on code in PR #11577: URL: https://github.com/apache/iceberg/pull/11577#discussion_r1867429049 ## azure/src/main/java/org/apache/iceberg/azure/adlsv2/VendedAzureSasCredentialProvider.java: ## @@ -0,0 +1,197 @@ +/* + * Licensed to the Apache Software Foundation (

Re: [I] the `where` sql in rewriteDataFilesAction is always case sensitive [iceberg]

ludlows closed issue #11438: the `where` sql in rewriteDataFilesAction is always case sensitive URL: https://github.com/apache/iceberg/issues/11438 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

Re: [PR] Hive: Optimize tableExists API in hive catalog [iceberg]

pvary commented on code in PR #11597: URL: https://github.com/apache/iceberg/pull/11597#discussion_r1867303394 ## api/src/main/java/org/apache/iceberg/catalog/Catalog.java: ## @@ -271,7 +271,7 @@ default Transaction newReplaceTableTransaction( } /** - * Check whether t

Re: [PR] Add basic CMake support for the iceberg library [iceberg-cpp]

gaborkaszab commented on code in PR #3: URL: https://github.com/apache/iceberg-cpp/pull/3#discussion_r1867249560 ## CMakeLists.txt: ## @@ -0,0 +1,63 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +

Re: [PR] Core, Spark: Refactor RewriteFileGroup planner to core [iceberg]

pvary commented on code in PR #11513: URL: https://github.com/apache/iceberg/pull/11513#discussion_r1867335924 ## core/src/main/java/org/apache/iceberg/actions/FileRewritePlan.java: ## @@ -0,0 +1,78 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or mor

Re: [I] Build Failure When Using JDK 1.8 with Iceberg 1.4.x [iceberg]

JackeeH commented on issue #11689: URL: https://github.com/apache/iceberg/issues/11689#issuecomment-2513971823 @nastra Yes, I can confirm that I did successfully build version 1.4.x by using the command `./gradlew build -x test -x integrationTest` after applying this patch. -- This i

Re: [I] Iceberg Kafka-Connect runtime not published as part of 1.7.0 release? [iceberg]

bryanck commented on issue #11685: URL: https://github.com/apache/iceberg/issues/11685#issuecomment-2514134826 I was going to look into the distribution of the connector for 1.8 (see https://github.com/apache/iceberg/issues/10745). I'll see if we can get something into the 1.7.1 release.

Re: [PR] Azure: Support vended credentials refresh in ADLSFileIO. [iceberg]

nastra commented on code in PR #11577: URL: https://github.com/apache/iceberg/pull/11577#discussion_r1867447807 ## azure/src/main/java/org/apache/iceberg/azure/AzureProperties.java: ## @@ -25,24 +25,40 @@ import java.util.Collections; import java.util.Map; import java.util.Op

Re: [PR] Azure: Support vended credentials refresh in ADLSFileIO. [iceberg]

nastra commented on code in PR #11577: URL: https://github.com/apache/iceberg/pull/11577#discussion_r1867451227 ## azure/src/main/java/org/apache/iceberg/azure/adlsv2/AzureSasCredentialRefresher.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache Software Foundation (ASF) u

Re: [PR] Azure: Support vended credentials refresh in ADLSFileIO. [iceberg]

nastra commented on code in PR #11577: URL: https://github.com/apache/iceberg/pull/11577#discussion_r1867454171 ## azure/src/main/java/org/apache/iceberg/azure/adlsv2/AzureSasCredentialRefresher.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache Software Foundation (ASF) u

Re: [PR] Azure: Support vended credentials refresh in ADLSFileIO. [iceberg]

nastra commented on code in PR #11577: URL: https://github.com/apache/iceberg/pull/11577#discussion_r1867457306 ## azure/src/main/java/org/apache/iceberg/azure/adlsv2/AzureSasCredentialRefresher.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache Software Foundation (ASF) u

Re: [PR] Azure: Support vended credentials refresh in ADLSFileIO. [iceberg]

nastra commented on code in PR #11577: URL: https://github.com/apache/iceberg/pull/11577#discussion_r1867457827 ## azure/src/test/java/org/apache/iceberg/azure/adlsv2/VendedAzureSasCredentialProviderTest.java: ## @@ -0,0 +1,323 @@ +/* + * Licensed to the Apache Software Foundati

Re: [PR] Encryption integration and test [iceberg]

nastra commented on PR #5544: URL: https://github.com/apache/iceberg/pull/5544#issuecomment-2514279859 @ggershinsky is this PR still active? Could you please rebase and fix the merge conflicts? I'll review this afterwards -- This is an automated message from the Apache Git Service. To res

Re: [PR] Core: Suppress exceptions in case of dropTableData [iceberg]

nastra commented on PR #9184: URL: https://github.com/apache/iceberg/pull/9184#issuecomment-2514287398 @nk1506 could you rebase the PR please and apply the review comments? I'll review it afterwards -- This is an automated message from the Apache Git Service. To respond to the message, pl

Re: [PR] Core: Generalize Util::blockLocations [iceberg]

nastra commented on code in PR #11053: URL: https://github.com/apache/iceberg/pull/11053#discussion_r1867552386 ## .palantir/revapi.yml: ## @@ -1100,6 +1100,12 @@ acceptedBreaks: - code: "java.class.removed" old: "enum org.apache.iceberg.BaseMetastoreTableOperations.

Re: [PR] Flink: Maintenance - RewriteDataFiles [iceberg]

pvary commented on code in PR #11497: URL: https://github.com/apache/iceberg/pull/11497#discussion_r1867807035 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/maintenance/api/RewriteDataFiles.java: ## @@ -0,0 +1,232 @@ +/* + * Licensed to the Apache Software Foundati

Re: [PR] Flink: Maintenance - RewriteDataFiles [iceberg]

pvary commented on code in PR #11497: URL: https://github.com/apache/iceberg/pull/11497#discussion_r1867809765 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/DataFileRewriteCommitter.java: ## @@ -0,0 +1,304 @@ +/* + * Licensed to the Apache Soft

[PR] docs: Add link to GitHub release notes [iceberg-python]

manuzhang opened a new pull request, #1394: URL: https://github.com/apache/iceberg-python/pull/1394 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscr

Re: [PR] Flink: Maintenance - RewriteDataFiles [iceberg]

pvary commented on code in PR #11497: URL: https://github.com/apache/iceberg/pull/11497#discussion_r1867817542 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/DataFileRewriteCommitter.java: ## @@ -0,0 +1,304 @@ +/* + * Licensed to the Apache Soft

Re: [PR] Flink: Maintenance - RewriteDataFiles [iceberg]

pvary commented on code in PR #11497: URL: https://github.com/apache/iceberg/pull/11497#discussion_r1867834564 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/DataFileRewriteCommitter.java: ## @@ -0,0 +1,304 @@ +/* + * Licensed to the Apache Soft

[I] Build Failure When Using JDK 1.8 with Iceberg 1.4.x [iceberg]

JackeeH opened a new issue, #11689: URL: https://github.com/apache/iceberg/issues/11689 ### Apache Iceberg version 1.4.3 ### Query engine None ### Please describe the bug 🐞 I am using OpenJDK 1.8 and trying to build Iceberg 1.4.x using the command `./gradle

Re: [I] Build Failure When Using JDK 1.8 with Iceberg 1.4.x [iceberg]

JackeeH commented on issue #11689: URL: https://github.com/apache/iceberg/issues/11689#issuecomment-2513885001 The stack picture is not very clear, you can see the picture below:  -- This is a

Re: [I] Iceberg Kafka-Connect runtime not published as part of 1.7.0 release? [iceberg]

bryanck commented on issue #11685: URL: https://github.com/apache/iceberg/issues/11685#issuecomment-2514235034 Sounds good, so for 1.7 @thjaeckle you will need to build it locally. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

Re: [I] Why shouldn't we return an `UnboundPartitionSpec` instead? [iceberg-rust]

c-thiel commented on issue #694: URL: https://github.com/apache/iceberg-rust/issues/694#issuecomment-2514191991 @Fokko didn't you try to remove the `buildUnchecked` in Java? Did you succeed? > Add a crate private buildUnchecked method. At least for `Lakekeeper` we would need `b

Re: [PR] Flink: Maintenance - RewriteDataFiles [iceberg]

pvary commented on code in PR #11497: URL: https://github.com/apache/iceberg/pull/11497#discussion_r1867869970 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/DataFileRewriteCommitter.java: ## @@ -0,0 +1,304 @@ +/* + * Licensed to the Apache Soft

[PR] Build: Don't run CI on unrelated changes [iceberg-python]

manuzhang opened a new pull request, #1395: URL: https://github.com/apache/iceberg-python/pull/1395 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscr

Re: [PR] Add basic CMake support for the iceberg library [iceberg-cpp]

wgtmac commented on code in PR #3: URL: https://github.com/apache/iceberg-cpp/pull/3#discussion_r1867875517 ## CMakeLists.txt: ## @@ -0,0 +1,63 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# dis

Re: [PR] Add basic CMake support for the iceberg library [iceberg-cpp]

wgtmac commented on code in PR #3: URL: https://github.com/apache/iceberg-cpp/pull/3#discussion_r1867879414 ## CMakeLists.txt: ## @@ -0,0 +1,63 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# dis

Re: [PR] Spark: Add view support to SparkSessionCatalog [iceberg]

danielcweeks commented on code in PR #11388: URL: https://github.com/apache/iceberg/pull/11388#discussion_r1868017844 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/SparkSessionCatalog.java: ## @@ -397,4 +412,155 @@ public UnboundFunction loadFunction(Identifier iden

Re: [PR] Spark: Add view support to SparkSessionCatalog [iceberg]

nastra commented on code in PR #11388: URL: https://github.com/apache/iceberg/pull/11388#discussion_r1868056254 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/SparkSessionCatalog.java: ## @@ -397,4 +412,155 @@ public UnboundFunction loadFunction(Identifier ident) th

Re: [PR] Flink: Maintenance - RewriteDataFiles [iceberg]

pvary commented on code in PR #11497: URL: https://github.com/apache/iceberg/pull/11497#discussion_r1868087488 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/DataFileRewriteCommitter.java: ## @@ -0,0 +1,304 @@ +/* + * Licensed to the Apache Soft

Re: [PR] Flink: Maintenance - RewriteDataFiles [iceberg]

pvary commented on code in PR #11497: URL: https://github.com/apache/iceberg/pull/11497#discussion_r1868086690 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/DataFileRewriteCommitter.java: ## @@ -0,0 +1,304 @@ +/* + * Licensed to the Apache Soft

Re: [PR] Flink: Maintenance - RewriteDataFiles [iceberg]

pvary commented on code in PR #11497: URL: https://github.com/apache/iceberg/pull/11497#discussion_r1868096866 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/DataFileRewriteCommitter.java: ## @@ -0,0 +1,304 @@ +/* + * Licensed to the Apache Soft

Re: [PR] Flink: Maintenance - RewriteDataFiles [iceberg]

pvary commented on code in PR #11497: URL: https://github.com/apache/iceberg/pull/11497#discussion_r1868095868 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/DataFileRewriteCommitter.java: ## @@ -0,0 +1,304 @@ +/* + * Licensed to the Apache Soft

Re: [PR] Flink: Maintenance - RewriteDataFiles [iceberg]

pvary commented on code in PR #11497: URL: https://github.com/apache/iceberg/pull/11497#discussion_r1868096866 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/DataFileRewriteCommitter.java: ## @@ -0,0 +1,304 @@ +/* + * Licensed to the Apache Soft

Re: [PR] Flink: Maintenance - RewriteDataFiles [iceberg]

pvary commented on code in PR #11497: URL: https://github.com/apache/iceberg/pull/11497#discussion_r1868101342 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/maintenance/api/RewriteDataFiles.java: ## @@ -0,0 +1,232 @@ +/* + * Licensed to the Apache Software Foundati

Re: [PR] Flink: Maintenance - RewriteDataFiles [iceberg]

pvary commented on code in PR #11497: URL: https://github.com/apache/iceberg/pull/11497#discussion_r1868100304 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/TaskResultAggregator.java: ## @@ -0,0 +1,154 @@ +/* + * Licensed to the Apache Software

Re: [PR] Flink: Maintenance - RewriteDataFiles [iceberg]

pvary commented on code in PR #11497: URL: https://github.com/apache/iceberg/pull/11497#discussion_r1868091732 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/DataFileRewriteCommitter.java: ## @@ -0,0 +1,304 @@ +/* + * Licensed to the Apache Soft

Re: [PR] Flink: Maintenance - RewriteDataFiles [iceberg]

pvary commented on code in PR #11497: URL: https://github.com/apache/iceberg/pull/11497#discussion_r1868098943 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/DataFileRewriteExecutor.java: ## @@ -0,0 +1,257 @@ +/* + * Licensed to the Apache Softw

Re: [PR] Flink: Maintenance - RewriteDataFiles [iceberg]

pvary commented on code in PR #11497: URL: https://github.com/apache/iceberg/pull/11497#discussion_r1868103159 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/maintenance/api/RewriteDataFiles.java: ## @@ -0,0 +1,232 @@ +/* + * Licensed to the Apache Software Foundati

Re: [PR] Flink: Maintenance - RewriteDataFiles [iceberg]

pvary commented on code in PR #11497: URL: https://github.com/apache/iceberg/pull/11497#discussion_r1868109673 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/maintenance/api/RewriteDataFiles.java: ## @@ -0,0 +1,232 @@ +/* + * Licensed to the Apache Software Foundati

Re: [PR] Flink: Maintenance - RewriteDataFiles [iceberg]

pvary commented on code in PR #11497: URL: https://github.com/apache/iceberg/pull/11497#discussion_r1868114928 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/DataFileRewritePlanner.java: ## @@ -0,0 +1,228 @@ +/* + * Licensed to the Apache Softwa

Re: [PR] Hive: Optimize tableExists API in hive catalog [iceberg]

dramaticlly commented on code in PR #11597: URL: https://github.com/apache/iceberg/pull/11597#discussion_r1868397742 ## hive-metastore/src/test/java/org/apache/iceberg/hive/HiveTableTest.java: ## @@ -388,6 +388,41 @@ public void testHiveTableAndIcebergTableWithSameName(TableTyp

Re: [PR] Hive: Optimize tableExists API in hive catalog [iceberg]

szehon-ho commented on code in PR #11597: URL: https://github.com/apache/iceberg/pull/11597#discussion_r1868396253 ## hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java: ## @@ -412,6 +412,34 @@ private void validateTableIsIcebergTableOrView( } } + @

Re: [PR] Hive: Optimize tableExists API in hive catalog [iceberg]

dramaticlly commented on code in PR #11597: URL: https://github.com/apache/iceberg/pull/11597#discussion_r1868398567 ## hive-metastore/src/test/java/org/apache/iceberg/hive/HiveTableTest.java: ## @@ -388,6 +388,41 @@ public void testHiveTableAndIcebergTableWithSameName(TableTyp

Re: [PR] Hive: Optimize tableExists API in hive catalog [iceberg]

dramaticlly commented on code in PR #11597: URL: https://github.com/apache/iceberg/pull/11597#discussion_r1868399294 ## hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java: ## @@ -412,6 +412,34 @@ private void validateTableIsIcebergTableOrView( } } +

Re: [PR] Materialized View Spec [iceberg]

szehon-ho commented on code in PR #11041: URL: https://github.com/apache/iceberg/pull/11041#discussion_r1868399418 ## format/view-spec.md: ## @@ -42,12 +42,28 @@ An atomic swap of one view metadata file for another provides the basis for maki Writers create view metadata fil

Re: [PR] Flink: Maintenance - RewriteDataFiles [iceberg]

stevenzwu commented on code in PR #11497: URL: https://github.com/apache/iceberg/pull/11497#discussion_r1868466879 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/maintenance/api/RewriteDataFiles.java: ## @@ -0,0 +1,232 @@ +/* + * Licensed to the Apache Software Foun

Re: [PR] REST: AuthManager API [iceberg]

danielcweeks commented on code in PR #10753: URL: https://github.com/apache/iceberg/pull/10753#discussion_r1868471895 ## core/src/main/java/org/apache/iceberg/rest/auth/AuthConfig.java: ## @@ -47,7 +47,7 @@ default String scope() { return OAuth2Properties.CATALOG_SCOPE;

Re: [PR] Flink: Maintenance - RewriteDataFiles [iceberg]

stevenzwu commented on code in PR #11497: URL: https://github.com/apache/iceberg/pull/11497#discussion_r1868471586 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/DataFileRewriteExecutor.java: ## @@ -0,0 +1,257 @@ +/* + * Licensed to the Apache S

Re: [PR] REST: AuthManager API [iceberg]

danielcweeks commented on code in PR #10753: URL: https://github.com/apache/iceberg/pull/10753#discussion_r1868471393 ## aws/src/main/java/org/apache/iceberg/aws/s3/signer/S3V4RestSignerClient.java: ## @@ -81,14 +71,12 @@ public abstract class S3V4RestSignerClient private sta

Re: [PR] Kafka Connect: Add mechanisms for routing records by topic name [iceberg]

bryanck commented on PR #11623: URL: https://github.com/apache/iceberg/pull/11623#issuecomment-2515748508 I feel your solution is reasonable, though I'm trying to reconcile this with the need for a more flexible, pluggable way to route records. For example, one case we had was to support dy

  1   2   >