[PR] BUG: Add assert spec id to requirements in spec update [iceberg-python]

2025-09-20 Thread via GitHub
gabeiglio opened a new pull request, #2489: URL: https://github.com/apache/iceberg-python/pull/2489 # Rationale for this change When doing a commit that evolves the partition spec we are missing a requirement for asserting that the base spec id has not changed.

Re: [I] cmake hang [iceberg-cpp]

2025-09-20 Thread via GitHub
Tangruilin commented on issue #240: URL: https://github.com/apache/iceberg-cpp/issues/240#issuecomment-3315538198 > I download CRoaring and build with the following commands: > > ``` > cmake -S . -B build -DENABLE_ROARING_MICROBENCHMARKS=ON -DHAVE_PTHREAD_AFFINITY=OFF > cmake --

[PR] Azure: add support to specify token credential provider [iceberg]

2025-09-20 Thread via GitHub
munendrasn opened a new pull request, #14136: URL: https://github.com/apache/iceberg/pull/14136 Similar to AWS, `adls.token-credential-provider` can be used to specify the `TokenCredential` provider. This is useful where user can configure custom token credential or in-built TokenCredent

Re: [PR] Build: Bump org.assertj:assertj-core from 3.27.4 to 3.27.5 [iceberg]

2025-09-20 Thread via GitHub
singhpk234 merged PR #14130: URL: https://github.com/apache/iceberg/pull/14130 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [PR] Build: Bump mkdocs-material from 9.6.19 to 9.6.20 [iceberg]

2025-09-20 Thread via GitHub
singhpk234 merged PR #14131: URL: https://github.com/apache/iceberg/pull/14131 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [PR] Build: Bump software.amazon.awssdk:bom from 2.33.9 to 2.34.0 [iceberg]

2025-09-20 Thread via GitHub
singhpk234 merged PR #14133: URL: https://github.com/apache/iceberg/pull/14133 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [I] cmake hang [iceberg-cpp]

2025-09-20 Thread via GitHub
zhjwpku commented on issue #240: URL: https://github.com/apache/iceberg-cpp/issues/240#issuecomment-3315499428 I download CRoaring and build with the following commands: ``` cmake -S . -B build -DENABLE_ROARING_MICROBENCHMARKS=ON -DHAVE_PTHREAD_AFFINITY=OFF cmake --build build

Re: [PR] Object Cache: caches parsed Manifests and ManifestLists for performance [iceberg-rust]

2025-09-20 Thread via GitHub
dentiny commented on code in PR #512: URL: https://github.com/apache/iceberg-rust/pull/512#discussion_r2365940293 ## crates/iceberg/src/scan.rs: ## @@ -477,24 +478,22 @@ impl ManifestFileContext { /// streaming its constituent [`ManifestEntries`] to the channel provided in

Re: [PR] Object Cache: caches parsed Manifests and ManifestLists for performance [iceberg-rust]

2025-09-20 Thread via GitHub
dentiny commented on code in PR #512: URL: https://github.com/apache/iceberg-rust/pull/512#discussion_r2365940293 ## crates/iceberg/src/scan.rs: ## @@ -477,24 +478,22 @@ impl ManifestFileContext { /// streaming its constituent [`ManifestEntries`] to the channel provided in

[PR] chore: Expose object cache [iceberg-rust]

2025-09-20 Thread via GitHub
dentiny opened a new pull request, #1698: URL: https://github.com/apache/iceberg-rust/pull/1698 ## What changes are included in this PR? object storage cache is introduced at https://github.com/apache/iceberg-rust/pull/512, which seems to be used for (internal) scan only for now.

Re: [PR] Support for TIME, TIMESTAMPNTZ_NANO, UUID types in Inclusive Metrics Evaluator [iceberg]

2025-09-20 Thread via GitHub
aihuaxu commented on code in PR #13195: URL: https://github.com/apache/iceberg/pull/13195#discussion_r236593 ## core/src/test/java/org/apache/iceberg/expressions/TestInclusiveMetricsEvaluatorWithExtract.java: ## @@ -683,4 +690,249 @@ public void testIntegerNotIn() {

Re: [I] add an option to close the catalog [iceberg-python]

2025-09-20 Thread via GitHub
djouallah commented on issue #2346: URL: https://github.com/apache/iceberg-python/issues/2346#issuecomment-3315486690 @kris-gaudel thank you, it is so convenient :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[PR] add “Writing Data” guide (append / overwrite / delete / overwrite_filter / upsert) [iceberg]

2025-09-20 Thread via GitHub
Username46786 opened a new pull request, #14135: URL: https://github.com/apache/iceberg/pull/14135 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscri

[PR] Build: Bump software.amazon.awssdk:bom from 2.33.9 to 2.34.0 [iceberg]

2025-09-20 Thread via GitHub
dependabot[bot] opened a new pull request, #14133: URL: https://github.com/apache/iceberg/pull/14133 Bumps software.amazon.awssdk:bom from 2.33.9 to 2.34.0. [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=softwa

[PR] Build: Bump com.google.errorprone:error_prone_annotations from 2.41.0 to 2.42.0 [iceberg]

2025-09-20 Thread via GitHub
dependabot[bot] opened a new pull request, #14132: URL: https://github.com/apache/iceberg/pull/14132 Bumps [com.google.errorprone:error_prone_annotations](https://github.com/google/error-prone) from 2.41.0 to 2.42.0. Release notes Sourced from https://github.com/google/error-prone

[PR] Build: Bump com.google.cloud:libraries-bom from 26.67.0 to 26.68.0 [iceberg]

2025-09-20 Thread via GitHub
dependabot[bot] opened a new pull request, #14134: URL: https://github.com/apache/iceberg/pull/14134 Bumps [com.google.cloud:libraries-bom](https://github.com/googleapis/java-cloud-bom) from 26.67.0 to 26.68.0. Release notes Sourced from https://github.com/googleapis/java-cloud-bo

[PR] Build: Bump org.assertj:assertj-core from 3.27.4 to 3.27.5 [iceberg]

2025-09-20 Thread via GitHub
dependabot[bot] opened a new pull request, #14130: URL: https://github.com/apache/iceberg/pull/14130 Bumps [org.assertj:assertj-core](https://github.com/assertj/assertj) from 3.27.4 to 3.27.5. Release notes Sourced from https://github.com/assertj/assertj/releases";>org.assertj:asse

Re: [PR] Build: Bump nessie from 0.104.5 to 0.105.0 [iceberg]

2025-09-20 Thread via GitHub
dependabot[bot] commented on PR #14007: URL: https://github.com/apache/iceberg/pull/14007#issuecomment-3315482102 Superseded by #14129. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

[PR] Build: Bump mkdocs-material from 9.6.19 to 9.6.20 [iceberg]

2025-09-20 Thread via GitHub
dependabot[bot] opened a new pull request, #14131: URL: https://github.com/apache/iceberg/pull/14131 Bumps [mkdocs-material](https://github.com/squidfunk/mkdocs-material) from 9.6.19 to 9.6.20. Release notes Sourced from https://github.com/squidfunk/mkdocs-material/releases";>mkdoc

Re: [PR] Build: Bump nessie from 0.104.5 to 0.105.0 [iceberg]

2025-09-20 Thread via GitHub
dependabot[bot] closed pull request #14007: Build: Bump nessie from 0.104.5 to 0.105.0 URL: https://github.com/apache/iceberg/pull/14007 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[PR] Build: Bump nessie from 0.104.5 to 0.105.2 [iceberg]

2025-09-20 Thread via GitHub
dependabot[bot] opened a new pull request, #14129: URL: https://github.com/apache/iceberg/pull/14129 Bumps `nessie` from 0.104.5 to 0.105.2. Updates `org.projectnessie.nessie:nessie-client` from 0.104.5 to 0.105.2 Release notes Sourced from https://github.com/projectnessie/nessie

[PR] Build: Bump guava from 33.4.8-jre to 33.5.0-jre [iceberg]

2025-09-20 Thread via GitHub
dependabot[bot] opened a new pull request, #14128: URL: https://github.com/apache/iceberg/pull/14128 Bumps `guava` from 33.4.8-jre to 33.5.0-jre. Updates `com.google.guava:guava` from 33.4.8-jre to 33.5.0-jre Release notes Sourced from https://github.com/google/guava/releases";>c

[PR] fix: disable CRoaring tests and microbenchmarks [iceberg-cpp]

2025-09-20 Thread via GitHub
zhjwpku opened a new pull request, #241: URL: https://github.com/apache/iceberg-cpp/pull/241 Previously, we mistakenly used the outdated option BUILD_TESTING for CRoaring, change it to ENABLE_ROARING_TESTS, and also disable ENABLE_ROARING_MICROBENCHMARKS. -- This is an automated message

Re: [PR] feat: add Metadata method to Reader and its implementations [iceberg-cpp]

2025-09-20 Thread via GitHub
wgtmac commented on code in PR #235: URL: https://github.com/apache/iceberg-cpp/pull/235#discussion_r2361833386 ## src/iceberg/avro/avro_reader.cc: ## @@ -173,6 +173,24 @@ class AvroReader::Impl { return arrow_schema; } + Result> Metadata() { +if (reader_ == nullp

Re: [PR] refactor(writer): Refactor writers for the future partitioning writers [iceberg-rust]

2025-09-20 Thread via GitHub
CTTY commented on code in PR #1657: URL: https://github.com/apache/iceberg-rust/pull/1657#discussion_r2365926622 ## crates/iceberg/src/writer/file_writer/rolling_writer.rs: ## @@ -15,67 +15,138 @@ // specific language governing permissions and limitations // under the License.

Re: [PR] refactor(writer): Refactor writers for the future partitioning writers [iceberg-rust]

2025-09-20 Thread via GitHub
CTTY commented on code in PR #1657: URL: https://github.com/apache/iceberg-rust/pull/1657#discussion_r2365926310 ## crates/iceberg/src/writer/partitioning/clustered.rs: ## @@ -0,0 +1,251 @@ +// // Licensed to the Apache Software Foundation (ASF) under one +// // or more contribu

Re: [PR] Draft: Added null and {} support for default values within SchemaParser [iceberg]

2025-09-20 Thread via GitHub
Verest commented on code in PR #14052: URL: https://github.com/apache/iceberg/pull/14052#discussion_r2342260052 ## api/src/test/java/org/apache/iceberg/TestSchema.java: ## @@ -213,7 +213,7 @@ public void testSupportedInitialDefault(int formatVersion) { @ParameterizedTest @

Re: [PR] Spark 4.0: Refactor Spark procedures to consistently use ProcedureInput for parameter handling. [iceberg]

2025-09-20 Thread via GitHub
dramaticlly commented on code in PR #13913: URL: https://github.com/apache/iceberg/pull/13913#discussion_r2363850099 ## spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/procedures/ExpireSnapshotsProcedure.java: ## @@ -104,13 +118,14 @@ public ProcedureParameter[] paramete

Re: [PR] refactor(writer): Refactor writers for the future partitioning writers [iceberg-rust]

2025-09-20 Thread via GitHub
CTTY commented on code in PR #1657: URL: https://github.com/apache/iceberg-rust/pull/1657#discussion_r2334450044 ## crates/iceberg/src/writer/mod.rs: ## @@ -229,20 +229,30 @@ pub mod file_writer; use arrow_array::RecordBatch; use crate::Result; -use crate::spec::DataFile; +u

Re: [PR] Add tests for Schema Evolution on Catalogs [iceberg-python]

2025-09-20 Thread via GitHub
gabeiglio commented on code in PR #2478: URL: https://github.com/apache/iceberg-python/pull/2478#discussion_r2365902430 ## tests/integration/test_catalog.py: ## @@ -343,3 +345,67 @@ def test_update_namespace_properties(test_catalog: Catalog, database_name: str) else:

Re: [PR] Core, Data: File Format API interfaces [iceberg]

2025-09-20 Thread via GitHub
pvary commented on code in PR #12774: URL: https://github.com/apache/iceberg/pull/12774#discussion_r2353692309 ## core/src/main/java/org/apache/iceberg/data/ContentFileWriteBuilderImpl.java: ## @@ -0,0 +1,336 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

Re: [PR] API, Core: Reference IRC to throw 304-NotModifiedException [iceberg]

2025-09-20 Thread via GitHub
gaborkaszab commented on code in PR #14035: URL: https://github.com/apache/iceberg/pull/14035#discussion_r2337148163 ## api/src/main/java/org/apache/iceberg/exceptions/NotModifiedException.java: ## @@ -0,0 +1,29 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under o

Re: [PR] fix: Make TableMetadataV3ValidMinimal actually v3 [iceberg]

2025-09-20 Thread via GitHub
Fokko commented on code in PR #14061: URL: https://github.com/apache/iceberg/pull/14061#discussion_r2344814537 ## core/src/test/resources/TableMetadataV3ValidMinimal.json: ## @@ -1,10 +1,11 @@ { - "format-version": 2, + "format-version": 3, "table-uuid": "9c12d441-03fe-469

Re: [I] cmake hang [iceberg-cpp]

2025-09-20 Thread via GitHub
zhjwpku commented on issue #240: URL: https://github.com/apache/iceberg-cpp/issues/240#issuecomment-3315442912 > > You might need to set up http(s) proxy. It may hang when trying to download vendored dependencies. > > https://private-user-images.githubusercontent.com/23651

Re: [PR] Build: Bump hadoop from 3.4.1 to 3.4.2. [iceberg]

2025-09-20 Thread via GitHub
slfan1989 commented on PR #14125: URL: https://github.com/apache/iceberg/pull/14125#issuecomment-3315419166 > Thanks @slfan1989 for the PR! Thanks @Fokko for the review! @huaxingao @Fokko Thank you very much for helping review the code! -- This is an automated message from the Apach

Re: [PR] Core: Throw error for DELETE operations in incremental scan [iceberg]

2025-09-20 Thread via GitHub
github-actions[bot] commented on PR #13717: URL: https://github.com/apache/iceberg/pull/13717#issuecomment-3276947573 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

Re: [I] Metadata stop to be written after random period of time. Commit failed, will try again next cycle. [iceberg]

2025-09-20 Thread via GitHub
0xPT commented on issue #13593: URL: https://github.com/apache/iceberg/issues/13593#issuecomment-3276763238 If we aren't going to fix this can we point to a latest version that works atleast? This is a complete blocker for any production workload. -- This is an automated message from the

Re: [PR] [SPEC] New revision on schema evolution [iceberg]

2025-09-20 Thread via GitHub
nvartolomei commented on code in PR #14004: URL: https://github.com/apache/iceberg/pull/14004#discussion_r272794 ## format/spec.md: ## @@ -1861,6 +1861,18 @@ Java writes `-1` for "no current snapshot" with V1 and V2 tables and considers t Some implementations require tha

Re: [PR] Server-side planning models [iceberg-python]

2025-09-20 Thread via GitHub
Fokko commented on code in PR #2435: URL: https://github.com/apache/iceberg-python/pull/2435#discussion_r2341950160 ## pyiceberg/catalog/rest/expression.py: ## @@ -0,0 +1,130 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreemen

[PR] Draft: Added null and {} support for default values within SchemaParser [iceberg]

2025-09-20 Thread via GitHub
Verest opened a new pull request, #14052: URL: https://github.com/apache/iceberg/pull/14052 Description WIP -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

Re: [I] RewriteTablePathUtil doesn't work with v3 deletes [iceberg]

2025-09-20 Thread via GitHub
enesyesil commented on issue #13671: URL: https://github.com/apache/iceberg/issues/13671#issuecomment-3300221199 Hey, I’d like to work on this issue if it’s still available? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] BigQuery: Add table validity check for BigQueryMetastoreCatalog [iceberg]

2025-09-20 Thread via GitHub
thomaschow commented on code in PR #14113: URL: https://github.com/apache/iceberg/pull/14113#discussion_r2364130634 ## bigquery/src/test/java/org/apache/iceberg/gcp/bigquery/TestBigQueryCatalog.java: ## @@ -171,4 +177,30 @@ public void renameTableNamespaceMissing() { public v

Re: [I] Add example for document [iceberg-rust]

2025-09-20 Thread via GitHub
github-actions[bot] commented on issue #986: URL: https://github.com/apache/iceberg-rust/issues/986#issuecomment-3287224315 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occu

Re: [PR] Flink: Dynamic Sink: Add error stream for graceful error handling [iceberg]

2025-09-20 Thread via GitHub
mxm commented on code in PR #13963: URL: https://github.com/apache/iceberg/pull/13963#discussion_r2348236686 ## flink/v2.0/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/DynamicIcebergSink.java: ## @@ -374,32 +416,86 @@ public DataStreamSink append() {

[PR] Build: Bump pyiceberg-core from 0.5.1 to 0.6.0 [iceberg-python]

2025-09-20 Thread via GitHub
dependabot[bot] opened a new pull request, #2445: URL: https://github.com/apache/iceberg-python/pull/2445 Bumps [pyiceberg-core](https://rust.iceberg.apache.org) from 0.5.1 to 0.6.0. [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_sc

Re: [PR] Fix variant type filtering in ParquetMetricsRowGroupFilter [iceberg]

2025-09-20 Thread via GitHub
amogh-jahagirdar commented on PR #14081: URL: https://github.com/apache/iceberg/pull/14081#issuecomment-3292305410 cc @aihuaxu -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

Re: [PR] Core,Api: Add overwrite option when register external table to catalog [iceberg]

2025-09-20 Thread via GitHub
dramaticlly commented on code in PR #12228: URL: https://github.com/apache/iceberg/pull/12228#discussion_r2364427744 ## core/src/main/java/org/apache/iceberg/BaseMetastoreCatalog.java: ## @@ -71,23 +70,34 @@ public Table loadTable(TableIdentifier identifier) { } @Overrid

[PR] Flink: Preserve row lineage in RewriteDataFiles on compaction [iceberg]

2025-09-20 Thread via GitHub
Guosmilesmile opened a new pull request, #14127: URL: https://github.com/apache/iceberg/pull/14127 This PR is split into two parts to support preserving lineage information in Flink RewriteDataFiles. It only supports RewriteDataFiles for streaming compaction. 1. Adds readers in Flink

Re: [PR] AWS: Add support to run all integration tests when S3 Analytics Accelerator is enabled [iceberg]

2025-09-20 Thread via GitHub
github-actions[bot] closed pull request #13347: AWS: Add support to run all integration tests when S3 Analytics Accelerator is enabled URL: https://github.com/apache/iceberg/pull/13347 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

Re: [PR] Parquet, Core: Allows Internal Parquet Readers to use Custom Types [iceberg]

2025-09-20 Thread via GitHub
RussellSpitzer commented on code in PR #14040: URL: https://github.com/apache/iceberg/pull/14040#discussion_r2334907095 ## parquet/src/main/java/org/apache/iceberg/data/parquet/BaseParquetReaders.java: ## @@ -75,6 +78,16 @@ protected ParquetValueReader createReader( } }

Re: [I] Support deletion in Apache Flink [iceberg]

2025-09-20 Thread via GitHub
pvary commented on issue #8718: URL: https://github.com/apache/iceberg/issues/8718#issuecomment-3298964452 @rlokugamage: Here is where we state, that update is possible: https://iceberg.apache.org/docs/nightly/flink-writes/#insert-into The documentation doesn't state how it is done, b

Re: [I] infra: improve MSRV check [iceberg-rust]

2025-09-20 Thread via GitHub
liurenjie1024 commented on issue #1667: URL: https://github.com/apache/iceberg-rust/issues/1667#issuecomment-3291400959 +1, this improves developer experience. I love everything locally reproducable. -- This is an automated message from the Apache Git Service. To respond to the message, p

Re: [PR] Parquet: Support `In` predicate pushdown for ParquetFilters [iceberg]

2025-09-20 Thread via GitHub
jshmchenxi commented on code in PR #14041: URL: https://github.com/apache/iceberg/pull/14041#discussion_r2348058389 ## parquet/src/main/java/org/apache/iceberg/parquet/ParquetFilters.java: ## @@ -175,7 +204,7 @@ public FilterPredicate predicate(UnboundPredicate pred) { @S

Re: [PR] API, Core: Reference IRC to throw 304-NotModifiedException [iceberg]

2025-09-20 Thread via GitHub
gaborkaszab commented on code in PR #14035: URL: https://github.com/apache/iceberg/pull/14035#discussion_r2340178014 ## core/src/test/java/org/apache/iceberg/rest/RESTCatalogAdapter.java: ## @@ -425,10 +429,23 @@ public T handleRequest( case LOAD_TABLE: { -

Re: [I] Is there a Slack or Discord community for Apache Iceberg contributors? [iceberg]

2025-09-20 Thread via GitHub
ebyhr commented on issue #14028: URL: https://github.com/apache/iceberg/issues/14028#issuecomment-3270317216 Hi, you can join the Iceberg Slack via https://iceberg.apache.org/community/#slack -- This is an automated message from the Apache Git Service. To respond to the message, please lo

Re: [I] Add files to add existing Parquet files to a table [iceberg-rust]

2025-09-20 Thread via GitHub
github-actions[bot] commented on issue #932: URL: https://github.com/apache/iceberg-rust/issues/932#issuecomment-3287224587 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occu

Re: [I] sort order(z-order) Support when calling rewrite file action from FLINK engine [iceberg]

2025-09-20 Thread via GitHub
pvary commented on issue #13928: URL: https://github.com/apache/iceberg/issues/13928#issuecomment-3285161414 I would recommend to use the implement this feature into the new TableMaintenance: https://iceberg.apache.org/docs/nightly/flink-maintenance/#flink-table-maintenance-streamingmode

Re: [PR] API, Core: Introduce classes for content stats [iceberg]

2025-09-20 Thread via GitHub
nastra commented on code in PR #13933: URL: https://github.com/apache/iceberg/pull/13933#discussion_r2349170460 ## api/src/main/java/org/apache/iceberg/stats/FieldStats.java: ## @@ -0,0 +1,40 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contr

Re: [PR] Closing the ParquetWriter in the correct spot [iceberg]

2025-09-20 Thread via GitHub
github-actions[bot] commented on PR #13565: URL: https://github.com/apache/iceberg/pull/13565#issuecomment-3314270526 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

[PR] fix: fix release tarball name [iceberg-cpp]

2025-09-20 Thread via GitHub
wgtmac opened a new pull request, #221: URL: https://github.com/apache/iceberg-cpp/pull/221 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-m

Re: [PR] Server-side planning models [iceberg-python]

2025-09-20 Thread via GitHub
Fokko commented on PR #2435: URL: https://github.com/apache/iceberg-python/pull/2435#issuecomment-3282111825 Thanks for adding these @rambleraptor. I think it would be nice to: - See if we can combine already existing classes like Transform to avoid duplication. - Add some tests to

Re: [PR] Core: Avoid removing reused metadata file when registerTable fails [iceberg]

2025-09-20 Thread via GitHub
RussellSpitzer commented on PR #14083: URL: https://github.com/apache/iceberg/pull/14083#issuecomment-3299730344 I think we had decided we were going to just completely remove the "commit based" register tables to try to avoid this issue. @dramaticlly were you working on that? Did I forget

Re: [PR] API, Core: Reference IRC to throw 304-NotModifiedException [iceberg]

2025-09-20 Thread via GitHub
gaborkaszab commented on code in PR #14035: URL: https://github.com/apache/iceberg/pull/14035#discussion_r2337040552 ## api/src/main/java/org/apache/iceberg/exceptions/NotModifiedException.java: ## @@ -0,0 +1,29 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under o

[PR] docs: Fix Podman and Orbstack pages not found [iceberg-rust]

2025-09-20 Thread via GitHub
manuzhang opened a new pull request, #1678: URL: https://github.com/apache/iceberg-rust/pull/1678 The underlined links currently point to 404 page. This PR fix them by moving corresponding docs from `docs/contributing` to `website/src/reference`. https://github.com/user-attachments/asset

Re: [PR] Core: Extended header support for RESTClient implementations [iceberg]

2025-09-20 Thread via GitHub
pvary merged PR #12194: URL: https://github.com/apache/iceberg/pull/12194 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] Translate truncate(col) == value to startsWith(value) [iceberg]

2025-09-20 Thread via GitHub
github-actions[bot] commented on PR #13805: URL: https://github.com/apache/iceberg/pull/13805#issuecomment-3288998084 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

[PR] feat(catalog): impl builder for SqlCatalog [iceberg-rust]

2025-09-20 Thread via GitHub
335g opened a new pull request, #1666: URL: https://github.com/apache/iceberg-rust/pull/1666 ## Which issue does this PR close? - Closes #1626 ## What changes are included in this PR? - Introduced a new `SqlCatalogBuilder` implementation for `iceberg::Cat

Re: [I] Spark Executors erroring out with Exit Code : 134, after running compaction [iceberg]

2025-09-20 Thread via GitHub
github-actions[bot] commented on issue #12588: URL: https://github.com/apache/iceberg/issues/12588#issuecomment-3300745251 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [I] Can we trim the storage scheme after parsing? [iceberg-rust]

2025-09-20 Thread via GitHub
Xuanwo commented on issue #1411: URL: https://github.com/apache/iceberg-rust/issues/1411#issuecomment-3287049337 ## 设计不一致性分析和架构简化建议 经过深入分析当前代码,我发现了一个重要的设计不一致性,它实际上支持了这个issue的原始建议。 ### 当前设计不一致性 **问题描述:** PR #1417(commit d1490372,2025年6月9日)创造了一个设计矛盾: 1. **验证逻辑(line 3

Re: [PR] Build: Bump requests from 2.32.4 to 2.32.5 [iceberg-python]

2025-09-20 Thread via GitHub
Fokko merged PR #2441: URL: https://github.com/apache/iceberg-python/pull/2441 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [I] Delegate `delete` to JUnit [iceberg]

2025-09-20 Thread via GitHub
yelianevich commented on issue #13506: URL: https://github.com/apache/iceberg/issues/13506#issuecomment-3311001638 I created a PR to clean up Spark module for all versions, it's ready for a review: https://github.com/apache/iceberg/pull/14095. -- This is an automated message from the Apac

Re: [PR] Fix variant type filtering in ParquetMetricsRowGroupFilter [iceberg]

2025-09-20 Thread via GitHub
RussellSpitzer commented on code in PR #14081: URL: https://github.com/apache/iceberg/pull/14081#discussion_r2349231348 ## parquet/src/main/java/org/apache/iceberg/parquet/ParquetMetricsRowGroupFilter.java: ## @@ -157,8 +157,10 @@ public Boolean notNull(BoundReference ref) {

Re: [PR] Build: Bump pyspark from 3.5.6 to 4.0.0 [iceberg-python]

2025-09-20 Thread via GitHub
dependabot[bot] closed pull request #2102: Build: Bump pyspark from 3.5.6 to 4.0.0 URL: https://github.com/apache/iceberg-python/pull/2102 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

Re: [PR] Add tests for Partition Spec Evolution [iceberg-python]

2025-09-20 Thread via GitHub
gabeiglio commented on code in PR #2479: URL: https://github.com/apache/iceberg-python/pull/2479#discussion_r2365885454 ## tests/integration/test_catalog.py: ## @@ -343,3 +350,64 @@ def test_update_namespace_properties(test_catalog: Catalog, database_name: str) else:

Re: [PR] Core: Allow reading metadata table when scanning table with dropped partition source field [iceberg]

2025-09-20 Thread via GitHub
gabeiglio commented on PR #14089: URL: https://github.com/apache/iceberg/pull/14089#issuecomment-3301519885 @huaxingao @Fokko thanks for the review! Added a comment 👍🏼 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

Re: [I] Consolidate FileIO [iceberg-python]

2025-09-20 Thread via GitHub
github-actions[bot] closed issue #310: Consolidate FileIO URL: https://github.com/apache/iceberg-python/issues/310 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscrib

Re: [PR] Spark: Support Trigger AvailableNow in SS [iceberg]

2025-09-20 Thread via GitHub
singhpk234 commented on PR #13824: URL: https://github.com/apache/iceberg/pull/13824#issuecomment-3268002231 Thank you @alexprosak ! Looking forward to Pr's ! Thank you @huaxingao for the review -- This is an automated message from the Apache Git Service. To respond to the message, plea

Re: [PR] Flink: Fix flaky tests for Iceberg sink [iceberg]

2025-09-20 Thread via GitHub
pvary commented on PR #14044: URL: https://github.com/apache/iceberg/pull/14044#issuecomment-3279448727 Merged to main. Thanks for the fix @mxm and @Guosmilesmile for the review! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

Re: [I] Credential Leak when using Remote Signing with multiple Catalogs [iceberg]

2025-09-20 Thread via GitHub
snazy commented on issue #14100: URL: https://github.com/apache/iceberg/issues/14100#issuecomment-3307011011 > Something we do in Polaris is a "warning/stop" in the GitHub Issue template to "explain" what should be done in case of security vulnerabilities: https://github.com/apache/polaris/

Re: [I] Tracking issues of Iceberg Rust 0.7 Release [iceberg-rust]

2025-09-20 Thread via GitHub
Xuanwo commented on issue #1631: URL: https://github.com/apache/iceberg-rust/issues/1631#issuecomment-3301041876 > Hi @Xuanwo , I'd love to! I can start working on this tomorrow Appreciated, thank you for taking this role! -- This is an automated message from the Apache Git Service.

Re: [PR] Core: Avoid removing reused metadata file when registerTable fails [iceberg]

2025-09-20 Thread via GitHub
hantangwangd commented on PR #14083: URL: https://github.com/apache/iceberg/pull/14083#issuecomment-3301663207 Thanks @RussellSpitzer for your detailed explanation. I completely understand your preference for the first solution as it seems to be a better long-term solution. I also agree wit

Re: [PR] feat: Add support for Meson build system [iceberg-cpp]

2025-09-20 Thread via GitHub
WillAyd commented on PR #233: URL: https://github.com/apache/iceberg-cpp/pull/233#issuecomment-3303637490 I see that Roaring was just integrated into main. It looks like the Meson wrapdb entry for that is out of date, so I'll submit a new one upstream and rebase here -- This is an automa

Re: [PR] Flink: Dynamic Sink: Add error stream for graceful error handling [iceberg]

2025-09-20 Thread via GitHub
pvary commented on code in PR #13963: URL: https://github.com/apache/iceberg/pull/13963#discussion_r2343900993 ## flink/v2.0/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/DynamicIcebergSink.java: ## @@ -374,32 +416,86 @@ public DataStreamSink append() {

Re: [PR] Core: HadoopFileIO to support bulk delete through the Hadoop Filesystem APIs [iceberg]

2025-09-20 Thread via GitHub
github-actions[bot] commented on PR #10233: URL: https://github.com/apache/iceberg/pull/10233#issuecomment-3276947083 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] Fix variant type filtering in ParquetMetricsRowGroupFilter [iceberg]

2025-09-20 Thread via GitHub
nastra commented on code in PR #14081: URL: https://github.com/apache/iceberg/pull/14081#discussion_r2364011229 ## spark/v4.0/spark/src/test/java/org/apache/iceberg/spark/SparkTestHelperBase.java: ## @@ -79,6 +80,12 @@ protected void assertEquals(String context, Object[] expect

Re: [PR] Core: Check redundant transforms when updating table partition spec [iceberg]

2025-09-20 Thread via GitHub
hantangwangd closed pull request #13885: Core: Check redundant transforms when updating table partition spec URL: https://github.com/apache/iceberg/pull/13885 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] chore: add release script and github workflow [iceberg-cpp]

2025-09-20 Thread via GitHub
HeartLinked commented on PR #193: URL: https://github.com/apache/iceberg-cpp/pull/193#issuecomment-3269099189 > Thanks @HeartLinked for the manual testing! I just checked related Github actions https://github.com/HeartLinked/iceberg-cpp/actions/runs/17546168928 and it looks good. > >

Re: [I] Delete a row (primary key ID) from iceberg table [iceberg]

2025-09-20 Thread via GitHub
github-actions[bot] closed issue #12490: Delete a row (primary key ID) from iceberg table URL: https://github.com/apache/iceberg/issues/12490 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

Re: [I] Delete a row (primary key ID) from iceberg table [iceberg]

2025-09-20 Thread via GitHub
github-actions[bot] commented on issue #12490: URL: https://github.com/apache/iceberg/issues/12490#issuecomment-3315386874 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache

Re: [PR] AWS: Make the number of retries for metadata location refresh configurable [iceberg]

2025-09-20 Thread via GitHub
github-actions[bot] commented on PR #13594: URL: https://github.com/apache/iceberg/pull/13594#issuecomment-3315386942 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] AWS: Support similar S3 Sync Client configurations for S3 Async Clients [iceberg]

2025-09-20 Thread via GitHub
github-actions[bot] commented on PR #13387: URL: https://github.com/apache/iceberg/pull/13387#issuecomment-3315386922 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [I] Deleting metadata(expire_snapshots doesn't help...) [iceberg]

2025-09-20 Thread via GitHub
github-actions[bot] commented on issue #11169: URL: https://github.com/apache/iceberg/issues/11169#issuecomment-3315386851 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [I] Delete Files in Table Scans [iceberg-rust]

2025-09-20 Thread via GitHub
github-actions[bot] commented on issue #630: URL: https://github.com/apache/iceberg-rust/issues/630#issuecomment-3315386663 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occu

Re: [PR] Core: Check redundant transforms when updating table partition spec [iceberg]

2025-09-20 Thread via GitHub
github-actions[bot] commented on PR #13885: URL: https://github.com/apache/iceberg/pull/13885#issuecomment-3315386985 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

[I] Roadmap for next releases [iceberg-cpp]

2025-09-20 Thread via GitHub
wgtmac opened a new issue, #226: URL: https://github.com/apache/iceberg-cpp/issues/226 | Version | Milestone | |-|---| | 0.2.0 | - Improve v2 table scan planning (projection, partition filtering, delete files, etc.)- Basic table writing support: append, overwrite- Ta

Re: [PR] Flink: Support parallel file search in deleting orphaned files [iceberg]

2025-09-20 Thread via GitHub
pvary commented on code in PR #13998: URL: https://github.com/apache/iceberg/pull/13998#discussion_r2343908980 ## docs/docs/flink-maintenance.md: ## @@ -222,15 +222,17 @@ env.execute("Table Maintenance Job"); DeleteOrphanFiles Configuration -| Method

Re: [PR] API, Spark 4.0: Add 'skip_file_list' option to RewriteTablePathProcedure for optional file-list generation. [iceberg]

2025-09-20 Thread via GitHub
nastra commented on code in PR #13837: URL: https://github.com/apache/iceberg/pull/13837#discussion_r2352704615 ## spark/v4.0/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestRewriteTablePathProcedure.java: ## @@ -171,8 +178,92 @@ public void testProcedureW

Re: [PR] feat: bump datafusion to 48 [iceberg-rust]

2025-09-20 Thread via GitHub
kevinjqliu commented on PR #1501: URL: https://github.com/apache/iceberg-rust/pull/1501#issuecomment-3275290217 I think the current error message from `substrait` is pretty informative already ``` Error: Custom { kind: NotFound, error: "Could not find `protoc`. If `protoc` is installe

Re: [PR] Flink: Add support for Flink 2.1.0 [iceberg]

2025-09-20 Thread via GitHub
mxm commented on code in PR #13714: URL: https://github.com/apache/iceberg/pull/13714#discussion_r2348129511 ## flink/v2.1/flink/src/main/java/org/apache/iceberg/flink/sink/IcebergWriteAggregator.java: ## @@ -76,11 +85,21 @@ public void open() throws Exception { @Override

Re: [PR] refactor: remove -isystem on local include header files [iceberg-cpp]

2025-09-20 Thread via GitHub
wgtmac commented on code in PR #211: URL: https://github.com/apache/iceberg-cpp/pull/211#discussion_r2329351264 ## cmake_modules/IcebergBuildUtils.cmake: ## @@ -131,7 +131,7 @@ function(add_iceberg_lib LIB_NAME) endif() if(LIB_INCLUDES) - target_include_director

  1   2   >