Re: [I] Spark: add IcebergConnectHiveDelegationTokenProvider [iceberg]

2025-05-26 Thread via GitHub
pvary commented on issue #13116: URL: https://github.com/apache/iceberg/issues/13116#issuecomment-2909014482 @zhangwl9: It would be nice to have both the Spark and the Flink delegation token supported by the HiveCatalog. -- This is an automated message from the Apache Git Service. To resp

Re: [PR] feat: Introduce snapshot summary properties [iceberg-rust]

2025-05-26 Thread via GitHub
dentiny commented on PR #1336: URL: https://github.com/apache/iceberg-rust/pull/1336#issuecomment-2909013780 @Xuanwo friendly ping :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] OpenAPI: Add missing schema field for TableMetadata [iceberg]

2025-05-26 Thread via GitHub
nastra commented on PR #13152: URL: https://github.com/apache/iceberg/pull/13152#issuecomment-2908991182 Since this is a Spec change, can you please mention this on the [d...@iceberg.apache.org](mailto:d...@iceberg.apache.org) list so that people are aware and can vote on this change? --

Re: [PR] AWS: update test cases to verify credentials for the prefixed S3 client [iceberg]

2025-05-26 Thread via GitHub
nastra commented on code in PR #13118: URL: https://github.com/apache/iceberg/pull/13118#discussion_r2106652089 ## aws/src/integration/java/org/apache/iceberg/aws/s3/TestS3FileIO.java: ## @@ -862,6 +919,110 @@ public void multipleStorageCredentialsConfigured() { s3FileIOPro

Re: [I] Spark: add IcebergConnectHiveDelegationTokenProvider [iceberg]

2025-05-26 Thread via GitHub
gaborgsomogyi commented on issue #13116: URL: https://github.com/apache/iceberg/issues/13116#issuecomment-2909024138 +1. Since I'm Flink and Spark committer and I'm the author of those DT frameworks on both side happy to help if there are some questions. -- This is an automated message fr

Re: [PR] Core: Add basic classes for writing table format-version 4 [iceberg]

2025-05-26 Thread via GitHub
nastra commented on code in PR #13123: URL: https://github.com/apache/iceberg/pull/13123#discussion_r2106866848 ## api/src/test/java/org/apache/iceberg/TestHelpers.java: ## @@ -54,7 +54,7 @@ public class TestHelpers { private TestHelpers() {} - public static final int MA

Re: [PR] feat: support decompress gzip metadata [iceberg-cpp]

2025-05-26 Thread via GitHub
yingcai-cy commented on code in PR #108: URL: https://github.com/apache/iceberg-cpp/pull/108#discussion_r2106756398 ## src/iceberg/util/gzip_internal.cc: ## @@ -0,0 +1,95 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreem

Re: [PR] Core: Add basic classes for writing table format-version 4 [iceberg]

2025-05-26 Thread via GitHub
nastra commented on code in PR #13123: URL: https://github.com/apache/iceberg/pull/13123#discussion_r2106866848 ## api/src/test/java/org/apache/iceberg/TestHelpers.java: ## @@ -54,7 +54,7 @@ public class TestHelpers { private TestHelpers() {} - public static final int MA

Re: [PR] Experiment implementation for catalog builder [iceberg-rust]

2025-05-26 Thread via GitHub
liurenjie1024 commented on PR #1231: URL: https://github.com/apache/iceberg-rust/pull/1231#issuecomment-2909044061 > Are we sure that we need the CatalogBuilder trait? What would be its purpose? Hi, @c-thiel Sorry for misclarification. For more background, please refer to https://git

[I] Incrementally computing partition stats can missing deleted files [iceberg]

2025-05-26 Thread via GitHub
lirui-apache opened a new issue, #13155: URL: https://github.com/apache/iceberg/issues/13155 ### Apache Iceberg version main (development) ### Query engine None ### Please describe the bug 🐞 It seems stats diff is computed only from the current snapshot, whi

Re: [PR] Hive: Throw exception for when listing a non-existing namespace [iceberg]

2025-05-26 Thread via GitHub
pvary commented on code in PR #13130: URL: https://github.com/apache/iceberg/pull/13130#discussion_r2106791809 ## hive-metastore/src/test/java/org/apache/iceberg/hive/TestHiveCatalog.java: ## @@ -1209,11 +1208,4 @@ public void testDatabaseLocationWithSlashInWarehouseDir() {

Re: [I] Incrementally computing partition stats can miss deleted files [iceberg]

2025-05-26 Thread via GitHub
lirui-apache commented on issue #13155: URL: https://github.com/apache/iceberg/issues/13155#issuecomment-2909052640 Hi @ajantha-bhat , could you help verify the issue, thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

Re: [PR] feat: support decompress gzip metadata [iceberg-cpp]

2025-05-26 Thread via GitHub
yingcai-cy commented on code in PR #108: URL: https://github.com/apache/iceberg-cpp/pull/108#discussion_r2103886619 ## src/iceberg/table_metadata.cc: ## @@ -153,14 +154,70 @@ Result TableMetadataUtil::CodecFromFileName( return MetadataFileCodecType::kNone; } +class GZipDe

Re: [PR] chore: introduce `nightly` feature flag to provide error backtrace [iceberg-rust]

2025-05-26 Thread via GitHub
Xuanwo commented on PR #1340: URL: https://github.com/apache/iceberg-rust/pull/1340#issuecomment-2908909319 Detecting nightly features is not recommended by the Rust community, as nightly features can be renamed or removed at any time. For example: https://github.com/tkaitchuck/aHash/

Re: [PR] Flink: Migrate Flink TableSchema for IcebergSource [iceberg]

2025-05-26 Thread via GitHub
pvary commented on code in PR #13072: URL: https://github.com/apache/iceberg/pull/13072#discussion_r2106783588 ## flink/v2.0/flink/src/main/java/org/apache/iceberg/flink/FlinkSchemaUtil.java: ## @@ -192,7 +222,9 @@ public static Type convert(LogicalType flinkType) { * *

Re: [PR] Flink: Support compact in iceberg sink v2 [iceberg]

2025-05-26 Thread via GitHub
pvary commented on code in PR #12979: URL: https://github.com/apache/iceberg/pull/12979#discussion_r2106809132 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/sink/CommittableToTableChangeConverter.java: ## @@ -0,0 +1,181 @@ +/* + * Licensed to the Apache Software Fo

[I] Add docs of Spark SQL functions for Iceberg transforms [iceberg]

2025-05-26 Thread via GitHub
manuzhang opened a new issue, #13156: URL: https://github.com/apache/iceberg/issues/13156 ### Feature Request / Improvement Add docs of following Spark SQL functions for Iceberg transforms - system.iceberg_version - system.bucket - system.years - system.month - system.d

Re: [PR] demo: an easy to use catalog loader [iceberg-rust]

2025-05-26 Thread via GitHub
liurenjie1024 commented on code in PR #1372: URL: https://github.com/apache/iceberg-rust/pull/1372#discussion_r2106849970 ## crates/iceberg/src/catalog/mod.rs: ## @@ -36,7 +37,19 @@ use crate::spec::{ use crate::table::Table; use crate::{Error, ErrorKind, Result}; +/// The C

[PR] Build: Bump huggingface-hub from 0.31.4 to 0.32.1 [iceberg-python]

2025-05-26 Thread via GitHub
dependabot[bot] opened a new pull request, #2046: URL: https://github.com/apache/iceberg-python/pull/2046 Bumps [huggingface-hub](https://github.com/huggingface/huggingface_hub) from 0.31.4 to 0.32.1. Release notes Sourced from https://github.com/huggingface/huggingface_hub/release

[PR] Build: Bump mkdocstrings-python from 1.16.10 to 1.16.11 [iceberg-python]

2025-05-26 Thread via GitHub
dependabot[bot] opened a new pull request, #2044: URL: https://github.com/apache/iceberg-python/pull/2044 Bumps [mkdocstrings-python](https://github.com/mkdocstrings/python) from 1.16.10 to 1.16.11. Release notes Sourced from https://github.com/mkdocstrings/python/releases";>mkdocs

[PR] Build: Bump coverage from 7.8.0 to 7.8.2 [iceberg-python]

2025-05-26 Thread via GitHub
dependabot[bot] opened a new pull request, #2047: URL: https://github.com/apache/iceberg-python/pull/2047 Bumps [coverage](https://github.com/nedbat/coveragepy) from 7.8.0 to 7.8.2. Changelog Sourced from https://github.com/nedbat/coveragepy/blob/master/CHANGES.rst";>coverage's cha

Re: [I] Tracking issues of Iceberg Rust 0.5.0 Release (May 2025) [iceberg-rust]

2025-05-26 Thread via GitHub
kevinjqliu commented on issue #1325: URL: https://github.com/apache/iceberg-rust/issues/1325#issuecomment-2910782152 https://rust.iceberg.apache.org/release.html#official-release ``` git checkout v0.5.0-rc.2 git tag -s "v0.5.0" git push origin "v0.5.0" ``` Tag push trigg

Re: [I] Tracking issues of Iceberg Rust 0.5.0 Release (May 2025) [iceberg-rust]

2025-05-26 Thread via GitHub
kevinjqliu commented on issue #1325: URL: https://github.com/apache/iceberg-rust/issues/1325#issuecomment-2910783263 I was able to verify `cargo publish` can run successfully with the `iceberg-catalog-memory` dev dependency commented out ``` cargo publish --all-features --dry-run --all

Re: [I] Tracking issues of Iceberg Rust 0.5.0 Release (May 2025) [iceberg-rust]

2025-05-26 Thread via GitHub
kevinjqliu commented on issue #1325: URL: https://github.com/apache/iceberg-rust/issues/1325#issuecomment-2910809436 hm, these are hardcoded. should we add more packages here? https://github.com/apache/iceberg-rust/blob/fbf9b92e0a201d1e37d74a568687174eac823539/.github/workflows/publi

Re: [I] ADLSFileIO cache DefaultAzureCredentials? [iceberg]

2025-05-26 Thread via GitHub
github-actions[bot] closed issue #11523: ADLSFileIO cache DefaultAzureCredentials? URL: https://github.com/apache/iceberg/issues/11523 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

Re: [I] Tracking issues of Iceberg Rust 0.5.0 Release (May 2025) [iceberg-rust]

2025-05-26 Thread via GitHub
kevinjqliu commented on issue #1325: URL: https://github.com/apache/iceberg-rust/issues/1325#issuecomment-2910894995 looks like `pyiceberg-core 0.5.0` already published on pypi https://pypi.org/project/pyiceberg-core/0.5.0/ I think we need to yank this, fix the underlying issue and

Re: [PR] feat: support decompress gzip metadata [iceberg-cpp]

2025-05-26 Thread via GitHub
wgtmac commented on code in PR #108: URL: https://github.com/apache/iceberg-cpp/pull/108#discussion_r2108002007 ## src/iceberg/util/gzip_internal.cc: ## @@ -0,0 +1,94 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements

Re: [I] Spark: add IcebergConnectHiveDelegationTokenProvider [iceberg]

2025-05-26 Thread via GitHub
zhangwl9 commented on issue #13116: URL: https://github.com/apache/iceberg/issues/13116#issuecomment-2910897592 @pvary @gaborgsomogyi For Spark , I'm going to use the scheme shown above to implement an IcebergHiveConnectorDelegationTokenProvider within the Iceberg_Spark module

[PR] static table metadata access support [iceberg-cpp]

2025-05-26 Thread via GitHub
lishuxu opened a new pull request, #111: URL: https://github.com/apache/iceberg-cpp/pull/111 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-

[I] Flink: add IcebergConnectHiveDelegationTokenProvider [iceberg]

2025-05-26 Thread via GitHub
zhangwl9 opened a new issue, #13159: URL: https://github.com/apache/iceberg/issues/13159 ### Feature Request / Improvement test ### Query engine Flink ### Willingness to contribute - [x] I can contribute this improvement/feature independently - [x] I woul

[I] Add IcebergHiveConnectorDelegationTokenProvider for Iceberg [iceberg]

2025-05-26 Thread via GitHub
zhangwl9 opened a new issue, #13158: URL: https://github.com/apache/iceberg/issues/13158 ### Feature Request / Improvement I want to implement both Spark and the Flink delegation token supported by the HiveCatalog; ### Query engine None ### Willingness to cont

Re: [I] Add IcebergHiveConnectorDelegationTokenProvider for Iceberg [iceberg]

2025-05-26 Thread via GitHub
zhangwl9 commented on issue #13158: URL: https://github.com/apache/iceberg/issues/13158#issuecomment-2910909964 - [ ] Add IcebergConnectHiveDelegationTokenProvider for HiveCatalog in Iceberg_Spark module - [ ] Add IcebergConnectHiveDelegationTokenProvider for HiveCatalog in Iceberg_Flink

[PR] fix 0.5.x release `cargo publish` [iceberg-rust]

2025-05-26 Thread via GitHub
kevinjqliu opened a new pull request, #1379: URL: https://github.com/apache/iceberg-rust/pull/1379 ## Which issue does this PR close? - Closes #. ## What changes are included in this PR? This PR removes `iceberg-catalog-memory` as a dev-dependency of `crates/

Re: [I] Tracking issues of Iceberg Rust 0.5.0 Release (May 2025) [iceberg-rust]

2025-05-26 Thread via GitHub
liurenjie1024 commented on issue #1325: URL: https://github.com/apache/iceberg-rust/issues/1325#issuecomment-2910937265 Seems we need a way to detect all crates. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] Spec: add sort order to spec [iceberg]

2025-05-26 Thread via GitHub
Nicholasjl commented on PR #2055: URL: https://github.com/apache/iceberg/pull/2055#issuecomment-2910943224 Hi, I noticed that in IceNerg, the expression -0 < 0 evaluates to true. However, according to the IEEE 754 standard, -0.0 and 0.0 are considered numerically equal, and -0.0 < 0.

Re: [PR] fix 0.5.x release `cargo publish` [iceberg-rust]

2025-05-26 Thread via GitHub
liurenjie1024 commented on PR #1379: URL: https://github.com/apache/iceberg-rust/pull/1379#issuecomment-2910950841 The ut failure is caused by generating doc. I think we could remove those docs for now. -- This is an automated message from the Apache Git Service. To respond to the message

Re: [I] Tracking issues of Iceberg Rust 0.5.0 Release (May 2025) [iceberg-rust]

2025-05-26 Thread via GitHub
kevinjqliu commented on issue #1325: URL: https://github.com/apache/iceberg-rust/issues/1325#issuecomment-2911104289 ASF release ``` git checkout v0.5.1-rc.1 ICEBERG_VERSION=0.5.1 ICEBERG_VERSION_RC=1 ./scripts/release.sh svn co https://dist.apache.org/repos/dist/dev/iceberg/

Re: [I] Tracking issues of Iceberg Rust 0.5.0 Release (May 2025) [iceberg-rust]

2025-05-26 Thread via GitHub
kevinjqliu commented on issue #1325: URL: https://github.com/apache/iceberg-rust/issues/1325#issuecomment-2911096775 > Do we need another vote for 0.5.1? yep, we should push a new tag for `v0.5.1-rc.1` https://github.com/apache/iceberg-rust/releases/tag/v0.5.1-rc.1 Tr

Re: [I] Tracking issues of Iceberg Rust 0.5.0 Release (May 2025) [iceberg-rust]

2025-05-26 Thread via GitHub
liurenjie1024 commented on issue #1325: URL: https://github.com/apache/iceberg-rust/issues/1325#issuecomment-2911090349 > Bump iceberg-rust version to 0.5.1 [#1380](https://github.com/apache/iceberg-rust/pull/1380) Do we need another vote for 0.5.1? -- This is an automated mess

Re: [PR] fix: add metadata_properties to _construct_parameters when update hive table [iceberg-python]

2025-05-26 Thread via GitHub
geruh commented on code in PR #2013: URL: https://github.com/apache/iceberg-python/pull/2013#discussion_r2108177674 ## pyiceberg/catalog/hive.py: ## @@ -211,11 +211,18 @@ def _construct_hive_storage_descriptor( DEFAULT_PROPERTIES = {TableProperties.PARQUET_COMPRESSION: TablePr

Re: [I] Tracking issues of Iceberg Rust 0.5.0 Release (May 2025) [iceberg-rust]

2025-05-26 Thread via GitHub
kevinjqliu commented on issue #1325: URL: https://github.com/apache/iceberg-rust/issues/1325#issuecomment-296409 0.5.1 RC1 devlist thread, https://lists.apache.org/thread/h1qw21cdc56vltt9mzkn0sq36mp9tmpw -- This is an automated message from the Apache Git Service. To respond to th

[I] Literals.LongLiteral conversion issue to TimestampNanoLiteral [iceberg]

2025-05-26 Thread via GitHub
xxlaykxx opened a new issue, #13160: URL: https://github.com/apache/iceberg/issues/13160 ### Apache Iceberg version 1.9.0 (latest release) ### Query engine Dremio ### Please describe the bug 🐞 Because for the conversion to TimestampNanoLiteral, LongLiteral u

Re: [PR] Bump iceberg-rust version to 0.5.1 [iceberg-rust]

2025-05-26 Thread via GitHub
kevinjqliu commented on code in PR #1380: URL: https://github.com/apache/iceberg-rust/pull/1380#discussion_r2108118748 ## bindings/python/pyproject.toml: ## @@ -33,7 +33,7 @@ classifiers = [ name = "pyiceberg-core" readme = "project-description.md" requires-python = "~=3.9" -

[PR] feat: basic table scan planning [iceberg-cpp]

2025-05-26 Thread via GitHub
gty404 opened a new pull request, #112: URL: https://github.com/apache/iceberg-cpp/pull/112 Introducing basic scan table data interface -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

Re: [PR] Flink: Migrate Flink TableSchema for IcebergSource [iceberg]

2025-05-26 Thread via GitHub
liamzwbao commented on code in PR #13072: URL: https://github.com/apache/iceberg/pull/13072#discussion_r2108061992 ## flink/v2.0/flink/src/main/java/org/apache/iceberg/flink/FlinkSchemaUtil.java: ## @@ -192,7 +222,9 @@ public static Type convert(LogicalType flinkType) { *

Re: [I] Tracking issues of Iceberg Rust 0.5.0 Release (May 2025) [iceberg-rust]

2025-05-26 Thread via GitHub
kevinjqliu commented on issue #1325: URL: https://github.com/apache/iceberg-rust/issues/1325#issuecomment-2910963966 `iceberg_catalog_memory` is used in `crates/iceberg`'s docstring. For example, https://github.com/apache/iceberg-rust/blob/fbf9b92e0a201d1e37d74a568687174eac823539/crates/ice

Re: [PR] static table metadata access support [iceberg-cpp]

2025-05-26 Thread via GitHub
lishuxu closed pull request #111: static table metadata access support URL: https://github.com/apache/iceberg-cpp/pull/111 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

Re: [PR] chore(deps): Bump aws-sdk-glue from 1.94.0 to 1.97.0 [iceberg-rust]

2025-05-26 Thread via GitHub
liurenjie1024 merged PR #1376: URL: https://github.com/apache/iceberg-rust/pull/1376 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@

Re: [I] Tracking issues of Iceberg Rust 0.5.0 Release (May 2025) [iceberg-rust]

2025-05-26 Thread via GitHub
kevinjqliu commented on issue #1325: URL: https://github.com/apache/iceberg-rust/issues/1325#issuecomment-2910985626 #1379 to fix forward the issue and release under 0.5.1 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

Re: [PR] fix 0.5.x release `cargo publish` [iceberg-rust]

2025-05-26 Thread via GitHub
kevinjqliu commented on code in PR #1379: URL: https://github.com/apache/iceberg-rust/pull/1379#discussion_r2108078879 ## crates/iceberg/src/lib.rs: ## @@ -21,7 +21,9 @@ //! //! ## Scan A Table //! -//! ```rust, no_run +//! ```rust, ignore Review Comment: @liurenjie1024 w

Re: [PR] fix 0.5.x release `cargo publish` [iceberg-rust]

2025-05-26 Thread via GitHub
liurenjie1024 commented on code in PR #1379: URL: https://github.com/apache/iceberg-rust/pull/1379#discussion_r2108084363 ## crates/iceberg/src/lib.rs: ## @@ -21,7 +21,9 @@ //! //! ## Scan A Table //! -//! ```rust, no_run +//! ```rust, ignore Review Comment: Yes, it's fin

[PR] Build: Bump getdaft from 0.4.15 to 0.4.16 [iceberg-python]

2025-05-26 Thread via GitHub
dependabot[bot] opened a new pull request, #2042: URL: https://github.com/apache/iceberg-python/pull/2042 Bumps [getdaft](https://github.com/Eventual-Inc/Daft) from 0.4.15 to 0.4.16. Release notes Sourced from https://github.com/Eventual-Inc/Daft/releases";>getdaft's releases.

[PR] Build: Bump thrift from 0.21.0 to 0.22.0 [iceberg-python]

2025-05-26 Thread via GitHub
dependabot[bot] opened a new pull request, #2043: URL: https://github.com/apache/iceberg-python/pull/2043 Bumps [thrift](https://github.com/apache/thrift) from 0.21.0 to 0.22.0. Release notes Sourced from https://github.com/apache/thrift/releases";>thrift's releases. Version

Re: [PR] Avoid Avro recursive schema for Variant schema. [iceberg]

2025-05-26 Thread via GitHub
github-actions[bot] closed pull request #12459: Avoid Avro recursive schema for Variant schema. URL: https://github.com/apache/iceberg/pull/12459 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] Avoid Avro recursive schema for Variant schema. [iceberg]

2025-05-26 Thread via GitHub
github-actions[bot] commented on PR #12459: URL: https://github.com/apache/iceberg/pull/12459#issuecomment-2910801756 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

Re: [I] Tracking issues of Iceberg Rust 0.5.0 Release (May 2025) [iceberg-rust]

2025-05-26 Thread via GitHub
kevinjqliu commented on issue #1325: URL: https://github.com/apache/iceberg-rust/issues/1325#issuecomment-2910995021 @liurenjie1024 / @Xuanwo / @sungwy are any of you owners for https://pypi.org/project/pyiceberg-core/? Only owners can yank a release -- This is an automated message from

Re: [I] Tracking issues of Iceberg Rust 0.5.0 Release (May 2025) [iceberg-rust]

2025-05-26 Thread via GitHub
liurenjie1024 commented on issue #1325: URL: https://github.com/apache/iceberg-rust/issues/1325#issuecomment-2910979921 > `iceberg_catalog_memory` is used in `crates/iceberg`'s docstring. For example, > > [iceberg-rust/crates/iceberg/src/lib.rs](https://github.com/apa

Re: [I] java.io.IOException: can not read class org.apache.iceberg.shaded.org.apache.parquet.format.PageHeader: Required field 'num_values' was not found in serialized data [iceberg]

2025-05-26 Thread via GitHub
github-actions[bot] commented on issue #11614: URL: https://github.com/apache/iceberg/issues/11614#issuecomment-2910801702 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [PR] Spark: SnapshotTableSparkAction add validation for non-overlapping source/dest table paths. [iceberg]

2025-05-26 Thread via GitHub
github-actions[bot] closed pull request #12779: Spark: SnapshotTableSparkAction add validation for non-overlapping source/dest table paths. URL: https://github.com/apache/iceberg/pull/12779 -- This is an automated message from the Apache Git Service. To respond to the message, please log on t

Re: [PR] Spark: SnapshotTableSparkAction add validation for non-overlapping source/dest table paths. [iceberg]

2025-05-26 Thread via GitHub
github-actions[bot] commented on PR #12779: URL: https://github.com/apache/iceberg/pull/12779#issuecomment-2910801791 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

Re: [I] ADLSFileIO cache DefaultAzureCredentials? [iceberg]

2025-05-26 Thread via GitHub
github-actions[bot] commented on issue #11523: URL: https://github.com/apache/iceberg/issues/11523#issuecomment-2910801639 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache

Re: [I] How to run streaming upserts and maintenance simultaneously? [iceberg]

2025-05-26 Thread via GitHub
github-actions[bot] commented on issue #11530: URL: https://github.com/apache/iceberg/issues/11530#issuecomment-2910801666 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache

Re: [I] How to run streaming upserts and maintenance simultaneously? [iceberg]

2025-05-26 Thread via GitHub
github-actions[bot] closed issue #11530: How to run streaming upserts and maintenance simultaneously? URL: https://github.com/apache/iceberg/issues/11530 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [I] Enhance `catalog.create_table` API to enable creation of table with matching `field_ids` to provided Schema [iceberg-python]

2025-05-26 Thread via GitHub
github-actions[bot] commented on issue #1284: URL: https://github.com/apache/iceberg-python/issues/1284#issuecomment-2910804096 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity

[PR] Build: Bump mypy-boto3-glue from 1.38.18 to 1.38.22 [iceberg-python]

2025-05-26 Thread via GitHub
dependabot[bot] opened a new pull request, #2038: URL: https://github.com/apache/iceberg-python/pull/2038 Bumps [mypy-boto3-glue](https://github.com/youtype/mypy_boto3_builder) from 1.38.18 to 1.38.22. Release notes Sourced from https://github.com/youtype/mypy_boto3_builder/release

[PR] Build: Bump mkdocs-autorefs from 1.4.1 to 1.4.2 [iceberg-python]

2025-05-26 Thread via GitHub
dependabot[bot] opened a new pull request, #2048: URL: https://github.com/apache/iceberg-python/pull/2048 Bumps [mkdocs-autorefs](https://github.com/mkdocstrings/autorefs) from 1.4.1 to 1.4.2. Release notes Sourced from https://github.com/mkdocstrings/autorefs/releases";>mkdocs-aut

Re: [I] Tracking issues of Iceberg Rust 0.5.0 Release (May 2025) [iceberg-rust]

2025-05-26 Thread via GitHub
kevinjqliu commented on issue #1325: URL: https://github.com/apache/iceberg-rust/issues/1325#issuecomment-2910764412 vote passed! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

Re: [PR] chore(deps): Bump uuid from 1.16.0 to 1.17.0 [iceberg-rust]

2025-05-26 Thread via GitHub
liurenjie1024 merged PR #1375: URL: https://github.com/apache/iceberg-rust/pull/1375 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@

Re: [PR] Bump iceberg-rust version to 0.5.0 [iceberg-rust]

2025-05-26 Thread via GitHub
kevinjqliu commented on PR #1345: URL: https://github.com/apache/iceberg-rust/pull/1345#issuecomment-2911019657 thanks for the review @liurenjie1024 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] fix 0.5.x release `cargo publish` [iceberg-rust]

2025-05-26 Thread via GitHub
kevinjqliu merged PR #1379: URL: https://github.com/apache/iceberg-rust/pull/1379 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ice

Re: [I] Tracking issues of Iceberg Rust 0.5.0 Release (May 2025) [iceberg-rust]

2025-05-26 Thread via GitHub
kevinjqliu commented on issue #1325: URL: https://github.com/apache/iceberg-rust/issues/1325#issuecomment-2911035129 Bump iceberg-rust version to 0.5.1 #1380 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] chore(deps): Bump ordered-float from 2.10.1 to 4.6.0 [iceberg-rust]

2025-05-26 Thread via GitHub
liurenjie1024 merged PR #1374: URL: https://github.com/apache/iceberg-rust/pull/1374 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@

Re: [I] Spark: add IcebergHiveConnectorDelegationTokenProvider for HiveCatalog [iceberg]

2025-05-26 Thread via GitHub
gaborgsomogyi commented on issue #13116: URL: https://github.com/apache/iceberg/issues/13116#issuecomment-2911197379 Happy to hear experiences when it's in place. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[PR] Build: Bump pydantic from 2.11.4 to 2.11.5 [iceberg-python]

2025-05-26 Thread via GitHub
dependabot[bot] opened a new pull request, #2045: URL: https://github.com/apache/iceberg-python/pull/2045 Bumps [pydantic](https://github.com/pydantic/pydantic) from 2.11.4 to 2.11.5. Release notes Sourced from https://github.com/pydantic/pydantic/releases";>pydantic's releases.

[PR] Build: Bump moto from 5.1.4 to 5.1.5 [iceberg-python]

2025-05-26 Thread via GitHub
dependabot[bot] opened a new pull request, #2039: URL: https://github.com/apache/iceberg-python/pull/2039 Bumps [moto](https://github.com/getmoto/moto) from 5.1.4 to 5.1.5. Changelog Sourced from https://github.com/getmoto/moto/blob/master/CHANGELOG.md";>moto's changelog. 5.

Re: [PR] Bump iceberg-rust version to 0.5.1 [iceberg-rust]

2025-05-26 Thread via GitHub
kevinjqliu commented on PR #1380: URL: https://github.com/apache/iceberg-rust/pull/1380#issuecomment-2911043321 Double checked publishing: ``` cd crates/iceberg cargo publish --all-features --dry-run ``` ran successfully -- This is an automated message from the Apache Git Se

Re: [I] Tracking issues of Iceberg Rust 0.5.0 Release (May 2025) [iceberg-rust]

2025-05-26 Thread via GitHub
kevinjqliu commented on issue #1325: URL: https://github.com/apache/iceberg-rust/issues/1325#issuecomment-2910986496 @liurenjie1024 should we yank pyiceberg-core 0.5.0 (https://pypi.org/project/pyiceberg-core/0.5.0/) since its not an official release? I dont have the proper permission t

Re: [PR] Bump iceberg-rust version to 0.5.1 [iceberg-rust]

2025-05-26 Thread via GitHub
liurenjie1024 merged PR #1380: URL: https://github.com/apache/iceberg-rust/pull/1380 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@

Re: [PR] feat: support decompress gzip metadata [iceberg-cpp]

2025-05-26 Thread via GitHub
dongxiao1198 commented on code in PR #108: URL: https://github.com/apache/iceberg-cpp/pull/108#discussion_r2107001574 ## src/iceberg/util/gzip_internal.cc: ## @@ -0,0 +1,95 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agre

Re: [PR] test: Add missing tests for update_namespace method in sql catalog [iceberg-rust]

2025-05-26 Thread via GitHub
liurenjie1024 commented on PR #1373: URL: https://github.com/apache/iceberg-rust/pull/1373#issuecomment-2909262129 > I'm looking at the behaviour of the method, and it seems as though an edge case may cause unwanted behaviour. > > The method is structured into two parts > > 1.

Re: [PR] AWS: Close the S3SeekableInputStreamFactory before removing from cache [iceberg]

2025-05-26 Thread via GitHub
nastra merged PR #12891: URL: https://github.com/apache/iceberg/pull/12891 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Feature/write to branch [iceberg-python]

2025-05-26 Thread via GitHub
SebastienN15 closed pull request #2009: Feature/write to branch URL: https://github.com/apache/iceberg-python/pull/2009 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsub

Re: [I] Incrementally computing partition stats can miss deleted files [iceberg]

2025-05-26 Thread via GitHub
ajantha-bhat commented on issue #13155: URL: https://github.com/apache/iceberg/issues/13155#issuecomment-2909461143 Hi, @lirui-apache thanks for reporting this. Looks like this is an edge case with copy-on-write delete. (Test cases covers only merge on read!). I assumed that

[PR] feat(catalog/glue): add option to customize IO loader function [iceberg-go]

2025-05-26 Thread via GitHub
vbekiaris opened a new pull request, #441: URL: https://github.com/apache/iceberg-go/pull/441 Adds an option to customize the function used to determine which `io.IO` implementation to load when constructing a Glue catalog. -- This is an automated message from the Apache Git Service. To r

Re: [PR] Docs: add Tinybird to the list of vendors and blog posts [iceberg]

2025-05-26 Thread via GitHub
nastra commented on code in PR #13128: URL: https://github.com/apache/iceberg/pull/13128#discussion_r2106662135 ## site/docs/vendors.md: ## @@ -124,10 +124,18 @@ Starburst is a commercial offering for the [Trino query engine](https://trino.io [Tabular](https://tabular.io/) i

Re: [PR] Flink: Backport fix npe in TaskResultAggregator when job recovery to Flink 1.19 and 1.20 [iceberg]

2025-05-26 Thread via GitHub
pvary merged PR #13140: URL: https://github.com/apache/iceberg/pull/13140 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] test: Add missing tests for update_namespace method in sql catalog [iceberg-rust]

2025-05-26 Thread via GitHub
liurenjie1024 merged PR #1373: URL: https://github.com/apache/iceberg-rust/pull/1373 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@

Re: [PR] Flink: Support compact in iceberg sink v2 [iceberg]

2025-05-26 Thread via GitHub
pvary commented on code in PR #12979: URL: https://github.com/apache/iceberg/pull/12979#discussion_r2106806912 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/sink/CommittableToTableChangeConverter.java: ## @@ -0,0 +1,181 @@ +/* + * Licensed to the Apache Software Fo

Re: [I] Spark: add IcebergConnectHiveDelegationTokenProvider [iceberg]

2025-05-26 Thread via GitHub
gaborgsomogyi commented on issue #13116: URL: https://github.com/apache/iceberg/issues/13116#issuecomment-2909026174 This can decrease significanly the number of authentication requests allowing to deploy more scalable applications. -- This is an automated message from the Apache Git Serv

Re: [PR] feat: implement initial MemoryCatalog functionality with namespace and table support [iceberg-cpp]

2025-05-26 Thread via GitHub
gty404 commented on PR #80: URL: https://github.com/apache/iceberg-cpp/pull/80#issuecomment-2909485546 @Xuanwo @Fokko Could you help review this? Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [I] Incrementally computing partition stats can miss deleted files [iceberg]

2025-05-26 Thread via GitHub
ajantha-bhat commented on issue #13155: URL: https://github.com/apache/iceberg/issues/13155#issuecomment-2909762512 > I think when a snapshot is being created, previous manifests without live entries will not be tracked. So maybe we have to apply the incremental snapshots one by one? And pr

Re: [PR] Flink: Support compact in iceberg sink v2 [iceberg]

2025-05-26 Thread via GitHub
Guosmilesmile commented on code in PR #12979: URL: https://github.com/apache/iceberg/pull/12979#discussion_r2107397429 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/sink/CommittableToTableChangeConverter.java: ## @@ -0,0 +1,181 @@ +/* + * Licensed to the Apache Sof

Re: [PR] Flink: Support compact in iceberg sink v2 [iceberg]

2025-05-26 Thread via GitHub
Guosmilesmile commented on code in PR #12979: URL: https://github.com/apache/iceberg/pull/12979#discussion_r2107399213 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/maintenance/api/RewriteDataFilesConfig.java: ## @@ -0,0 +1,100 @@ +/* + * Licensed to the Apache Sof

Re: [PR] Flink: Support compact in iceberg sink v2 [iceberg]

2025-05-26 Thread via GitHub
Guosmilesmile commented on code in PR #12979: URL: https://github.com/apache/iceberg/pull/12979#discussion_r2107402061 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/maintenance/api/TableMaintenanceConfig.java: ## @@ -0,0 +1,61 @@ +/* + * Licensed to the Apache Soft

Re: [PR] Core: Add basic classes for writing table format-version 4 [iceberg]

2025-05-26 Thread via GitHub
nastra commented on code in PR #13123: URL: https://github.com/apache/iceberg/pull/13123#discussion_r2106692536 ## api/src/test/java/org/apache/iceberg/TestHelpers.java: ## @@ -54,7 +54,7 @@ public class TestHelpers { private TestHelpers() {} - public static final int MA

Re: [PR] Core: Add basic classes for writing table format-version 4 [iceberg]

2025-05-26 Thread via GitHub
ajantha-bhat commented on code in PR #13123: URL: https://github.com/apache/iceberg/pull/13123#discussion_r2106699014 ## api/src/test/java/org/apache/iceberg/TestHelpers.java: ## @@ -54,7 +54,7 @@ public class TestHelpers { private TestHelpers() {} - public static final

Re: [PR] Core: Add basic classes for writing table format-version 4 [iceberg]

2025-05-26 Thread via GitHub
ajantha-bhat commented on code in PR #13123: URL: https://github.com/apache/iceberg/pull/13123#discussion_r2106699014 ## api/src/test/java/org/apache/iceberg/TestHelpers.java: ## @@ -54,7 +54,7 @@ public class TestHelpers { private TestHelpers() {} - public static final

Re: [PR] Docs, Flink: Fix equality fields requirement in upsert mode [iceberg]

2025-05-26 Thread via GitHub
pvary commented on code in PR #13127: URL: https://github.com/apache/iceberg/pull/13127#discussion_r2106716484 ## docs/docs/flink-writes.md: ## @@ -75,7 +75,7 @@ Iceberg supports `UPSERT` based on the primary key when writing data into v2 tab ``` !!! info -OVERWRITE

Re: [PR] Flink: Backport fix npe in TaskResultAggregator when job recovery to Flink 1.19 and 1.20 [iceberg]

2025-05-26 Thread via GitHub
pvary commented on PR #13140: URL: https://github.com/apache/iceberg/pull/13140#issuecomment-2908797656 Merged to main. Thanks for the backport @Guosmilesmile! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[I] doc: Add how to verify release section in doc. [iceberg-rust]

2025-05-26 Thread via GitHub
liurenjie1024 opened a new issue, #1378: URL: https://github.com/apache/iceberg-rust/issues/1378 ### Is your feature request related to a problem or challenge? Add a section of how to verify release in doc. ### Describe the solution you'd like _No response_ ### Wil

  1   2   >