[I] Literals.LongLiteral conversion issue to TimestampNanoLiteral [iceberg]

2025-05-26 Thread via GitHub
xxlaykxx opened a new issue, #13160: URL: https://github.com/apache/iceberg/issues/13160 ### Apache Iceberg version 1.9.0 (latest release) ### Query engine Dremio ### Please describe the bug šŸž Because for the conversion to TimestampNanoLiteral, LongLiteral u

[PR] feat: basic table scan planning [iceberg-cpp]

2025-05-26 Thread via GitHub
gty404 opened a new pull request, #112: URL: https://github.com/apache/iceberg-cpp/pull/112 Introducing basic scan table data interface -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

Re: [PR] Bump iceberg-rust version to 0.5.1 [iceberg-rust]

2025-05-26 Thread via GitHub
kevinjqliu commented on code in PR #1380: URL: https://github.com/apache/iceberg-rust/pull/1380#discussion_r2108118748 ## bindings/python/pyproject.toml: ## @@ -33,7 +33,7 @@ classifiers = [ name = "pyiceberg-core" readme = "project-description.md" requires-python = "~=3.9" -

Re: [I] Spark: add IcebergHiveConnectorDelegationTokenProvider for HiveCatalog [iceberg]

2025-05-26 Thread via GitHub
gaborgsomogyi commented on issue #13116: URL: https://github.com/apache/iceberg/issues/13116#issuecomment-2911197379 Happy to hear experiences when it's in place. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] chore(deps): Bump ordered-float from 2.10.1 to 4.6.0 [iceberg-rust]

2025-05-26 Thread via GitHub
liurenjie1024 merged PR #1374: URL: https://github.com/apache/iceberg-rust/pull/1374 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@

Re: [I] Tracking issues of Iceberg Rust 0.5.0 Release (May 2025) [iceberg-rust]

2025-05-26 Thread via GitHub
kevinjqliu commented on issue #1325: URL: https://github.com/apache/iceberg-rust/issues/1325#issuecomment-296409 0.5.1 RC1 devlist thread, https://lists.apache.org/thread/h1qw21cdc56vltt9mzkn0sq36mp9tmpw -- This is an automated message from the Apache Git Service. To respond to th

Re: [PR] fix: add metadata_properties to _construct_parameters when update hive table [iceberg-python]

2025-05-26 Thread via GitHub
geruh commented on code in PR #2013: URL: https://github.com/apache/iceberg-python/pull/2013#discussion_r2108177674 ## pyiceberg/catalog/hive.py: ## @@ -211,11 +211,18 @@ def _construct_hive_storage_descriptor( DEFAULT_PROPERTIES = {TableProperties.PARQUET_COMPRESSION: TablePr

Re: [I] Tracking issues of Iceberg Rust 0.5.0 Release (May 2025) [iceberg-rust]

2025-05-26 Thread via GitHub
liurenjie1024 commented on issue #1325: URL: https://github.com/apache/iceberg-rust/issues/1325#issuecomment-2911090349 > Bump iceberg-rust version to 0.5.1 [#1380](https://github.com/apache/iceberg-rust/pull/1380) Do we need another vote for 0.5.1? -- This is an automated mess

Re: [I] Tracking issues of Iceberg Rust 0.5.0 Release (May 2025) [iceberg-rust]

2025-05-26 Thread via GitHub
kevinjqliu commented on issue #1325: URL: https://github.com/apache/iceberg-rust/issues/1325#issuecomment-2911104289 ASF release ``` git checkout v0.5.1-rc.1 ICEBERG_VERSION=0.5.1 ICEBERG_VERSION_RC=1 ./scripts/release.sh svn co https://dist.apache.org/repos/dist/dev/iceberg/

Re: [I] Tracking issues of Iceberg Rust 0.5.0 Release (May 2025) [iceberg-rust]

2025-05-26 Thread via GitHub
kevinjqliu commented on issue #1325: URL: https://github.com/apache/iceberg-rust/issues/1325#issuecomment-2911096775 > Do we need another vote for 0.5.1? yep, we should push a new tag for `v0.5.1-rc.1` https://github.com/apache/iceberg-rust/releases/tag/v0.5.1-rc.1 Tr

Re: [PR] Bump iceberg-rust version to 0.5.1 [iceberg-rust]

2025-05-26 Thread via GitHub
liurenjie1024 merged PR #1380: URL: https://github.com/apache/iceberg-rust/pull/1380 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@

Re: [I] Tracking issues of Iceberg Rust 0.5.0 Release (May 2025) [iceberg-rust]

2025-05-26 Thread via GitHub
kevinjqliu commented on issue #1325: URL: https://github.com/apache/iceberg-rust/issues/1325#issuecomment-2910986496 @liurenjie1024 should we yank pyiceberg-core 0.5.0 (https://pypi.org/project/pyiceberg-core/0.5.0/) since its not an official release? I dont have the proper permission t

Re: [PR] Bump iceberg-rust version to 0.5.1 [iceberg-rust]

2025-05-26 Thread via GitHub
kevinjqliu commented on PR #1380: URL: https://github.com/apache/iceberg-rust/pull/1380#issuecomment-2911043321 Double checked publishing: ``` cd crates/iceberg cargo publish --all-features --dry-run ``` ran successfully -- This is an automated message from the Apache Git Se

Re: [I] Tracking issues of Iceberg Rust 0.5.0 Release (May 2025) [iceberg-rust]

2025-05-26 Thread via GitHub
kevinjqliu commented on issue #1325: URL: https://github.com/apache/iceberg-rust/issues/1325#issuecomment-2911035129 Bump iceberg-rust version to 0.5.1 #1380 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] fix 0.5.x release `cargo publish` [iceberg-rust]

2025-05-26 Thread via GitHub
kevinjqliu merged PR #1379: URL: https://github.com/apache/iceberg-rust/pull/1379 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ice

Re: [PR] Bump iceberg-rust version to 0.5.0 [iceberg-rust]

2025-05-26 Thread via GitHub
kevinjqliu commented on PR #1345: URL: https://github.com/apache/iceberg-rust/pull/1345#issuecomment-2911019657 thanks for the review @liurenjie1024 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] chore(deps): Bump uuid from 1.16.0 to 1.17.0 [iceberg-rust]

2025-05-26 Thread via GitHub
liurenjie1024 merged PR #1375: URL: https://github.com/apache/iceberg-rust/pull/1375 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@

Re: [I] Tracking issues of Iceberg Rust 0.5.0 Release (May 2025) [iceberg-rust]

2025-05-26 Thread via GitHub
liurenjie1024 commented on issue #1325: URL: https://github.com/apache/iceberg-rust/issues/1325#issuecomment-2910979921 > `iceberg_catalog_memory` is used in `crates/iceberg`'s docstring. For example, > > [iceberg-rust/crates/iceberg/src/lib.rs](https://github.com/apa

Re: [I] Tracking issues of Iceberg Rust 0.5.0 Release (May 2025) [iceberg-rust]

2025-05-26 Thread via GitHub
kevinjqliu commented on issue #1325: URL: https://github.com/apache/iceberg-rust/issues/1325#issuecomment-2910995021 @liurenjie1024 / @Xuanwo / @sungwy are any of you owners for https://pypi.org/project/pyiceberg-core/? Only owners can yank a release -- This is an automated message from

Re: [PR] fix 0.5.x release `cargo publish` [iceberg-rust]

2025-05-26 Thread via GitHub
liurenjie1024 commented on code in PR #1379: URL: https://github.com/apache/iceberg-rust/pull/1379#discussion_r2108084363 ## crates/iceberg/src/lib.rs: ## @@ -21,7 +21,9 @@ //! //! ## Scan A Table //! -//! ```rust, no_run +//! ```rust, ignore Review Comment: Yes, it's fin

Re: [I] Tracking issues of Iceberg Rust 0.5.0 Release (May 2025) [iceberg-rust]

2025-05-26 Thread via GitHub
kevinjqliu commented on issue #1325: URL: https://github.com/apache/iceberg-rust/issues/1325#issuecomment-2910985626 #1379 to fix forward the issue and release under 0.5.1 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

Re: [PR] fix 0.5.x release `cargo publish` [iceberg-rust]

2025-05-26 Thread via GitHub
kevinjqliu commented on code in PR #1379: URL: https://github.com/apache/iceberg-rust/pull/1379#discussion_r2108078879 ## crates/iceberg/src/lib.rs: ## @@ -21,7 +21,9 @@ //! //! ## Scan A Table //! -//! ```rust, no_run +//! ```rust, ignore Review Comment: @liurenjie1024 w

Re: [PR] chore(deps): Bump aws-sdk-glue from 1.94.0 to 1.97.0 [iceberg-rust]

2025-05-26 Thread via GitHub
liurenjie1024 merged PR #1376: URL: https://github.com/apache/iceberg-rust/pull/1376 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@

Re: [PR] static table metadata access support [iceberg-cpp]

2025-05-26 Thread via GitHub
lishuxu closed pull request #111: static table metadata access support URL: https://github.com/apache/iceberg-cpp/pull/111 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

Re: [I] Tracking issues of Iceberg Rust 0.5.0 Release (May 2025) [iceberg-rust]

2025-05-26 Thread via GitHub
kevinjqliu commented on issue #1325: URL: https://github.com/apache/iceberg-rust/issues/1325#issuecomment-2910963966 `iceberg_catalog_memory` is used in `crates/iceberg`'s docstring. For example, https://github.com/apache/iceberg-rust/blob/fbf9b92e0a201d1e37d74a568687174eac823539/crates/ice

Re: [PR] Flink: Migrate Flink TableSchema for IcebergSource [iceberg]

2025-05-26 Thread via GitHub
liamzwbao commented on code in PR #13072: URL: https://github.com/apache/iceberg/pull/13072#discussion_r2108061992 ## flink/v2.0/flink/src/main/java/org/apache/iceberg/flink/FlinkSchemaUtil.java: ## @@ -192,7 +222,9 @@ public static Type convert(LogicalType flinkType) { *

Re: [PR] fix 0.5.x release `cargo publish` [iceberg-rust]

2025-05-26 Thread via GitHub
liurenjie1024 commented on PR #1379: URL: https://github.com/apache/iceberg-rust/pull/1379#issuecomment-2910950841 The ut failure is caused by generating doc. I think we could remove those docs for now. -- This is an automated message from the Apache Git Service. To respond to the message

Re: [PR] Spec: add sort order to spec [iceberg]

2025-05-26 Thread via GitHub
Nicholasjl commented on PR #2055: URL: https://github.com/apache/iceberg/pull/2055#issuecomment-2910943224 Hi, I noticed that in IceNerg, the expression -0 < 0 evaluates to true. However, according to the IEEE 754 standard, -0.0 and 0.0 are considered numerically equal, and -0.0 < 0.

Re: [I] Tracking issues of Iceberg Rust 0.5.0 Release (May 2025) [iceberg-rust]

2025-05-26 Thread via GitHub
liurenjie1024 commented on issue #1325: URL: https://github.com/apache/iceberg-rust/issues/1325#issuecomment-2910937265 Seems we need a way to detect all crates. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[PR] fix 0.5.x release `cargo publish` [iceberg-rust]

2025-05-26 Thread via GitHub
kevinjqliu opened a new pull request, #1379: URL: https://github.com/apache/iceberg-rust/pull/1379 ## Which issue does this PR close? - Closes #. ## What changes are included in this PR? This PR removes `iceberg-catalog-memory` as a dev-dependency of `crates/

[PR] static table metadata access support [iceberg-cpp]

2025-05-26 Thread via GitHub
lishuxu opened a new pull request, #111: URL: https://github.com/apache/iceberg-cpp/pull/111 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-

[I] Flink: add IcebergConnectHiveDelegationTokenProvider [iceberg]

2025-05-26 Thread via GitHub
zhangwl9 opened a new issue, #13159: URL: https://github.com/apache/iceberg/issues/13159 ### Feature Request / Improvement test ### Query engine Flink ### Willingness to contribute - [x] I can contribute this improvement/feature independently - [x] I woul

Re: [I] Add IcebergHiveConnectorDelegationTokenProvider for Iceberg [iceberg]

2025-05-26 Thread via GitHub
zhangwl9 commented on issue #13158: URL: https://github.com/apache/iceberg/issues/13158#issuecomment-2910909964 - [ ] Add IcebergConnectHiveDelegationTokenProvider for HiveCatalog in Iceberg_Spark module - [ ] Add IcebergConnectHiveDelegationTokenProvider for HiveCatalog in Iceberg_Flink

[I] Add IcebergHiveConnectorDelegationTokenProvider for Iceberg [iceberg]

2025-05-26 Thread via GitHub
zhangwl9 opened a new issue, #13158: URL: https://github.com/apache/iceberg/issues/13158 ### Feature Request / Improvement I want to implement both Spark and the Flink delegation token supported by the HiveCatalog; ### Query engine None ### Willingness to cont

Re: [I] Spark: add IcebergConnectHiveDelegationTokenProvider [iceberg]

2025-05-26 Thread via GitHub
zhangwl9 commented on issue #13116: URL: https://github.com/apache/iceberg/issues/13116#issuecomment-2910897592 @pvary @gaborgsomogyi For Spark , I'm going to use the scheme shown above to implement an IcebergHiveConnectorDelegationTokenProvider within the Iceberg_Spark module

Re: [PR] feat: support decompress gzip metadata [iceberg-cpp]

2025-05-26 Thread via GitHub
wgtmac commented on code in PR #108: URL: https://github.com/apache/iceberg-cpp/pull/108#discussion_r2108002007 ## src/iceberg/util/gzip_internal.cc: ## @@ -0,0 +1,94 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements

Re: [I] Tracking issues of Iceberg Rust 0.5.0 Release (May 2025) [iceberg-rust]

2025-05-26 Thread via GitHub
kevinjqliu commented on issue #1325: URL: https://github.com/apache/iceberg-rust/issues/1325#issuecomment-2910894995 looks like `pyiceberg-core 0.5.0` already published on pypi https://pypi.org/project/pyiceberg-core/0.5.0/ I think we need to yank this, fix the underlying issue and

Re: [I] Tracking issues of Iceberg Rust 0.5.0 Release (May 2025) [iceberg-rust]

2025-05-26 Thread via GitHub
kevinjqliu commented on issue #1325: URL: https://github.com/apache/iceberg-rust/issues/1325#issuecomment-2910809436 hm, these are hardcoded. should we add more packages here? https://github.com/apache/iceberg-rust/blob/fbf9b92e0a201d1e37d74a568687174eac823539/.github/workflows/publi

Re: [I] ADLSFileIO cache DefaultAzureCredentials? [iceberg]

2025-05-26 Thread via GitHub
github-actions[bot] closed issue #11523: ADLSFileIO cache DefaultAzureCredentials? URL: https://github.com/apache/iceberg/issues/11523 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

Re: [PR] Avoid Avro recursive schema for Variant schema. [iceberg]

2025-05-26 Thread via GitHub
github-actions[bot] commented on PR #12459: URL: https://github.com/apache/iceberg/pull/12459#issuecomment-2910801756 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

Re: [PR] Avoid Avro recursive schema for Variant schema. [iceberg]

2025-05-26 Thread via GitHub
github-actions[bot] closed pull request #12459: Avoid Avro recursive schema for Variant schema. URL: https://github.com/apache/iceberg/pull/12459 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [I] Enhance `catalog.create_table` API to enable creation of table with matching `field_ids` to provided Schema [iceberg-python]

2025-05-26 Thread via GitHub
github-actions[bot] commented on issue #1284: URL: https://github.com/apache/iceberg-python/issues/1284#issuecomment-2910804096 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity

Re: [I] How to run streaming upserts and maintenance simultaneously? [iceberg]

2025-05-26 Thread via GitHub
github-actions[bot] closed issue #11530: How to run streaming upserts and maintenance simultaneously? URL: https://github.com/apache/iceberg/issues/11530 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [I] How to run streaming upserts and maintenance simultaneously? [iceberg]

2025-05-26 Thread via GitHub
github-actions[bot] commented on issue #11530: URL: https://github.com/apache/iceberg/issues/11530#issuecomment-2910801666 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache

Re: [PR] Spark: SnapshotTableSparkAction add validation for non-overlapping source/dest table paths. [iceberg]

2025-05-26 Thread via GitHub
github-actions[bot] commented on PR #12779: URL: https://github.com/apache/iceberg/pull/12779#issuecomment-2910801791 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

Re: [PR] Spark: SnapshotTableSparkAction add validation for non-overlapping source/dest table paths. [iceberg]

2025-05-26 Thread via GitHub
github-actions[bot] closed pull request #12779: Spark: SnapshotTableSparkAction add validation for non-overlapping source/dest table paths. URL: https://github.com/apache/iceberg/pull/12779 -- This is an automated message from the Apache Git Service. To respond to the message, please log on t

Re: [I] java.io.IOException: can not read class org.apache.iceberg.shaded.org.apache.parquet.format.PageHeader: Required field 'num_values' was not found in serialized data [iceberg]

2025-05-26 Thread via GitHub
github-actions[bot] commented on issue #11614: URL: https://github.com/apache/iceberg/issues/11614#issuecomment-2910801702 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [I] ADLSFileIO cache DefaultAzureCredentials? [iceberg]

2025-05-26 Thread via GitHub
github-actions[bot] commented on issue #11523: URL: https://github.com/apache/iceberg/issues/11523#issuecomment-2910801639 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache

Re: [I] Tracking issues of Iceberg Rust 0.5.0 Release (May 2025) [iceberg-rust]

2025-05-26 Thread via GitHub
kevinjqliu commented on issue #1325: URL: https://github.com/apache/iceberg-rust/issues/1325#issuecomment-2910783263 I was able to verify `cargo publish` can run successfully with the `iceberg-catalog-memory` dev dependency commented out ``` cargo publish --all-features --dry-run --all

Re: [I] Tracking issues of Iceberg Rust 0.5.0 Release (May 2025) [iceberg-rust]

2025-05-26 Thread via GitHub
kevinjqliu commented on issue #1325: URL: https://github.com/apache/iceberg-rust/issues/1325#issuecomment-2910782152 https://rust.iceberg.apache.org/release.html#official-release ``` git checkout v0.5.0-rc.2 git tag -s "v0.5.0" git push origin "v0.5.0" ``` Tag push trigg

Re: [I] Tracking issues of Iceberg Rust 0.5.0 Release (May 2025) [iceberg-rust]

2025-05-26 Thread via GitHub
kevinjqliu commented on issue #1325: URL: https://github.com/apache/iceberg-rust/issues/1325#issuecomment-2910764412 vote passed! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

[PR] Build: Bump moto from 5.1.4 to 5.1.5 [iceberg-python]

2025-05-26 Thread via GitHub
dependabot[bot] opened a new pull request, #2039: URL: https://github.com/apache/iceberg-python/pull/2039 Bumps [moto](https://github.com/getmoto/moto) from 5.1.4 to 5.1.5. Changelog Sourced from https://github.com/getmoto/moto/blob/master/CHANGELOG.md";>moto's changelog. 5.

[PR] Build: Bump mypy-boto3-glue from 1.38.18 to 1.38.22 [iceberg-python]

2025-05-26 Thread via GitHub
dependabot[bot] opened a new pull request, #2038: URL: https://github.com/apache/iceberg-python/pull/2038 Bumps [mypy-boto3-glue](https://github.com/youtype/mypy_boto3_builder) from 1.38.18 to 1.38.22. Release notes Sourced from https://github.com/youtype/mypy_boto3_builder/release

[PR] Build: Bump mkdocs-autorefs from 1.4.1 to 1.4.2 [iceberg-python]

2025-05-26 Thread via GitHub
dependabot[bot] opened a new pull request, #2048: URL: https://github.com/apache/iceberg-python/pull/2048 Bumps [mkdocs-autorefs](https://github.com/mkdocstrings/autorefs) from 1.4.1 to 1.4.2. Release notes Sourced from https://github.com/mkdocstrings/autorefs/releases";>mkdocs-aut

[PR] Build: Bump coverage from 7.8.0 to 7.8.2 [iceberg-python]

2025-05-26 Thread via GitHub
dependabot[bot] opened a new pull request, #2047: URL: https://github.com/apache/iceberg-python/pull/2047 Bumps [coverage](https://github.com/nedbat/coveragepy) from 7.8.0 to 7.8.2. Changelog Sourced from https://github.com/nedbat/coveragepy/blob/master/CHANGES.rst";>coverage's cha

[PR] Build: Bump huggingface-hub from 0.31.4 to 0.32.1 [iceberg-python]

2025-05-26 Thread via GitHub
dependabot[bot] opened a new pull request, #2046: URL: https://github.com/apache/iceberg-python/pull/2046 Bumps [huggingface-hub](https://github.com/huggingface/huggingface_hub) from 0.31.4 to 0.32.1. Release notes Sourced from https://github.com/huggingface/huggingface_hub/release

[PR] Build: Bump mkdocstrings-python from 1.16.10 to 1.16.11 [iceberg-python]

2025-05-26 Thread via GitHub
dependabot[bot] opened a new pull request, #2044: URL: https://github.com/apache/iceberg-python/pull/2044 Bumps [mkdocstrings-python](https://github.com/mkdocstrings/python) from 1.16.10 to 1.16.11. Release notes Sourced from https://github.com/mkdocstrings/python/releases";>mkdocs

[PR] Build: Bump pydantic from 2.11.4 to 2.11.5 [iceberg-python]

2025-05-26 Thread via GitHub
dependabot[bot] opened a new pull request, #2045: URL: https://github.com/apache/iceberg-python/pull/2045 Bumps [pydantic](https://github.com/pydantic/pydantic) from 2.11.4 to 2.11.5. Release notes Sourced from https://github.com/pydantic/pydantic/releases";>pydantic's releases.

[PR] Build: Bump thrift from 0.21.0 to 0.22.0 [iceberg-python]

2025-05-26 Thread via GitHub
dependabot[bot] opened a new pull request, #2043: URL: https://github.com/apache/iceberg-python/pull/2043 Bumps [thrift](https://github.com/apache/thrift) from 0.21.0 to 0.22.0. Release notes Sourced from https://github.com/apache/thrift/releases";>thrift's releases. Version

[PR] Build: Bump getdaft from 0.4.15 to 0.4.16 [iceberg-python]

2025-05-26 Thread via GitHub
dependabot[bot] opened a new pull request, #2042: URL: https://github.com/apache/iceberg-python/pull/2042 Bumps [getdaft](https://github.com/Eventual-Inc/Daft) from 0.4.15 to 0.4.16. Release notes Sourced from https://github.com/Eventual-Inc/Daft/releases";>getdaft's releases.

[PR] Build: Bump cachetools from 5.5.2 to 6.0.0 [iceberg-python]

2025-05-26 Thread via GitHub
dependabot[bot] opened a new pull request, #2041: URL: https://github.com/apache/iceberg-python/pull/2041 Bumps [cachetools](https://github.com/tkem/cachetools) from 5.5.2 to 6.0.0. Changelog Sourced from https://github.com/tkem/cachetools/blob/master/CHANGELOG.rst";>cachetools's c

[PR] Build: Bump pytest-mock from 3.14.0 to 3.14.1 [iceberg-python]

2025-05-26 Thread via GitHub
dependabot[bot] opened a new pull request, #2040: URL: https://github.com/apache/iceberg-python/pull/2040 Bumps [pytest-mock](https://github.com/pytest-dev/pytest-mock) from 3.14.0 to 3.14.1. Release notes Sourced from https://github.com/pytest-dev/pytest-mock/releases";>pytest-moc

Re: [PR] feat(catalog/sql): Add views related ops [iceberg-go]

2025-05-26 Thread via GitHub
dttung2905 commented on PR #414: URL: https://github.com/apache/iceberg-go/pull/414#issuecomment-2910693026 > just a few comments but overall looks good to me so far. > > Is there any way we can set up an integration test to compare against other implementations? @zeroshade Tha

Re: [PR] feat(catalog/sql): Add views related ops [iceberg-go]

2025-05-26 Thread via GitHub
zeroshade commented on PR #414: URL: https://github.com/apache/iceberg-go/pull/414#issuecomment-2910706995 A follow up PR for the integration test is fine. If you're not gonna start immediately after this gets merged, then let's file an issue to track it so we don't forget. But if you'll be

[I] Improve performance of filter pre-processing [iceberg-python]

2025-05-26 Thread via GitHub
ypsah opened a new issue, #2037: URL: https://github.com/apache/iceberg-python/issues/2037 ### Feature Request / Improvement Hi, apologies in advance for the rather broad scope. I am using `pyiceberg` to store datasets partitioned in two dimensions and am using basic filtering

Re: [I] Iceberg + MinIO S3 - Invalid signature after 3 hours [iceberg]

2025-05-26 Thread via GitHub
omurad commented on issue #13045: URL: https://github.com/apache/iceberg/issues/13045#issuecomment-2910467944 @c-thiel > This ensures automatic background token refreshing, addressing a known limitation of the built-in manager https://www.dremio.com/blog/introducing-dremio-au

[I] Add iam-policy-builder into iceberg-aws-bundle [iceberg]

2025-05-26 Thread via GitHub
drnta opened a new issue, #13154: URL: https://github.com/apache/iceberg/issues/13154 ### Feature Request / Improvement It can be convenient to have [iam-policy-builder](https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/feature-iam-policy-builder.html) available in `ice

[I] Support for Variant PhysicalType.TIME, TIMESTAMPNTZ_NANO, TIMESTAMPTZ_NANO & UUID types in InclusiveMetricsEvaluator [iceberg]

2025-05-26 Thread via GitHub
manirajv06 opened a new issue, #13157: URL: https://github.com/apache/iceberg/issues/13157 ### Feature Request / Improvement VariantExpressionUtil doesn't support Variant fields of below data types: PhysicalType.TIME PhysicalType.TIMESTAMPNTZ_NANO PhysicalType.TIMESTAMPTZ_

Re: [I] Support for Variant PhysicalType.TIME, TIMESTAMPNTZ_NANO, TIMESTAMPTZ_NANO & UUID types in InclusiveMetricsEvaluator [iceberg]

2025-05-26 Thread via GitHub
manirajv06 commented on issue #13157: URL: https://github.com/apache/iceberg/issues/13157#issuecomment-2910323069 @aihuaxu Working on this. Will raise a pr soon. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] Incremental Append Scan [iceberg-python]

2025-05-26 Thread via GitHub
smaheshwar-pltr commented on code in PR #2031: URL: https://github.com/apache/iceberg-python/pull/2031#discussion_r2102769661 ## pyiceberg/table/__init__.py: ## @@ -1688,102 +1887,252 @@ def _match_deletes_to_data_file(data_entry: ManifestEntry, positional_delete_ent r

Re: [PR] Flink: Support compact in iceberg sink v2 [iceberg]

2025-05-26 Thread via GitHub
Guosmilesmile commented on code in PR #12979: URL: https://github.com/apache/iceberg/pull/12979#discussion_r2107355078 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/sink/CommittableToTableChangeConverter.java: ## @@ -0,0 +1,181 @@ +/* + * Licensed to the Apache Sof

Re: [PR] Fix UUID support [iceberg-python]

2025-05-26 Thread via GitHub
simw commented on PR #2007: URL: https://github.com/apache/iceberg-python/pull/2007#issuecomment-2910181483 Following issue #1986 , I was about to make a smaller PR without the knowledge of the extra spark-related complications. In case it's useful, the only extra thing I had that you

Re: [I] Error creating table from pyarrow schema with pa.uuid() [iceberg-python]

2025-05-26 Thread via GitHub
simw commented on issue #1986: URL: https://github.com/apache/iceberg-python/issues/1986#issuecomment-2910156608 This would be fixed by #2007 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [I] Iceberg + MinIO S3 - Invalid signature after 3 hours [iceberg]

2025-05-26 Thread via GitHub
c-thiel commented on issue #13045: URL: https://github.com/apache/iceberg/issues/13045#issuecomment-2910045109 Where do you find the information that this is a known issue? Did you find an existing ticket? -- This is an automated message from the Apache Git Service. To respond to the mess

Re: [PR] Docs: add Tinybird to the list of vendors and blog posts [iceberg]

2025-05-26 Thread via GitHub
nastra merged PR #13128: URL: https://github.com/apache/iceberg/pull/13128 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Flink: Support compact in iceberg sink v2 [iceberg]

2025-05-26 Thread via GitHub
Guosmilesmile commented on code in PR #12979: URL: https://github.com/apache/iceberg/pull/12979#discussion_r2107355078 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/sink/CommittableToTableChangeConverter.java: ## @@ -0,0 +1,181 @@ +/* + * Licensed to the Apache Sof

Re: [PR] Flink: Support compact in iceberg sink v2 [iceberg]

2025-05-26 Thread via GitHub
Guosmilesmile commented on code in PR #12979: URL: https://github.com/apache/iceberg/pull/12979#discussion_r2107406941 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/sink/CommittableToTableChangeConverter.java: ## @@ -0,0 +1,181 @@ +/* + * Licensed to the Apache Sof

Re: [PR] Docs: add Tinybird to the list of vendors and blog posts [iceberg]

2025-05-26 Thread via GitHub
futurepastori commented on code in PR #13128: URL: https://github.com/apache/iceberg/pull/13128#discussion_r2107403321 ## site/docs/vendors.md: ## @@ -124,10 +124,18 @@ Starburst is a commercial offering for the [Trino query engine](https://trino.io [Tabular](https://tabular

Re: [PR] Flink: Support compact in iceberg sink v2 [iceberg]

2025-05-26 Thread via GitHub
Guosmilesmile commented on PR #12979: URL: https://github.com/apache/iceberg/pull/12979#issuecomment-2909857289 > LGTM > > Are we going to add the options in [#12979 (comment)](https://github.com/apache/iceberg/pull/12979#issuecomment-2854062247) to the docs? @mxm I think we

Re: [PR] Flink: Support compact in iceberg sink v2 [iceberg]

2025-05-26 Thread via GitHub
Guosmilesmile commented on code in PR #12979: URL: https://github.com/apache/iceberg/pull/12979#discussion_r2107410468 ## flink/v1.20/flink/src/test/java/org/apache/iceberg/flink/TestFlinkTableSinkCompaction.java: ## @@ -0,0 +1,141 @@ +/* + * Licensed to the Apache Software Foun

Re: [PR] Flink: Support compact in iceberg sink v2 [iceberg]

2025-05-26 Thread via GitHub
Guosmilesmile commented on code in PR #12979: URL: https://github.com/apache/iceberg/pull/12979#discussion_r2107399568 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/maintenance/api/TableMaintenance.java: ## @@ -59,6 +59,12 @@ public class TableMaintenance { stat

Re: [I] Incrementally computing partition stats can miss deleted files [iceberg]

2025-05-26 Thread via GitHub
lirui-apache commented on issue #13155: URL: https://github.com/apache/iceberg/issues/13155#issuecomment-2909799838 > > I think when a snapshot is being created, previous manifests without live entries will not be tracked. So maybe we have to apply the incremental snapshots one by one? And

Re: [PR] Flink: Support compact in iceberg sink v2 [iceberg]

2025-05-26 Thread via GitHub
Guosmilesmile commented on code in PR #12979: URL: https://github.com/apache/iceberg/pull/12979#discussion_r2107410807 ## flink/v1.20/flink/src/test/java/org/apache/iceberg/flink/maintenance/operator/TestZkLockFactoryBuilder.java: ## @@ -0,0 +1,77 @@ +/* + * Licensed to the Apac

Re: [PR] Flink: Support compact in iceberg sink v2 [iceberg]

2025-05-26 Thread via GitHub
Guosmilesmile commented on code in PR #12979: URL: https://github.com/apache/iceberg/pull/12979#discussion_r2107396941 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/maintenance/api/LockConfig.java: ## @@ -0,0 +1,46 @@ +/* + * Licensed to the Apache Software Foundat

Re: [PR] Flink: Support compact in iceberg sink v2 [iceberg]

2025-05-26 Thread via GitHub
Guosmilesmile commented on code in PR #12979: URL: https://github.com/apache/iceberg/pull/12979#discussion_r2107409024 ## flink/v1.20/flink/src/test/java/org/apache/iceberg/flink/TestFlinkTableSinkCompaction.java: ## @@ -0,0 +1,141 @@ +/* + * Licensed to the Apache Software Foun

Re: [PR] Flink: Support compact in iceberg sink v2 [iceberg]

2025-05-26 Thread via GitHub
Guosmilesmile commented on code in PR #12979: URL: https://github.com/apache/iceberg/pull/12979#discussion_r2107408488 ## flink/v1.20/flink/src/test/java/org/apache/iceberg/flink/TestFlinkTableSinkCompaction.java: ## @@ -0,0 +1,141 @@ +/* + * Licensed to the Apache Software Foun

Re: [PR] Flink: Support compact in iceberg sink v2 [iceberg]

2025-05-26 Thread via GitHub
Guosmilesmile commented on code in PR #12979: URL: https://github.com/apache/iceberg/pull/12979#discussion_r2107407959 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/sink/IcebergSink.java: ## @@ -216,7 +235,44 @@ public SimpleVersionedSerializer getCommittableSeria

Re: [PR] Flink: Support compact in iceberg sink v2 [iceberg]

2025-05-26 Thread via GitHub
Guosmilesmile commented on code in PR #12979: URL: https://github.com/apache/iceberg/pull/12979#discussion_r2107407568 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/sink/IcebergSink.java: ## @@ -216,7 +235,44 @@ public SimpleVersionedSerializer getCommittableSeria

Re: [PR] Flink: Support compact in iceberg sink v2 [iceberg]

2025-05-26 Thread via GitHub
Guosmilesmile commented on code in PR #12979: URL: https://github.com/apache/iceberg/pull/12979#discussion_r2107402061 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/maintenance/api/TableMaintenanceConfig.java: ## @@ -0,0 +1,61 @@ +/* + * Licensed to the Apache Soft

Re: [PR] Flink: Support compact in iceberg sink v2 [iceberg]

2025-05-26 Thread via GitHub
Guosmilesmile commented on code in PR #12979: URL: https://github.com/apache/iceberg/pull/12979#discussion_r2107399213 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/maintenance/api/RewriteDataFilesConfig.java: ## @@ -0,0 +1,100 @@ +/* + * Licensed to the Apache Sof

Re: [PR] Flink: Support compact in iceberg sink v2 [iceberg]

2025-05-26 Thread via GitHub
Guosmilesmile commented on code in PR #12979: URL: https://github.com/apache/iceberg/pull/12979#discussion_r2107397429 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/sink/CommittableToTableChangeConverter.java: ## @@ -0,0 +1,181 @@ +/* + * Licensed to the Apache Sof

Re: [I] Incrementally computing partition stats can miss deleted files [iceberg]

2025-05-26 Thread via GitHub
ajantha-bhat commented on issue #13155: URL: https://github.com/apache/iceberg/issues/13155#issuecomment-2909762512 > I think when a snapshot is being created, previous manifests without live entries will not be tracked. So maybe we have to apply the incremental snapshots one by one? And pr

Re: [I] Incrementally computing partition stats can miss deleted files [iceberg]

2025-05-26 Thread via GitHub
lirui-apache commented on issue #13155: URL: https://github.com/apache/iceberg/issues/13155#issuecomment-2909728763 I think when a snapshot is being created, previous manifests without live entries will not be tracked. So maybe we have to apply the incremental snapshots one by one? And prob

Re: [PR] Flink: Support compact in iceberg sink v2 [iceberg]

2025-05-26 Thread via GitHub
Guosmilesmile commented on code in PR #12979: URL: https://github.com/apache/iceberg/pull/12979#discussion_r2107355078 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/sink/CommittableToTableChangeConverter.java: ## @@ -0,0 +1,181 @@ +/* + * Licensed to the Apache Sof

Re: [PR] chore(deps): Bump aws-sdk-s3tables from 1.20.0 to 1.22.0 [iceberg-rust]

2025-05-26 Thread via GitHub
liurenjie1024 merged PR #1377: URL: https://github.com/apache/iceberg-rust/pull/1377 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@

Re: [PR] Feature/write to branch [iceberg-python]

2025-05-26 Thread via GitHub
SebastienN15 closed pull request #2009: Feature/write to branch URL: https://github.com/apache/iceberg-python/pull/2009 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsub

Re: [PR] OpenAPI: Add missing schema field for TableMetadata [iceberg]

2025-05-26 Thread via GitHub
Bhargavkonidena commented on PR #13152: URL: https://github.com/apache/iceberg/pull/13152#issuecomment-2909573917 Sure @nastra will send out a mail to [d...@iceberg.apache.org](mailto:d...@iceberg.apache.org) accordingly. -- This is an automated message from the Apache Git Service. To res

Re: [I] Add docs of Spark SQL functions for Iceberg transforms [iceberg]

2025-05-26 Thread via GitHub
VerneZhong commented on issue #13156: URL: https://github.com/apache/iceberg/issues/13156#issuecomment-2909567379 Hi, I’d like to work on this documentation issue. I can contribute with some guidance from the Iceberg community. -- This is an automated message from the Apache Git Servi

Re: [I] Spark: add IcebergConnectHiveDelegationTokenProvider [iceberg]

2025-05-26 Thread via GitHub
gaborgsomogyi commented on issue #13116: URL: https://github.com/apache/iceberg/issues/13116#issuecomment-2909026174 This can decrease significanly the number of authentication requests allowing to deploy more scalable applications. -- This is an automated message from the Apache Git Serv

Re: [PR] feat: implement initial MemoryCatalog functionality with namespace and table support [iceberg-cpp]

2025-05-26 Thread via GitHub
gty404 commented on PR #80: URL: https://github.com/apache/iceberg-cpp/pull/80#issuecomment-2909485546 @Xuanwo @Fokko Could you help review this? Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

  1   2   >