Re: [PR] Spark3.4: Enable Native execution if ParquetReaderType is Comet [iceberg]

huaxingao commented on PR #12709: URL: https://github.com/apache/iceberg/pull/12709#issuecomment-2893169619 @jiayuasu Yes, I will be working on this. One issue I'm currently facing is the shading issue with Iceberg. Since Iceberg shades Parquet, I can't pass Parquet objects and types from I

Re: [I] Tracking issues of Iceberg Rust 0.5.0 Release (May 2025) [iceberg-rust]

liurenjie1024 commented on issue #1325: URL: https://github.com/apache/iceberg-rust/issues/1325#issuecomment-2893125730 > Hi, I'm guessing we didn't finish the release process yet? Yes, it's the `close ...` pr description which closed it automatically. -- This is an automated messag

Re: [I] Tracking issues of Iceberg Rust 0.5.0 Release (May 2025) [iceberg-rust]

Xuanwo commented on issue #1325: URL: https://github.com/apache/iceberg-rust/issues/1325#issuecomment-2893104356 Hi, I'm guessing we didn't finish the release process yet? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and us

Re: [I] Add column projection using `schema.name-mapping.default` [iceberg-rust]

liurenjie1024 commented on issue #1335: URL: https://github.com/apache/iceberg-rust/issues/1335#issuecomment-2893093013 > Hey Kevin, I can pick this up if you're okay with it! Feel free to submit pr! -- This is an automated message from the Apache Git Service. To respond to the mess

Re: [I] Tracking issues of Iceberg Rust 0.5.0 Release (May 2025) [iceberg-rust]

liurenjie1024 closed issue #1325: Tracking issues of Iceberg Rust 0.5.0 Release (May 2025) URL: https://github.com/apache/iceberg-rust/issues/1325 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] Bump iceberg-rust version to 0.5.0 [iceberg-rust]

liurenjie1024 merged PR #1345: URL: https://github.com/apache/iceberg-rust/pull/1345 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@

Re: [I] Make FileIO a Trait [iceberg-rust]

tustvold commented on issue #1314: URL: https://github.com/apache/iceberg-rust/issues/1314#issuecomment-2893032985 > I still recommend wrapping it in a struct when passing around in the crate So are you proposing making Storage a trait instead? If so for the sake of argument, could th

Re: [I] Partition Query with more details for spark SQL [iceberg]

Shekharrajak commented on issue #13079: URL: https://github.com/apache/iceberg/issues/13079#issuecomment-2893039302 @RussellSpitzer , I need to find out through schema for the partitions keys and then map it into the query. For example I have partition key as `dt=<>/hhmm=<>` then I need to

Re: [PR] Bump iceberg-rust version to 0.5.0 [iceberg-rust]

kevinjqliu commented on PR #1345: URL: https://github.com/apache/iceberg-rust/pull/1345#issuecomment-2893006769 Added `CHANGELOG.md` and `DEPENDENCIES.rust.tsv` Feel free to merge this and i'll push the RC tag once its merge :) -- This is an automated message from the Apache Git Ser

Re: [I] [REST Catalog] OAuth 2 grant type "refresh_token" not implemented [iceberg]

nika-qubit commented on issue #12196: URL: https://github.com/apache/iceberg/issues/12196#issuecomment-2892959543 Hi @adutra, do we have any plan to support the authorization code flow in the new AuthManager 😄 ? Picking up the context from https://github.com/apache/iceberg/pull/12362#issuec

Re: [PR] Spark 4.0: Reapply changes removed during the Spark 4.0 reversion [iceberg]

amogh-jahagirdar merged PR #13096: URL: https://github.com/apache/iceberg/pull/13096 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@

Re: [PR] Spark3.4: Enable Native execution if ParquetReaderType is Comet [iceberg]

jiayuasu commented on PR #12709: URL: https://github.com/apache/iceberg/pull/12709#issuecomment-2892788593 @huaxingao Is this PR still active? We are actually really looking forward to having this for comet! -- This is an automated message from the Apache Git Service. To respond to the me

Re: [PR] Bump iceberg-rust version to 0.5.0 [iceberg-rust]

liurenjie1024 commented on PR #1345: URL: https://github.com/apache/iceberg-rust/pull/1345#issuecomment-289276 cc @kevinjqliu Are we going to merge this or we should hold on a while for CHANGELOG.md and DEPENDENCIES.rust.tsv? -- This is an automated message from the Apache Git Service

Re: [I] Make FileIO a Trait [iceberg-rust]

liurenjie1024 commented on issue #1314: URL: https://github.com/apache/iceberg-rust/issues/1314#issuecomment-2892787039 Despite the discussion points raised by @tustvold , I have other things to discuss: 1. Should we have an unified `FileIOBuilder` trait, just like what we did for `Catal

Re: [I] Make FileIO a Trait [iceberg-rust]

liurenjie1024 commented on issue #1314: URL: https://github.com/apache/iceberg-rust/issues/1314#issuecomment-2892773303 Hi, sorry for being late for this party, and thanks @tustvold for the summary of the discussions. > 1. Should implementations pass around Archttps://github.com/apach

Re: [I] Add description of columns for entries metadata table [iceberg]

BhargavKumarKonidena commented on issue #13076: URL: https://github.com/apache/iceberg/issues/13076#issuecomment-2892703280 if its still open, can I work on this issue please? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

Re: [PR] Decouple Committer from Kafka and Enable Custom Coordinator Election [iceberg]

kumarpritam863 commented on PR #12460: URL: https://github.com/apache/iceberg/pull/12460#issuecomment-2892685712 Commenting for keeping the PR alive. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] Spark 3.4: Structured Streaming read limit support follow-up [iceberg]

wypoon commented on PR #13099: URL: https://github.com/apache/iceberg/pull/13099#issuecomment-2892675215 cc @huaxingao @singhpk234 This is a clean backport. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

Re: [I] Modify REST Tests to Bind Loopback instead of Localhost [iceberg]

stevenzwu commented on issue #13097: URL: https://github.com/apache/iceberg/issues/13097#issuecomment-2892663568 The tests fixed in the PR #13101 was using port `0` for random selected port. `RESTCatalogServer` finds a free port, which seems to make the difference. If I change the port conf

[PR] Build: Bump setuptools from 78.1.0 to 78.1.1 [iceberg-python]

dependabot[bot] opened a new pull request, #2027: URL: https://github.com/apache/iceberg-python/pull/2027 Bumps [setuptools](https://github.com/pypa/setuptools) from 78.1.0 to 78.1.1. Changelog Sourced from https://github.com/pypa/setuptools/blob/main/NEWS.rst";>setuptools's change

Re: [I] Modify REST Tests to Bind Loopback instead of Localhost [iceberg]

stevenzwu commented on issue #13097: URL: https://github.com/apache/iceberg/issues/13097#issuecomment-2892609446 Created a PR to apply the suggestion: https://github.com/apache/iceberg/pull/13101 I didn't understand why `RESTCatalogServer` doesn't suffer this problem. It is primarily

[PR] Core, AWS: use loopback address explicitly for jetty server in test. [iceberg]

stevenzwu opened a new pull request, #13101: URL: https://github.com/apache/iceberg/pull/13101 See issue for more details: https://github.com/apache/iceberg/issues/13097 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] Spark 4.0: Reapply changes removed during the Spark 4.0 reversion [iceberg]

tomtongue commented on PR #13096: URL: https://github.com/apache/iceberg/pull/13096#issuecomment-2892596207 @amogh-jahagirdar Thanks for reviewing this! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [I] Unable to read table from Trino after `AppendTable` [iceberg-go]

zeroshade commented on issue #434: URL: https://github.com/apache/iceberg-go/issues/434#issuecomment-2892572711 Hi! Thanks for filling the issue. It turns out that I tracked down a problem today that is likely the same issue as this. Assuming it's the same issue I'll have a fix up for

Re: [I] [Bug] Pyiceberg row filter expression "In" takes longer to query than using "EqualTo" [iceberg-python]

github-actions[bot] commented on issue #1295: URL: https://github.com/apache/iceberg-python/issues/1295#issuecomment-2892561806 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the A

Re: [I] Implement rewriteDataFiles as a Part of metadata management [iceberg-python]

github-actions[bot] commented on issue #1277: URL: https://github.com/apache/iceberg-python/issues/1277#issuecomment-2892561843 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the A

Re: [I] Implement rewriteDataFiles as a Part of metadata management [iceberg-python]

github-actions[bot] closed issue #1277: Implement rewriteDataFiles as a Part of metadata management URL: https://github.com/apache/iceberg-python/issues/1277 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

Re: [I] [Bug] Pyiceberg row filter expression "In" takes longer to query than using "EqualTo" [iceberg-python]

github-actions[bot] closed issue #1295: [Bug] Pyiceberg row filter expression "In" takes longer to query than using "EqualTo" URL: https://github.com/apache/iceberg-python/issues/1295 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

Re: [I] remove warning filter for `datetime.datetime.utcfromtimestamp` [iceberg-python]

github-actions[bot] commented on issue #1349: URL: https://github.com/apache/iceberg-python/issues/1349#issuecomment-2892561767 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity

Re: [I] Including Iceberg Version in metadata json file for better traceability of PendingUpdate [iceberg]

github-actions[bot] closed issue #11471: Including Iceberg Version in metadata json file for better traceability of PendingUpdate URL: https://github.com/apache/iceberg/issues/11471 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [I] Including Iceberg Version in metadata json file for better traceability of PendingUpdate [iceberg]

github-actions[bot] commented on issue #11471: URL: https://github.com/apache/iceberg/issues/11471#issuecomment-2892558327 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache

Re: [I] Add column projection using `schema.name-mapping.default` [iceberg-rust]

kevinjqliu commented on issue #1335: URL: https://github.com/apache/iceberg-rust/issues/1335#issuecomment-2892543670 sure! heres where we apply the name-mapping logic in pyiceberg https://github.com/apache/iceberg-python/blob/main/pyiceberg/io/pyarrow.py#L965-L977 there mig

[I] Unable to read table from Trino after `AppendTable` [iceberg-go]

hendoxc opened a new issue, #434: URL: https://github.com/apache/iceberg-go/issues/434 ### Apache Iceberg version main (development) ### Please describe the bug 🐞 Hey there, amazing project. I have been playing around with writes, specifically writing to a table, and rea

Re: [I] Add column projection using `schema.name-mapping.default` [iceberg-rust]

geruh commented on issue #1335: URL: https://github.com/apache/iceberg-rust/issues/1335#issuecomment-2892486127 Hey Kevin, I can pick this up if you're okay with it! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[I] Partition file filtering logic is incorrect for logical `not()` function [iceberg-rust]

Nathan-Fenner opened a new issue, #1355: URL: https://github.com/apache/iceberg-rust/issues/1355 ### Apache Iceberg Rust version 0.4.0 (latest version) ### Describe the bug Consider the following example table partition file: | `id` | `count` | | -

[PR] Build: Bump fastavro from 1.10.0 to 1.11.1 [iceberg-python]

dependabot[bot] opened a new pull request, #2026: URL: https://github.com/apache/iceberg-python/pull/2026 Bumps [fastavro](https://github.com/fastavro/fastavro) from 1.10.0 to 1.11.1. Changelog Sourced from https://github.com/fastavro/fastavro/blob/master/ChangeLog";>fastavro's cha

[PR] Build: Bump sqlalchemy from 2.0.40 to 2.0.41 [iceberg-python]

dependabot[bot] opened a new pull request, #2025: URL: https://github.com/apache/iceberg-python/pull/2025 Bumps [sqlalchemy](https://github.com/sqlalchemy/sqlalchemy) from 2.0.40 to 2.0.41. Release notes Sourced from https://github.com/sqlalchemy/sqlalchemy/releases";>sqlalchemy's

[PR] Build: Bump mkdocs-material from 9.6.13 to 9.6.14 [iceberg-python]

dependabot[bot] opened a new pull request, #2024: URL: https://github.com/apache/iceberg-python/pull/2024 Bumps [mkdocs-material](https://github.com/squidfunk/mkdocs-material) from 9.6.13 to 9.6.14. Release notes Sourced from https://github.com/squidfunk/mkdocs-material/releases";>

[PR] Build: Bump getdaft from 0.4.14 to 0.4.15 [iceberg-python]

dependabot[bot] opened a new pull request, #2023: URL: https://github.com/apache/iceberg-python/pull/2023 Bumps [getdaft](https://github.com/Eventual-Inc/Daft) from 0.4.14 to 0.4.15. Release notes Sourced from https://github.com/Eventual-Inc/Daft/releases";>getdaft's releases.

[PR] Build: Bump cython from 3.0.12 to 3.1.1 [iceberg-python]

dependabot[bot] opened a new pull request, #2022: URL: https://github.com/apache/iceberg-python/pull/2022 Bumps [cython](https://github.com/cython/cython) from 3.0.12 to 3.1.1. Release notes Sourced from https://github.com/cython/cython/releases";>cython's releases. 3.1.1

[PR] Build: Bump mypy-boto3-glue from 1.38.12 to 1.38.18 [iceberg-python]

dependabot[bot] opened a new pull request, #2021: URL: https://github.com/apache/iceberg-python/pull/2021 Bumps [mypy-boto3-glue](https://github.com/youtype/mypy_boto3_builder) from 1.38.12 to 1.38.18. Release notes Sourced from https://github.com/youtype/mypy_boto3_builder/release

[PR] Build: Bump huggingface-hub from 0.31.2 to 0.31.4 [iceberg-python]

dependabot[bot] opened a new pull request, #2020: URL: https://github.com/apache/iceberg-python/pull/2020 Bumps [huggingface-hub](https://github.com/huggingface/huggingface_hub) from 0.31.2 to 0.31.4. Release notes Sourced from https://github.com/huggingface/huggingface_hub/release

[I] Missing `lower_bound`/`upper_bound` incorrectly treated as though all values for column in partition are NULL [iceberg-rust]

Nathan-Fenner opened a new issue, #1354: URL: https://github.com/apache/iceberg-rust/issues/1354 ### Apache Iceberg Rust version None ### Describe the bug The current implementation for partition filtering treats a missing `lower_bound`/`upper_bound` value as though all

[PR] feat(transaction): Support snapshot validation [iceberg-rust]

CTTY opened a new pull request, #1353: URL: https://github.com/apache/iceberg-rust/pull/1353 ## Which issue does this PR close? - This solves a part of issue: https://github.com/apache/iceberg-rust/issues/1344 ## What changes are included in this PR? - Added

Re: [PR] core: Send single snapshot to remove rather than in bulk [iceberg]

aihuaxu commented on PR #13100: URL: https://github.com/apache/iceberg/pull/13100#issuecomment-2892398076 @amogh-jahagirdar and @ricardopereira33 Can you please take a look? Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Gi

[I] Unable to create/register an Iceberg table in HMS [iceberg-python]

vaultah opened a new issue, #2019: URL: https://github.com/apache/iceberg-python/issues/2019 ### Apache Iceberg version None ### Please describe the bug 🐞 I'm not able to create a table in Hive Metastore using the example code from https://py.iceberg.apache.org/api/#crea

Re: [I] [EPIC] feat: support metadata tables [iceberg-rust]

jonathanc-n commented on issue #823: URL: https://github.com/apache/iceberg-rust/issues/823#issuecomment-2892279866 @geruh https://github.com/apache/iceberg-rust/pull/863 there is quite a big blocker here, if you would like to review, that would be great! -- This is an automated message f

Re: [PR] add_docs_and_backport_max_files_rewrite_option [iceberg]

coderfender commented on PR #13082: URL: https://github.com/apache/iceberg/pull/13082#issuecomment-2892253191 @RussellSpitzer , @pvary the endline fix is unfortunately not recognized by Git (but it would let me commit after removing the whitespace though) . Ref : https://github.com/apac

Re: [I] [EPIC] feat: support metadata tables [iceberg-rust]

geruh commented on issue #823: URL: https://github.com/apache/iceberg-rust/issues/823#issuecomment-2892252832 I can take up the Refs table if nobody is working on it! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] add_docs_and_backport_max_files_rewrite_option [iceberg]

coderfender commented on code in PR #13082: URL: https://github.com/apache/iceberg/pull/13082#discussion_r2096481304 ## docs/docs/spark-procedures.md: ## @@ -1055,4 +1056,4 @@ metadata files and data files to the target location. Lastly, the [register_table](#register_table) pr

Re: [PR] add_docs_and_backport_max_files_rewrite_option [iceberg]

RussellSpitzer commented on PR #13082: URL: https://github.com/apache/iceberg/pull/13082#issuecomment-2892238638 I'll let @pvary comment if he has any final comments, otherwise I'll merge tomorrow -- This is an automated message from the Apache Git Service. To respond to the message, plea

Re: [PR] add_docs_and_backport_max_files_rewrite_option [iceberg]

RussellSpitzer commented on code in PR #13082: URL: https://github.com/apache/iceberg/pull/13082#discussion_r2096468977 ## docs/docs/spark-procedures.md: ## @@ -1055,4 +1056,4 @@ metadata files and data files to the target location. Lastly, the [register_table](#register_table)

Re: [PR] add_docs_and_backport_max_files_rewrite_option [iceberg]

RussellSpitzer commented on code in PR #13082: URL: https://github.com/apache/iceberg/pull/13082#discussion_r2096467291 ## docs/docs/spark-procedures.md: ## @@ -1055,4 +1056,4 @@ metadata files and data files to the target location. Lastly, the [register_table](#register_table)

Re: [PR] add_docs_and_backport_max_files_rewrite_option [iceberg]

RussellSpitzer commented on code in PR #13082: URL: https://github.com/apache/iceberg/pull/13082#discussion_r2096467291 ## docs/docs/spark-procedures.md: ## @@ -1055,4 +1056,4 @@ metadata files and data files to the target location. Lastly, the [register_table](#register_table)

Re: [PR] add_docs_and_backport_max_files_rewrite_option [iceberg]

RussellSpitzer commented on code in PR #13082: URL: https://github.com/apache/iceberg/pull/13082#discussion_r2096465692 ## docs/docs/spark-procedures.md: ## @@ -533,6 +533,7 @@ Dangling deletes are always filtered out during rewriting. | `min-input-files` | 5 | Any file group e

Re: [PR] add_docs_and_backport_max_files_rewrite_option [iceberg]

coderfender commented on code in PR #13082: URL: https://github.com/apache/iceberg/pull/13082#discussion_r2096457057 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/maintenance/api/RewriteDataFiles.java: ## @@ -170,6 +170,12 @@ public Builder maxFileGroupSizeBytes(lo

Re: [PR] add_docs_and_backport_max_files_rewrite_option [iceberg]

coderfender commented on code in PR #13082: URL: https://github.com/apache/iceberg/pull/13082#discussion_r2096455975 ## docs/docs/spark-procedures.md: ## @@ -1055,4 +1056,4 @@ metadata files and data files to the target location. Lastly, the [register_table](#register_table) pr

Re: [PR] add_docs_and_backport_max_files_rewrite_option [iceberg]

coderfender commented on code in PR #13082: URL: https://github.com/apache/iceberg/pull/13082#discussion_r2096455311 ## docs/docs/spark-procedures.md: ## @@ -1055,4 +1056,4 @@ metadata files and data files to the target location. Lastly, the [register_table](#register_table) pr

Re: [PR] add_docs_and_backport_max_files_rewrite_option [iceberg]

coderfender commented on code in PR #13082: URL: https://github.com/apache/iceberg/pull/13082#discussion_r2096432993 ## docs/docs/spark-procedures.md: ## @@ -1055,4 +1056,4 @@ metadata files and data files to the target location. Lastly, the [register_table](#register_table) pr

Re: [PR] add_docs_and_backport_max_files_rewrite_option [iceberg]

pvary commented on code in PR #13082: URL: https://github.com/apache/iceberg/pull/13082#discussion_r2096431739 ## docs/docs/spark-procedures.md: ## @@ -1055,4 +1056,4 @@ metadata files and data files to the target location. Lastly, the [register_table](#register_table) procedur

Re: [PR] add_docs_and_backport_max_files_rewrite_option [iceberg]

coderfender commented on code in PR #13082: URL: https://github.com/apache/iceberg/pull/13082#discussion_r2096423860 ## docs/docs/spark-procedures.md: ## @@ -389,24 +389,25 @@ Iceberg can compact data files in parallel using Spark with the `rewriteDataFile Options

Re: [PR] Add ADLFS property adls.account-host [iceberg-python]

vsdsantos commented on PR #2016: URL: https://github.com/apache/iceberg-python/pull/2016#issuecomment-2892166460 > Thanks for adding this @vsdsantos 🙌 No problem 😄 , thank you all for all the work on this tool! -- This is an automated message from the Apache Git Service. To respond

Re: [PR] Add ADLFS property adls.account-host [iceberg-python]

Fokko commented on PR #2016: URL: https://github.com/apache/iceberg-python/pull/2016#issuecomment-2892134181 Thanks for adding this @vsdsantos 🙌 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

Re: [PR] Add ADLFS property adls.account-host [iceberg-python]

Fokko merged PR #2016: URL: https://github.com/apache/iceberg-python/pull/2016 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [I] Add ADLS account_host Property [iceberg-python]

Fokko closed issue #2015: Add ADLS account_host Property URL: https://github.com/apache/iceberg-python/issues/2015 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscrib

Re: [PR] fix: add metadata_properties to _construct_parameters when update hive table [iceberg-python]

Fokko commented on code in PR #2013: URL: https://github.com/apache/iceberg-python/pull/2013#discussion_r2096362214 ## pyiceberg/catalog/hive.py: ## @@ -541,6 +548,7 @@ def commit_table( hive_table.parameters = _construct_parameters(

Re: [PR] feat: add zlib dependency [iceberg-cpp]

Fokko merged PR #101: URL: https://github.com/apache/iceberg-cpp/pull/101 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] feat(catalog/sql): Add views related ops [iceberg-go]

zeroshade commented on code in PR #414: URL: https://github.com/apache/iceberg-go/pull/414#discussion_r2096347983 ## catalog/sql/sql.go: ## @@ -133,8 +136,12 @@ type sqlIcebergTable struct { CatalogName string `bun:",pk"` TableNamespace str

Re: [PR] Add ADLFS property adls.account-host [iceberg-python]

Fokko commented on PR #2016: URL: https://github.com/apache/iceberg-python/pull/2016#issuecomment-2892031481 @vsdsantos can you run `make lint`? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

Re: [PR] Add ADLFS property adls.account-host [iceberg-python]

Fokko commented on code in PR #2016: URL: https://github.com/apache/iceberg-python/pull/2016#discussion_r2096338022 ## pyiceberg/io/fsspec.py: ## @@ -207,7 +208,8 @@ def _adls(properties: Properties) -> AbstractFileSystem: sas_token=properties.get(ADLS_SAS_TOKEN),

[PR] Spark 3.4: Structured Streaming read limit support follow-up [iceberg]

wypoon opened a new pull request, #13099: URL: https://github.com/apache/iceberg/pull/13099 Use the `ReadLimit` passed in to `SparkMicroBatchStream::latestOffset(Offset, ReadLimit)`. In testing this, a bug was found in `SparkMicroBatchStream::getDefaultReadLimit()` and fixed. This is

Re: [PR] Core, Data: File Format API interfaces [iceberg]

stevenzwu commented on code in PR #12774: URL: https://github.com/apache/iceberg/pull/12774#discussion_r2096253249 ## core/src/main/java/org/apache/iceberg/io/ObjectModel.java: ## @@ -0,0 +1,106 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more co

[PR] Revert "Core: Enhance remove snapshots efficiency by executing them in bulk (#12670) [iceberg]

aihuaxu opened a new pull request, #13098: URL: https://github.com/apache/iceberg/pull/13098 This reverts commit 06f667ada5a5b9edeaa20ae9269ff5de1721b91d. Since it introduces the behavior change that new client with 1.9.x cannot work with the REST catalog with existing Iceberg library

Re: [PR] Core, Data: File Format API interfaces [iceberg]

stevenzwu commented on code in PR #12774: URL: https://github.com/apache/iceberg/pull/12774#discussion_r2096242698 ## core/src/main/java/org/apache/iceberg/io/ObjectModel.java: ## @@ -0,0 +1,106 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more co

Re: [PR] Core, Data: File Format API interfaces [iceberg]

stevenzwu commented on code in PR #12774: URL: https://github.com/apache/iceberg/pull/12774#discussion_r2096253249 ## core/src/main/java/org/apache/iceberg/io/ObjectModel.java: ## @@ -0,0 +1,106 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more co

Re: [PR] Core, Data: File Format API interfaces [iceberg]

stevenzwu commented on code in PR #12774: URL: https://github.com/apache/iceberg/pull/12774#discussion_r2096242698 ## core/src/main/java/org/apache/iceberg/io/ObjectModel.java: ## @@ -0,0 +1,106 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more co

Re: [PR] Core, Data: File Format API interfaces [iceberg]

stevenzwu commented on code in PR #12774: URL: https://github.com/apache/iceberg/pull/12774#discussion_r2096231250 ## core/src/main/java/org/apache/iceberg/io/ObjectModel.java: ## @@ -0,0 +1,106 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more co

Re: [PR] Core, Data: File Format API interfaces [iceberg]

stevenzwu commented on code in PR #12774: URL: https://github.com/apache/iceberg/pull/12774#discussion_r2096231250 ## core/src/main/java/org/apache/iceberg/io/ObjectModel.java: ## @@ -0,0 +1,106 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more co

Re: [PR] Spark-3.5: Add spark action to compute partition stats [iceberg]

karuppayya commented on code in PR #12450: URL: https://github.com/apache/iceberg/pull/12450#discussion_r2093404118 ## api/src/main/java/org/apache/iceberg/actions/ComputePartitionStats.java: ## @@ -0,0 +1,43 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

Re: [I] Flink and iceberg table created using AWS Athena [iceberg]

tomtongue commented on issue #12633: URL: https://github.com/apache/iceberg/issues/12633#issuecomment-2891753218 Amazon Athena uses merge-on-read for the writes. When you update/delete/merge the data on your Iceberg table with Athena, it creates delete files in your S3 bucket. If you want t

[I] Modify REST Tests to Bind Loopback instead of Localhost [iceberg]

RussellSpitzer opened a new issue, #13097: URL: https://github.com/apache/iceberg/issues/13097 ### Feature Request / Improvement Currently a lot of our tests use ```java this.httpServer = new Server(0); // Bind all available interfaces at a random port ``` Which

Re: [I] BaseDeleteLoader may ignore delete records for binary columns [iceberg]

RussellSpitzer closed issue #11239: BaseDeleteLoader may ignore delete records for binary columns URL: https://github.com/apache/iceberg/issues/11239 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Core: Deep copy Record values for equality deletes [iceberg]

RussellSpitzer merged PR #12855: URL: https://github.com/apache/iceberg/pull/12855 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

[I] Custom Credential Refresh Client (fsspec, s3fs) [iceberg-python]

snowman2 opened a new issue, #2018: URL: https://github.com/apache/iceberg-python/issues/2018 ### Feature Request / Improvement Related #2008 I have a custom credential refreshing process using `RefreshableCredentials` that is helpful for long running processes. I would like t

Re: [PR] Core, Data: File Format API interfaces [iceberg]

stevenzwu commented on code in PR #12774: URL: https://github.com/apache/iceberg/pull/12774#discussion_r2096136844 ## core/src/main/java/org/apache/iceberg/io/AppenderBuilder.java: ## @@ -0,0 +1,109 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or mor

Re: [PR] Flink: Dynamic Iceberg Sink: Add table update code for schema comparison and evolution [iceberg]

mxm commented on PR #13032: URL: https://github.com/apache/iceberg/pull/13032#issuecomment-2891659295 Here is the link to the discussion on the mailing list for the core changes: https://lists.apache.org/thread/rmqjbwd8wcqpmkow51m843pwh8fbq4bt -- This is an automated message from the Apac

Re: [PR] feat: add manifest related structures [iceberg-cpp]

Fokko commented on code in PR #91: URL: https://github.com/apache/iceberg-cpp/pull/91#discussion_r2095503674 ## src/iceberg/manifest_list.h: ## @@ -0,0 +1,235 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See t

Re: [PR] Spark 4.0: Structured Streaming read limit support follow-up [iceberg]

wypoon commented on PR #13095: URL: https://github.com/apache/iceberg/pull/13095#issuecomment-2891589782 @huaxingao this is a clean forward port. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

Re: [PR] Spark: Structured Streaming read limit support follow-up [iceberg]

wypoon commented on PR #12260: URL: https://github.com/apache/iceberg/pull/12260#issuecomment-2891588422 Thanks @huaxingao! This PR predates the Spark 4.0 support and I opened https://github.com/apache/iceberg/pull/13095 to port it to Spark 4.0, rather than update this PR. -- This is

[PR] Spark 4.0: Structured Streaming read limit support follow-up [iceberg]

wypoon opened a new pull request, #13095: URL: https://github.com/apache/iceberg/pull/13095 Use the `ReadLimit` passed in to `SparkMicroBatchStream::latestOffset(Offset, ReadLimit)`. In testing this, a bug was found in `SparkMicroBatchStream::getDefaultReadLimit()` and fixed. This is

Re: [PR] Implement snapshot expiration [iceberg-go]

zeroshade commented on code in PR #401: URL: https://github.com/apache/iceberg-go/pull/401#discussion_r2096071405 ## table/updates.go: ## @@ -382,7 +389,74 @@ func NewRemoveSnapshotsUpdate(ids []int64) Update { } func (u *removeSnapshotsUpdate) Apply(builder *MetadataBuilder

Re: [PR] Implement snapshot expiration [iceberg-go]

zeroshade commented on code in PR #401: URL: https://github.com/apache/iceberg-go/pull/401#discussion_r2096050696 ## table/updates.go: ## @@ -382,7 +389,74 @@ func NewRemoveSnapshotsUpdate(ids []int64) Update { } func (u *removeSnapshotsUpdate) Apply(builder *MetadataBuilder

Re: [PR] Implement snapshot expiration [iceberg-go]

zeroshade commented on code in PR #401: URL: https://github.com/apache/iceberg-go/pull/401#discussion_r2096041216 ## table/transaction.go: ## @@ -142,6 +143,120 @@ func (t *Transaction) SetProperties(props iceberg.Properties) error { return nil } +type expireSnapshot

Re: [PR] Implement snapshot expiration [iceberg-go]

zeroshade commented on code in PR #401: URL: https://github.com/apache/iceberg-go/pull/401#discussion_r2096041216 ## table/transaction.go: ## @@ -142,6 +143,120 @@ func (t *Transaction) SetProperties(props iceberg.Properties) error { return nil } +type expireSnapshot

Re: [PR] Implement snapshot expiration [iceberg-go]

zeroshade commented on code in PR #401: URL: https://github.com/apache/iceberg-go/pull/401#discussion_r2096028620 ## table/metadata.go: ## @@ -187,14 +187,14 @@ func MetadataBuilderFromBase(metadata Metadata) (*MetadataBuilder, error) { b.loc = metadata.Location()

Re: [PR] Implement snapshot expiration [iceberg-go]

zeroshade commented on code in PR #401: URL: https://github.com/apache/iceberg-go/pull/401#discussion_r2096038254 ## table/transaction.go: ## @@ -142,6 +143,120 @@ func (t *Transaction) SetProperties(props iceberg.Properties) error { return nil } +type expireSnapshot

Re: [PR] Core: Deep copy Record values for equality deletes [iceberg]

RussellSpitzer commented on PR #12855: URL: https://github.com/apache/iceberg/pull/12855#issuecomment-2891487195 Looking good! I'm waiting on the tests to pass and then I'll merge. Thanks so much @hsingh574 for the patch. -- This is an automated message from the Apache Git Service. To res

Re: [PR] Core, Data: File Format API interfaces [iceberg]

pvary commented on code in PR #12774: URL: https://github.com/apache/iceberg/pull/12774#discussion_r2095199785 ## core/src/main/java/org/apache/iceberg/io/ObjectModel.java: ## @@ -0,0 +1,106 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contri

Re: [PR] feat: add zlib dependency [iceberg-cpp]

dongxiao1198 commented on code in PR #101: URL: https://github.com/apache/iceberg-cpp/pull/101#discussion_r2095224359 ## cmake_modules/IcebergThirdpartyToolchain.cmake: ## @@ -86,6 +86,9 @@ function(resolve_arrow_dependency) set(ARROW_DEPENDENCY_SOURCE "BUNDLED"

Re: [PR] Core, Data: File Format API interfaces [iceberg]

pvary commented on code in PR #12774: URL: https://github.com/apache/iceberg/pull/12774#discussion_r2095261258 ## core/src/main/java/org/apache/iceberg/io/ObjectModel.java: ## @@ -0,0 +1,106 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contri

  1   2   3   >