Re: [PR] feat: support `arrow_struct_to_iceberg_struct` [iceberg-rust]

ZENOTME commented on code in PR #731: URL: https://github.com/apache/iceberg-rust/pull/731#discussion_r1899344509 ## crates/iceberg/src/spec/schema.rs: ## @@ -1132,6 +1132,213 @@ impl ReassignFieldIds { } } +/// A post order schema visitor with partner. +/// +/// For ord

Re: [PR] feat: support `arrow_struct_to_iceberg_struct` [iceberg-rust]

ZENOTME commented on code in PR #731: URL: https://github.com/apache/iceberg-rust/pull/731#discussion_r1899344509 ## crates/iceberg/src/spec/schema.rs: ## @@ -1132,6 +1132,213 @@ impl ReassignFieldIds { } } +/// A post order schema visitor with partner. +/// +/// For ord

Re: [PR] feat: support `arrow_struct_to_iceberg_struct` [iceberg-rust]

ZENOTME commented on code in PR #731: URL: https://github.com/apache/iceberg-rust/pull/731#discussion_r1899343049 ## crates/iceberg/src/spec/schema.rs: ## @@ -1132,6 +1132,213 @@ impl ReassignFieldIds { } } +/// A post order schema visitor with partner. +/// +/// For ord

Re: [PR] feat: support `arrow_struct_to_iceberg_struct` [iceberg-rust]

ZENOTME commented on code in PR #731: URL: https://github.com/apache/iceberg-rust/pull/731#discussion_r1899342562 ## crates/iceberg/src/spec/schema.rs: ## @@ -1132,6 +1132,213 @@ impl ReassignFieldIds { } } +/// A post order schema visitor with partner. +/// +/// For ord

Re: [PR] feat: support `arrow_struct_to_iceberg_struct` [iceberg-rust]

ZENOTME commented on code in PR #731: URL: https://github.com/apache/iceberg-rust/pull/731#discussion_r1899341695 ## crates/iceberg/src/spec/schema.rs: ## @@ -1132,6 +1132,213 @@ impl ReassignFieldIds { } } +/// A post order schema visitor with partner. +/// +/// For ord

Re: [PR] feat: support `arrow_struct_to_iceberg_struct` [iceberg-rust]

liurenjie1024 commented on code in PR #731: URL: https://github.com/apache/iceberg-rust/pull/731#discussion_r1899291231 ## crates/iceberg/src/spec/schema.rs: ## @@ -1132,6 +1132,213 @@ impl ReassignFieldIds { } } +/// A post order schema visitor with partner. +/// +/// F

Re: [I] i meet a question，time travel has some error to the query of timestamp [iceberg]

hashmapybx commented on issue #11889: URL: https://github.com/apache/iceberg/issues/11889#issuecomment-2565089773 > Thanks @hashmapybx for raising this question, unfortunately, the information that you provide is very little. At first glance, the SQL seems to be correct. What kind of versio

Re: [I] i meet a question，time travel has some error to the query of timestamp [iceberg]

hashmapybx commented on issue #11889: URL: https://github.com/apache/iceberg/issues/11889#issuecomment-2565084250 Additional Notes： https://github.com/user-attachments/assets/efbf317f-db87-4c65-84eb-2fe4d2da955b"; /> https://github.com/user-attachments/assets/96cabcc5-7f5e-412f-955b-d3

Re: [I] i meet a question，time travel has some error to the query of timestamp [iceberg]

Fokko commented on issue #11889: URL: https://github.com/apache/iceberg/issues/11889#issuecomment-2565070938 Thanks @hashmapybx for raising this question, unfortunately, the information that you provide is very little. At first glance, the SQL seems to be correct. What kind of version of Sp

Re: [PR] Add orbstack guide [iceberg-rust]

lewiszlw commented on code in PR #856: URL: https://github.com/apache/iceberg-rust/pull/856#discussion_r1899296910 ## docs/contributing/orbstack.md: ## @@ -0,0 +1,39 @@ + + +# OrbStack as a docker alternative on macOS +1. Install OrbStack by downloading [installer](https://orbst

[I] i meet a question. [iceberg]

hashmapybx opened a new issue, #11889: URL: https://github.com/apache/iceberg/issues/11889 ### Query engine spark-sql ### Question 命令行根据时间戳做时间旅行查询报错 spark-sql (cjh_test)> select * from cjh_test.datalake_iceberg_table timestamp as of '2024-12-30 10:28:57.321'; Inval

[I] Iceberg has data consistency issues when processing CDC deleted records [iceberg]

beyond-up opened a new issue, #11888: URL: https://github.com/apache/iceberg/issues/11888 ### Apache Iceberg version 1.5.2 ### Query engine Flink ### Please describe the bug 🐞 ![image](https://github.com/user-attachments/assets/836cca53-810c-47f1-b477-27839

Re: [PR] docs: various spelling fixes [iceberg-python]

kevinjqliu commented on PR #1471: URL: https://github.com/apache/iceberg-python/pull/1471#issuecomment-2564996100 thank you! this tool is very helpful. Do you think its useful to include it as part of pre-commit hook? https://github.com/codespell-project/codespell?tab=readme-ov-file#pre

Re: [PR] Add orbstack guide [iceberg-rust]

kevinjqliu commented on code in PR #856: URL: https://github.com/apache/iceberg-rust/pull/856#discussion_r1899255390 ## docs/contributing/orbstack.md: ## @@ -0,0 +1,39 @@ + + +# OrbStack as a docker alternative on macOS +1. Install OrbStack by downloading [installer](https://orb

Re: [PR] Add orbstack guide [iceberg-rust]

liurenjie1024 merged PR #856: URL: https://github.com/apache/iceberg-rust/pull/856 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [PR] fix: parse var len of decimal for parquet statistic [iceberg-rust]

ZENOTME commented on PR #837: URL: https://github.com/apache/iceberg-rust/pull/837#issuecomment-2564967032 Hi @liurenjie1024 @Xuanwo , is there other question about this PR? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] Add orbstack guide [iceberg-rust]

lewiszlw commented on code in PR #856: URL: https://github.com/apache/iceberg-rust/pull/856#discussion_r1899234586 ## CONTRIBUTING.md: ## @@ -116,7 +116,7 @@ cargo 1.69.0 (6e9a83356 2023-04-12) Currently, iceberg-rust uses Docker to set up environment for integration tests.

Re: [I] Overwrite with condition from spark failed [iceberg]

github-actions[bot] commented on issue #10619: URL: https://github.com/apache/iceberg/issues/10619#issuecomment-2564877839 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [PR] Handling NO Coordinator Scenario and Data Loss in the current Design [iceberg]

github-actions[bot] commented on PR #11298: URL: https://github.com/apache/iceberg/pull/11298#issuecomment-2564878061 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

Re: [PR] Core: Set missing table-default property in RESTSessionCatalog [iceberg]

github-actions[bot] commented on PR #11646: URL: https://github.com/apache/iceberg/pull/11646#issuecomment-2564878961 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] Core,Format: Deprecate embedded manifests [iceberg]

github-actions[bot] commented on PR #11586: URL: https://github.com/apache/iceberg/pull/11586#issuecomment-2564878805 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

Re: [PR] Core,Format: Deprecate embedded manifests [iceberg]

github-actions[bot] closed pull request #11586: Core,Format: Deprecate embedded manifests URL: https://github.com/apache/iceberg/pull/11586 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

Re: [PR] Spark: 4.0 snapshot support [iceberg]

github-actions[bot] commented on PR #11583: URL: https://github.com/apache/iceberg/pull/11583#issuecomment-2564878514 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [I] Concurrency issues in `o.a.i.hive.MetastoreLock` [iceberg]

github-actions[bot] commented on issue #10595: URL: https://github.com/apache/iceberg/issues/10595#issuecomment-2564877814 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [PR] Handling NO Coordinator Scenario and Data Loss in the current Design [iceberg]

github-actions[bot] closed pull request #11298: Handling NO Coordinator Scenario and Data Loss in the current Design URL: https://github.com/apache/iceberg/pull/11298 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

Re: [I] Describes how to set up a catalog that starts with the Rest Catalog(like Unity Catalog). [iceberg]

github-actions[bot] commented on issue #10519: URL: https://github.com/apache/iceberg/issues/10519#issuecomment-256488 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache

Re: [I] throws "Caused by: org.apache.iceberg.exceptions.NoSuchTableException: Table does not exist at location:" [iceberg]

github-actions[bot] closed issue #10509: throws "Caused by: org.apache.iceberg.exceptions.NoSuchTableException: Table does not exist at location:" URL: https://github.com/apache/iceberg/issues/10509 -- This is an automated message from the Apache Git Service. To respond to the message, pleas

Re: [I] Describes how to set up a catalog that starts with the Rest Catalog(like Unity Catalog). [iceberg]

github-actions[bot] closed issue #10519: Describes how to set up a catalog that starts with the Rest Catalog(like Unity Catalog). URL: https://github.com/apache/iceberg/issues/10519 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [I] throws "Caused by: org.apache.iceberg.exceptions.NoSuchTableException: Table does not exist at location:" [iceberg]

github-actions[bot] commented on issue #10509: URL: https://github.com/apache/iceberg/issues/10509#issuecomment-2564877764 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache

Re: [PR] Core: Fix typo in HadoopTableOperations [iceberg]

okumin commented on PR #11880: URL: https://github.com/apache/iceberg/pull/11880#issuecomment-2564867843 Thank you! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsub

Re: [PR] Kafka Connect: Add config to route to tables using topic name [iceberg]

xiasongh commented on PR #11313: URL: https://github.com/apache/iceberg/pull/11313#issuecomment-2564847217 Hey @bryanck @mun1r0b0t! Sorry! I've been a bit busy and this PR got lost. I can finish up this PR, but I'd like to get some thoughts on my previous comment so I can make

Re: [PR] chore(deps): Bump aws-sdk-s3tables from 1.1.0 to 1.2.0 [iceberg-rust]

Fokko merged PR #858: URL: https://github.com/apache/iceberg-rust/pull/858 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] chore(deps): Bump reqwest from 0.12.10 to 0.12.11 [iceberg-rust]

Fokko merged PR #859: URL: https://github.com/apache/iceberg-rust/pull/859 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] chore(deps): Bump serde from 1.0.216 to 1.0.217 [iceberg-rust]

Fokko merged PR #860: URL: https://github.com/apache/iceberg-rust/pull/860 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Core: Fix typo in HadoopTableOperations [iceberg]

Fokko commented on PR #11880: URL: https://github.com/apache/iceberg/pull/11880#issuecomment-2564835789 Thanks for fixing this @okumin and thanks for the review @ebyhr 🙌 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] Core: Fix typo in HadoopTableOperations [iceberg]

Fokko merged PR #11880: URL: https://github.com/apache/iceberg/pull/11880 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] Feat/update sort order [iceberg-python]

JasperHG90 commented on PR #1465: URL: https://github.com/apache/iceberg-python/pull/1465#issuecomment-2564834930 Thanks @Fokko ! Hadn't realized that I opened the PR against this repo already. This is just scaffolding code. I will update it the coming week with the actual logic. -- This

Re: [PR] Build: Bump mkdocs-awesome-pages-plugin from 2.10.0 to 2.10.1 [iceberg]

Fokko merged PR #11885: URL: https://github.com/apache/iceberg/pull/11885 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] Build: Bump software.amazon.awssdk:bom from 2.29.39 to 2.29.43 [iceberg]

Fokko merged PR #11886: URL: https://github.com/apache/iceberg/pull/11886 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] Fix read from multiple s3 regions [iceberg-python]

jiakai-li commented on code in PR #1453: URL: https://github.com/apache/iceberg-python/pull/1453#discussion_r1899180251 ## pyiceberg/io/pyarrow.py: ## @@ -362,6 +362,12 @@ def _initialize_fs(self, scheme: str, netloc: Optional[str] = None) -> FileSyste "region"

Re: [PR] Rest catalog integration testing [iceberg-python]

AhmedNader42 commented on PR #1469: URL: https://github.com/apache/iceberg-python/pull/1469#issuecomment-2564818223 Hello @kevinjqliu, Please can we have a review for this PR -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

Re: [PR] Fix read from multiple s3 regions [iceberg-python]

jiakai-li commented on code in PR #1453: URL: https://github.com/apache/iceberg-python/pull/1453#discussion_r1899180251 ## pyiceberg/io/pyarrow.py: ## @@ -362,6 +362,12 @@ def _initialize_fs(self, scheme: str, netloc: Optional[str] = None) -> FileSyste "region"

Re: [PR] Fix read from multiple s3 regions [iceberg-python]

jiakai-li commented on code in PR #1453: URL: https://github.com/apache/iceberg-python/pull/1453#discussion_r1899180251 ## pyiceberg/io/pyarrow.py: ## @@ -362,6 +362,12 @@ def _initialize_fs(self, scheme: str, netloc: Optional[str] = None) -> FileSyste "region"

Re: [PR] Iceberg/Comet integration POC [iceberg]

huaxingao commented on code in PR #9841: URL: https://github.com/apache/iceberg/pull/9841#discussion_r1899171737 ## spark/v3.4/spark/src/main/java/org/apache/iceberg/spark/data/vectorized/ColumnVectorBuilder.java: ## @@ -46,8 +46,8 @@ public ColumnVector build(VectorHolder holde

Re: [PR] feat: search current working directory for config file [iceberg-python]

IndexSeek commented on PR #1464: URL: https://github.com/apache/iceberg-python/pull/1464#issuecomment-2564799798 > I've left one comment, but apart from that, it looks good to me 👍 Thank you! I applied that change! ✅ -- This is an automated message from the Apache Git Service. To r

Re: [I] Spark can not delete table metadata and data when drop table [iceberg]

zzeekk commented on issue #9990: URL: https://github.com/apache/iceberg/issues/9990#issuecomment-2564779267 how can i reference parameters in an ArgoCD template? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] WIP: feat: support S3 Table Buckets with S3TablesCatalog [iceberg-python]

felixscherz commented on PR #1429: URL: https://github.com/apache/iceberg-python/pull/1429#issuecomment-2564764850 I think you can now review this PR if you have time @kevinjqliu :) The biggest issue for now will be that testing is only possible against AWS itself since moto does not supp

Re: [PR] feat: support metadata table "snapshots" [iceberg-rust]

rshkv commented on PR #822: URL: https://github.com/apache/iceberg-rust/pull/822#issuecomment-2564747091 Yeah, totally! > BTW I think we can also change the details like signatures and fields on demand later. Yeah that's what I was worried about. Sg. 👍🏻 -- This is an autom

[PR] feat: Support metadata table "Manifests" [iceberg-rust]

flaneur2020 opened a new pull request, #861: URL: https://github.com/apache/iceberg-rust/pull/861 Re https://github.com/apache/iceberg-rust/issues/823. Extends @xxchan's https://github.com/apache/iceberg-rust/pull/822 to add support for the [Manifests](https://github.com/apache/iceberg/blob

[PR] chore(deps): Bump serde from 1.0.216 to 1.0.217 [iceberg-rust]

dependabot[bot] opened a new pull request, #860: URL: https://github.com/apache/iceberg-rust/pull/860 Bumps [serde](https://github.com/serde-rs/serde) from 1.0.216 to 1.0.217. Release notes Sourced from https://github.com/serde-rs/serde/releases";>serde's releases. v1.0.217

[PR] chore(deps): Bump reqwest from 0.12.10 to 0.12.11 [iceberg-rust]

dependabot[bot] opened a new pull request, #859: URL: https://github.com/apache/iceberg-rust/pull/859 Bumps [reqwest](https://github.com/seanmonstar/reqwest) from 0.12.10 to 0.12.11. Release notes Sourced from https://github.com/seanmonstar/reqwest/releases";>reqwest's releases.

[PR] chore(deps): Bump aws-sdk-s3tables from 1.1.0 to 1.2.0 [iceberg-rust]

dependabot[bot] opened a new pull request, #858: URL: https://github.com/apache/iceberg-rust/pull/858 Bumps [aws-sdk-s3tables](https://github.com/awslabs/aws-sdk-rust) from 1.1.0 to 1.2.0. Commits See full diff in https://github.com/awslabs/aws-sdk-rust/commits";>compare view

Re: [PR] feat: support metadata table "snapshots" [iceberg-rust]

flaneur2020 commented on code in PR #822: URL: https://github.com/apache/iceberg-rust/pull/822#discussion_r1899100103 ## crates/iceberg/src/metadata_scan.rs: ## @@ -0,0 +1,265 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agree

[PR] build(deps): bump github.com/aws/aws-sdk-go-v2/service/s3 from 1.71.0 to 1.71.1 [iceberg-go]

dependabot[bot] opened a new pull request, #239: URL: https://github.com/apache/iceberg-go/pull/239 Bumps [github.com/aws/aws-sdk-go-v2/service/s3](https://github.com/aws/aws-sdk-go-v2) from 1.71.0 to 1.71.1. Commits https://github.com/aws/aws-sdk-go-v2/commit/5a964704cb2640ed