Re: [PR] Add iceberg_arrow library [iceberg-cpp]

2024-12-30 Thread via GitHub
wgtmac commented on code in PR #6: URL: https://github.com/apache/iceberg-cpp/pull/6#discussion_r1899970387 ## src/CMakeLists.txt: ## @@ -15,5 +15,20 @@ # specific language governing permissions and limitations # under the License. +add_subdirectory(arrow) add_subdirectory(

Re: [PR] feat: support serialize/deserialize DataFile into avro bytes [iceberg-rust]

2024-12-30 Thread via GitHub
Xuanwo commented on code in PR #797: URL: https://github.com/apache/iceberg-rust/pull/797#discussion_r1899967184 ## crates/iceberg/src/spec/manifest.rs: ## @@ -1189,6 +1211,47 @@ impl DataFile { self.sort_order_id } } + +/// Convert data files to avro bytes. +pub

Re: [PR] Add iceberg_arrow library [iceberg-cpp]

2024-12-30 Thread via GitHub
wgtmac commented on code in PR #6: URL: https://github.com/apache/iceberg-cpp/pull/6#discussion_r1899965461 ## example/CMakeLists.txt: ## @@ -22,10 +22,10 @@ project(example) set(CMAKE_CXX_STANDARD 20) -find_package(iceberg CONFIG REQUIRED) -find_package(puffin CONFIG REQUI

Re: [PR] feat: sql catalog support update table [iceberg-rust]

2024-12-30 Thread via GitHub
Li0k commented on PR #862: URL: https://github.com/apache/iceberg-rust/pull/862#issuecomment-2566204936 cc @Xuanwo @liurenjie1024 @ZENOTME -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

Re: [PR] feat: support serialize/deserialize DataFile into avro bytes [iceberg-rust]

2024-12-30 Thread via GitHub
liurenjie1024 commented on code in PR #797: URL: https://github.com/apache/iceberg-rust/pull/797#discussion_r1899955848 ## crates/iceberg/src/spec/manifest.rs: ## @@ -1189,6 +1211,47 @@ impl DataFile { self.sort_order_id } } + +/// Convert data files to avro bytes

Re: [PR] feat: support serialize/deserialize DataFile into avro bytes [iceberg-rust]

2024-12-30 Thread via GitHub
liurenjie1024 commented on code in PR #797: URL: https://github.com/apache/iceberg-rust/pull/797#discussion_r1899954979 ## crates/iceberg/src/spec/manifest.rs: ## @@ -1189,6 +1211,47 @@ impl DataFile { self.sort_order_id } } + +/// Convert data files to avro bytes

[I] Flink Iceberg Writer : To be able to use copy-on-write mode to write the iceberg tables for batch jobs [iceberg]

2024-12-30 Thread via GitHub
infoankitp opened a new issue, #11893: URL: https://github.com/apache/iceberg/issues/11893 ### Feature Request / Improvement Below is a PR which was not merged and closed for the exact same feature. Any reason behind that ? I am actually looking up for the feature. https://git

[PR] Doc: Fix format of Hive [iceberg]

2024-12-30 Thread via GitHub
ebyhr opened a new pull request, #11892: URL: https://github.com/apache/iceberg/pull/11892 The original format is broken: ![Screenshot 2024-12-31 at 15 09 16](https://github.com/user-attachments/assets/94dfc5af-52b2-421a-ae28-a63a4e5d9609) -- This is an automated message from t

Re: [PR] Add iceberg_arrow library [iceberg-cpp]

2024-12-30 Thread via GitHub
wgtmac commented on code in PR #6: URL: https://github.com/apache/iceberg-cpp/pull/6#discussion_r1899930371 ## cmake_modules/ThirdpartyToolchain.cmake: ## @@ -0,0 +1,142 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements.

Re: [PR] Add iceberg_arrow library [iceberg-cpp]

2024-12-30 Thread via GitHub
wgtmac commented on code in PR #6: URL: https://github.com/apache/iceberg-cpp/pull/6#discussion_r1899929235 ## src/CMakeLists.txt: ## @@ -15,5 +15,20 @@ # specific language governing permissions and limitations # under the License. +add_subdirectory(arrow) add_subdirectory(

Re: [PR] Add iceberg_arrow library [iceberg-cpp]

2024-12-30 Thread via GitHub
wgtmac commented on code in PR #6: URL: https://github.com/apache/iceberg-cpp/pull/6#discussion_r1899927285 ## cmake_modules/ThirdpartyToolchain.cmake: ## @@ -0,0 +1,139 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements.

Re: [PR] Add GitHub cpp-linter-action [iceberg-cpp]

2024-12-30 Thread via GitHub
lidavidm commented on PR #20: URL: https://github.com/apache/iceberg-cpp/pull/20#issuecomment-2566143404 I think it's a security issue: the `write` permission doesn't discriminate between who opened the PR or where you can write to, so someone could open a malicious PR to write to the repos

Re: [PR] feat: support serialize/deserialize DataFile into avro bytes [iceberg-rust]

2024-12-30 Thread via GitHub
ZENOTME commented on code in PR #797: URL: https://github.com/apache/iceberg-rust/pull/797#discussion_r1899922784 ## crates/iceberg/src/spec/manifest.rs: ## @@ -1189,6 +1211,47 @@ impl DataFile { self.sort_order_id } } + +/// Convert data files to avro bytes. +pub

Re: [PR] Add GitHub cpp-linter-action [iceberg-cpp]

2024-12-30 Thread via GitHub
wgtmac commented on PR #20: URL: https://github.com/apache/iceberg-cpp/pull/20#issuecomment-2566113136 Thanks @lidavidm for providing the detail! After reading relevant documentation, I agree that it is really tricky to workaround the token permission. Without comments on PR, the cpp-linter

Re: [PR] feat: support serialize/deserialize DataFile into avro bytes [iceberg-rust]

2024-12-30 Thread via GitHub
liurenjie1024 commented on code in PR #797: URL: https://github.com/apache/iceberg-rust/pull/797#discussion_r1899889761 ## crates/iceberg/src/spec/manifest.rs: ## @@ -656,6 +656,38 @@ mod _const_schema { }) }; +fn data_file_fields_v2(partition_type: StructTyp

Re: [PR] fix: gurantee the deserialize order of struct is same as the struct type [iceberg-rust]

2024-12-30 Thread via GitHub
ZENOTME commented on code in PR #795: URL: https://github.com/apache/iceberg-rust/pull/795#discussion_r1899896614 ## crates/iceberg/src/spec/values.rs: ## @@ -2728,22 +2730,24 @@ mod _serde { optional: _, }) => match ty {

Re: [PR] Add GitHub cpp-linter-action [iceberg-cpp]

2024-12-30 Thread via GitHub
lidavidm commented on PR #20: URL: https://github.com/apache/iceberg-cpp/pull/20#issuecomment-2566091765 Note that pull_request_target (1) only runs from the base branch, not the PR and (2) doesn't check out the PR HEAD (it gets the base branch) so it's not suitable for this by default. (Yo

[I] Fields are out of order in partitioned-table equality delete files [iceberg]

2024-12-30 Thread via GitHub
beyond-up opened a new issue, #11891: URL: https://github.com/apache/iceberg/issues/11891 ### Apache Iceberg version 1.5.2 ### Query engine Flink ### Please describe the bug 🐞 https://github.com/user-attachments/assets/5324df05-b641-4201-b529-87cd8c9f18c5";

Re: [PR] Add orbstack guide [iceberg-rust]

2024-12-30 Thread via GitHub
kevinjqliu commented on code in PR #856: URL: https://github.com/apache/iceberg-rust/pull/856#discussion_r1899887407 ## docs/contributing/orbstack.md: ## @@ -0,0 +1,39 @@ + + +# OrbStack as a docker alternative on macOS +1. Install OrbStack by downloading [installer](https://orb

Re: [PR] Add orbstack guide [iceberg-rust]

2024-12-30 Thread via GitHub
kevinjqliu commented on code in PR #856: URL: https://github.com/apache/iceberg-rust/pull/856#discussion_r1899887407 ## docs/contributing/orbstack.md: ## @@ -0,0 +1,39 @@ + + +# OrbStack as a docker alternative on macOS +1. Install OrbStack by downloading [installer](https://orb

Re: [PR] Add GitHub cpp-linter-action [iceberg-cpp]

2024-12-30 Thread via GitHub
lidavidm commented on PR #20: URL: https://github.com/apache/iceberg-cpp/pull/20#issuecomment-2566084541 It seems since the PR comes from a fork, Github will never let you get anything but read access. https://docs.github.com/en/actions/security-for-github-actions/security-guides/automatic-

Re: [PR] Add GitHub cpp-linter-action [iceberg-cpp]

2024-12-30 Thread via GitHub
lidavidm commented on PR #20: URL: https://github.com/apache/iceberg-cpp/pull/20#issuecomment-2566084128 The token still says only read permissions https://github.com/apache/iceberg-cpp/actions/runs/12554533849/job/35003344629?pr=20#step:1:17 -- This is an automated message from the Apac

Re: [PR] Add GitHub cpp-linter-action [iceberg-cpp]

2024-12-30 Thread via GitHub
wgtmac commented on PR #20: URL: https://github.com/apache/iceberg-cpp/pull/20#issuecomment-2566081248 > It appears either the token doesn't have comment permissions or the token isn't being passed into the action I have added permission for `pull-requests: write` but still hit the f

Re: [PR] feat: Add RemovePartitionSpecs table update [iceberg-rust]

2024-12-30 Thread via GitHub
liurenjie1024 commented on code in PR #804: URL: https://github.com/apache/iceberg-rust/pull/804#discussion_r1899881544 ## crates/iceberg/src/catalog/mod.rs: ## @@ -446,6 +446,12 @@ pub enum TableUpdate { /// Properties to remove removals: Vec, }, +///

Re: [I] i meet a question,time travel has some error to the query of timestamp [iceberg]

2024-12-30 Thread via GitHub
hashmapybx commented on issue #11889: URL: https://github.com/apache/iceberg/issues/11889#issuecomment-2566073821 change after config. https://github.com/user-attachments/assets/c03e4f40-aa2c-4f16-9a21-7ed8c43afb54"; /> -- This is an automated message from the Apache Git Servic

Re: [I] i meet a question,time travel has some error to the query of timestamp [iceberg]

2024-12-30 Thread via GitHub
hashmapybx closed issue #11889: i meet a question,time travel has some error to the query of timestamp URL: https://github.com/apache/iceberg/issues/11889 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [I] i meet a question,time travel has some error to the query of timestamp [iceberg]

2024-12-30 Thread via GitHub
hashmapybx commented on issue #11889: URL: https://github.com/apache/iceberg/issues/11889#issuecomment-2566072875 > `TimeTravelRelation`, this seems to be hudi's logical plan. could you confirm if hudi has been included? thanks,i find that the content of spark-default.conf file has

Re: [PR] Add GitHub cpp-linter-action [iceberg-cpp]

2024-12-30 Thread via GitHub
lidavidm commented on PR #20: URL: https://github.com/apache/iceberg-cpp/pull/20#issuecomment-2566068656 It appears either the token doesn't have comment permissions or the token isn't being passed into the action -- This is an automated message from the Apache Git Service. To respond to

Re: [PR] fix: gurantee the deserialize order of struct is same as the struct type [iceberg-rust]

2024-12-30 Thread via GitHub
liurenjie1024 commented on code in PR #795: URL: https://github.com/apache/iceberg-rust/pull/795#discussion_r1899878300 ## crates/iceberg/src/spec/values.rs: ## @@ -2728,22 +2730,24 @@ mod _serde { optional: _, }) => match ty {

Re: [PR] Add GitHub cpp-linter-action [iceberg-cpp]

2024-12-30 Thread via GitHub
wgtmac commented on code in PR #20: URL: https://github.com/apache/iceberg-cpp/pull/20#discussion_r1899875630 ## api/iceberg/table.h: ## @@ -20,15 +20,24 @@ #pragma once #include -#include + Review Comment: I have inserted a build step before the cpp-linter to create t

Re: [I] Add Multi-Table Transaction API [iceberg]

2024-12-30 Thread via GitHub
github-actions[bot] commented on issue #10617: URL: https://github.com/apache/iceberg/issues/10617#issuecomment-2566026989 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [I] Support metadata tables for views [iceberg]

2024-12-30 Thread via GitHub
chadlagore commented on issue #9844: URL: https://github.com/apache/iceberg/issues/9844#issuecomment-2566014324 Related, but might need a separate issue - do we intend to support VERSION AS OF for views? Or named versions? Simple use case would be for views built on tags on underlying

Re: [PR] Add iceberg_arrow library [iceberg-cpp]

2024-12-30 Thread via GitHub
kou commented on code in PR #6: URL: https://github.com/apache/iceberg-cpp/pull/6#discussion_r1899827873 ## cmake_modules/ThirdpartyToolchain.cmake: ## @@ -0,0 +1,142 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See

Re: [I] java.lang.IllegalStateException: Connection pool shut down in Spark [iceberg]

2024-12-30 Thread via GitHub
amogh-jahagirdar closed issue #11633: java.lang.IllegalStateException: Connection pool shut down in Spark URL: https://github.com/apache/iceberg/issues/11633 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

Re: [I] HiveCatalog incorrectly uses FileIOTracker [iceberg]

2024-12-30 Thread via GitHub
amogh-jahagirdar closed issue #11783: HiveCatalog incorrectly uses FileIOTracker URL: https://github.com/apache/iceberg/issues/11783 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

Re: [PR] Revert "Hive: close the fileIO client when closing the hive catalog" [iceberg]

2024-12-30 Thread via GitHub
amogh-jahagirdar merged PR #11858: URL: https://github.com/apache/iceberg/pull/11858 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@

Re: [PR] Revert "Hive: close the fileIO client when closing the hive catalog" [iceberg]

2024-12-30 Thread via GitHub
amogh-jahagirdar commented on PR #11858: URL: https://github.com/apache/iceberg/pull/11858#issuecomment-2565910095 I'll go ahead and merge, thanks for reviewing @hussein-awala @bryanck ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [PR] Feat/update sort order [iceberg-python]

2024-12-30 Thread via GitHub
JasperHG90 closed pull request #1465: Feat/update sort order URL: https://github.com/apache/iceberg-python/pull/1465 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscr

Re: [I] Security improvements in the Iceberg REST specification [iceberg]

2024-12-30 Thread via GitHub
dimas-b commented on issue #10537: URL: https://github.com/apache/iceberg/issues/10537#issuecomment-2565802048 I believe this issue is still relevant. More specifically, M3 is addressed by #11844 (and possibly some follow-up PRs). -- This is an automated message from the Apache Git

Re: [PR] feat: Support metadata table "Metadata Log Entries" [iceberg-rust]

2024-12-30 Thread via GitHub
rshkv commented on PR #846: URL: https://github.com/apache/iceberg-rust/pull/846#issuecomment-2565741489 Thank you, @xuanwo. This is rebased and ready for review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] feat: Support metadata table "History" [iceberg-rust]

2024-12-30 Thread via GitHub
rshkv commented on PR #841: URL: https://github.com/apache/iceberg-rust/pull/841#issuecomment-2565687241 Thank you for letting me know. Rebased on `main`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

Re: [PR] Implemented Remaining Catalog operations for REST catalog [iceberg-go]

2024-12-30 Thread via GitHub
piyushsingariya commented on code in PR #240: URL: https://github.com/apache/iceberg-go/pull/240#discussion_r1899655889 ## catalog/rest.go: ## @@ -710,3 +777,54 @@ func (r *RestCatalog) UpdateNamespaceProperties(ctx context.Context, namespace t return doPost[payload, Pr

Re: [PR] Implemented Remaining Catalog operations for REST catalog [iceberg-go]

2024-12-30 Thread via GitHub
zeroshade commented on PR #240: URL: https://github.com/apache/iceberg-go/pull/240#issuecomment-2565678980 @chil-pavn In general this is looking good, I'll give it a more in-depth review in the next couple days. Could you add some unit tests for these as we have for the other functions? (ba

Re: [PR] build(deps): bump github.com/aws/aws-sdk-go-v2/service/s3 from 1.71.0 to 1.71.1 [iceberg-go]

2024-12-30 Thread via GitHub
zeroshade merged PR #239: URL: https://github.com/apache/iceberg-go/pull/239 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] release_rc.sh upload artifacts to apache dist [iceberg-go]

2024-12-30 Thread via GitHub
zeroshade merged PR #237: URL: https://github.com/apache/iceberg-go/pull/237 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] feat: Support metadata table "Metadata Log Entries" [iceberg-rust]

2024-12-30 Thread via GitHub
Xuanwo commented on PR #846: URL: https://github.com/apache/iceberg-rust/pull/846#issuecomment-2565637928 https://github.com/apache/iceberg-rust/pull/822 has been merged, let's move on! -- This is an automated message from the Apache Git Service. To respond to the message, please log on t

Re: [PR] feat: Support metadata table "History" [iceberg-rust]

2024-12-30 Thread via GitHub
Xuanwo commented on PR #841: URL: https://github.com/apache/iceberg-rust/pull/841#issuecomment-2565637704 https://github.com/apache/iceberg-rust/pull/822 has been merged, let's move on! -- This is an automated message from the Apache Git Service. To respond to the message, please log on t

Re: [PR] feat: Support metadata table "Manifests" [iceberg-rust]

2024-12-30 Thread via GitHub
Xuanwo commented on PR #861: URL: https://github.com/apache/iceberg-rust/pull/861#issuecomment-2565637534 https://github.com/apache/iceberg-rust/pull/822 has been merged, let's move on! -- This is an automated message from the Apache Git Service. To respond to the message, please log on t

Re: [PR] backport #11301(rowconverter) to Flink 1.19 and 1.18 [iceberg]

2024-12-30 Thread via GitHub
abharath9 commented on PR #11826: URL: https://github.com/apache/iceberg/pull/11826#issuecomment-2565628138 @stevenzwu It is not yet ready for review. I want https://github.com/apache/iceberg/pull/11838 to merge before backporting. -- This is an automated message from the Apache Git Serv

Re: [PR] feat: support metadata table "snapshots" [iceberg-rust]

2024-12-30 Thread via GitHub
Xuanwo merged PR #822: URL: https://github.com/apache/iceberg-rust/pull/822 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.a

Re: [PR] feat: support metadata table "snapshots" [iceberg-rust]

2024-12-30 Thread via GitHub
Xuanwo commented on code in PR #822: URL: https://github.com/apache/iceberg-rust/pull/822#discussion_r1899617626 ## crates/iceberg/src/lib.rs: ## @@ -73,6 +73,7 @@ mod avro; pub mod io; pub mod spec; +pub mod metadata_scan; Review Comment: Should be changed in to `metada

Re: [PR] feat: support metadata table "snapshots" [iceberg-rust]

2024-12-30 Thread via GitHub
xxchan commented on PR #822: URL: https://github.com/apache/iceberg-rust/pull/822#issuecomment-2565569786 Thanks for the review. Updated. Please take a another look. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] feat: support metadata table "snapshots" [iceberg-rust]

2024-12-30 Thread via GitHub
Xuanwo commented on code in PR #822: URL: https://github.com/apache/iceberg-rust/pull/822#discussion_r1899563681 ## crates/iceberg/src/metadata_scan.rs: ## @@ -0,0 +1,256 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements

[PR] feat: configure codespell in pre-commit [iceberg-python]

2024-12-30 Thread via GitHub
IndexSeek opened a new pull request, #1478: URL: https://github.com/apache/iceberg-python/pull/1478 Following up with https://github.com/apache/iceberg-python/pull/1471 to detect spelling issues with pre-commit. I went with adding a `.codespellrc` file to store the `ignore-words-list

Re: [PR] feat: support metadata table "snapshots" [iceberg-rust]

2024-12-30 Thread via GitHub
flaneur2020 commented on code in PR #822: URL: https://github.com/apache/iceberg-rust/pull/822#discussion_r1899550638 ## crates/iceberg/src/metadata_scan.rs: ## @@ -0,0 +1,265 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agree

Re: [PR] feat: support metadata table "snapshots" [iceberg-rust]

2024-12-30 Thread via GitHub
flaneur2020 commented on code in PR #822: URL: https://github.com/apache/iceberg-rust/pull/822#discussion_r1899550638 ## crates/iceberg/src/metadata_scan.rs: ## @@ -0,0 +1,265 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agree

Re: [PR] docs: various spelling fixes [iceberg-python]

2024-12-30 Thread via GitHub
IndexSeek commented on PR #1471: URL: https://github.com/apache/iceberg-python/pull/1471#issuecomment-2565477428 > thank you! this tool is very helpful. Do you think its useful to include it as part of pre-commit hook? https://github.com/codespell-project/codespell?tab=readme-ov-file#pre-co

Re: [PR] feat: support metadata table "snapshots" [iceberg-rust]

2024-12-30 Thread via GitHub
xxchan commented on code in PR #822: URL: https://github.com/apache/iceberg-rust/pull/822#discussion_r1899524723 ## crates/iceberg/src/metadata_scan.rs: ## @@ -0,0 +1,265 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements

Re: [PR] feat: support metadata table "snapshots" [iceberg-rust]

2024-12-30 Thread via GitHub
xxchan commented on code in PR #822: URL: https://github.com/apache/iceberg-rust/pull/822#discussion_r1899521913 ## crates/iceberg/src/metadata_scan.rs: ## @@ -0,0 +1,265 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements

Re: [PR] feat: support metadata table "snapshots" [iceberg-rust]

2024-12-30 Thread via GitHub
xxchan commented on code in PR #822: URL: https://github.com/apache/iceberg-rust/pull/822#discussion_r1899506899 ## crates/iceberg/src/metadata_scan.rs: ## @@ -0,0 +1,265 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements

Re: [PR] feat: support metadata table "snapshots" [iceberg-rust]

2024-12-30 Thread via GitHub
Xuanwo commented on code in PR #822: URL: https://github.com/apache/iceberg-rust/pull/822#discussion_r1899497857 ## crates/iceberg/src/metadata_scan.rs: ## @@ -0,0 +1,265 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements

Re: [I] i meet a question,time travel has some error to the query of timestamp [iceberg]

2024-12-30 Thread via GitHub
KnightChess commented on issue #11889: URL: https://github.com/apache/iceberg/issues/11889#issuecomment-2565361920 `TimeTravelRelation`, this seems to be hudi's logical plan. could you confirm if hudi has been included? -- This is an automated message from the Apache Git Service. To respo

Re: [PR] feat: support metadata table "snapshots" [iceberg-rust]

2024-12-30 Thread via GitHub
xxchan commented on code in PR #822: URL: https://github.com/apache/iceberg-rust/pull/822#discussion_r1899477072 ## crates/iceberg/src/metadata_scan.rs: ## @@ -0,0 +1,265 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements

Re: [I] Iceberg has data consistency issues when processing CDC deleted records [iceberg]

2024-12-30 Thread via GitHub
beyond-up closed issue #11888: Iceberg has data consistency issues when processing CDC deleted records URL: https://github.com/apache/iceberg/issues/11888 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

Re: [PR] Add GitHub cpp-linter-action [iceberg-cpp]

2024-12-30 Thread via GitHub
raulcd commented on code in PR #20: URL: https://github.com/apache/iceberg-cpp/pull/20#discussion_r1899439993 ## api/iceberg/table.h: ## @@ -20,15 +20,24 @@ #pragma once #include -#include + Review Comment: I've been able to reproduce the memory error locally: ```

Re: [PR] Add GitHub cpp-linter-action [iceberg-cpp]

2024-12-30 Thread via GitHub
raulcd commented on PR #20: URL: https://github.com/apache/iceberg-cpp/pull/20#issuecomment-2565246479 > cpp-linter/cpp-linter-action@v2.13.4 is not allowed to be used in apache/iceberg-cpp. Thanks for trying! -- This is an automated message from the Apache Git Service. To resp

Re: [PR] Add GitHub cpp-linter-action [iceberg-cpp]

2024-12-30 Thread via GitHub
wgtmac commented on PR #20: URL: https://github.com/apache/iceberg-cpp/pull/20#issuecomment-2565241547 https://github.com/apache/iceberg-cpp/actions/runs/12544655200 cpp-linter/cpp-linter-action@v2.13.4 is not allowed to be used in apache/iceberg-cpp. -- This is an automated messag

Re: [PR] Add GitHub cpp-linter-action [iceberg-cpp]

2024-12-30 Thread via GitHub
wgtmac commented on code in PR #20: URL: https://github.com/apache/iceberg-cpp/pull/20#discussion_r1899408870 ## .github/workflows/cpp-linter.yml: ## @@ -0,0 +1,51 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See th

Re: [PR] Add GitHub cpp-linter-action [iceberg-cpp]

2024-12-30 Thread via GitHub
raulcd commented on code in PR #20: URL: https://github.com/apache/iceberg-cpp/pull/20#discussion_r1899401009 ## .github/workflows/cpp-linter.yml: ## @@ -0,0 +1,51 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See th

Re: [PR] feat: support metadata table "snapshots" [iceberg-rust]

2024-12-30 Thread via GitHub
Xuanwo commented on code in PR #822: URL: https://github.com/apache/iceberg-rust/pull/822#discussion_r1899399546 ## crates/iceberg/src/metadata_scan.rs: ## @@ -0,0 +1,265 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements

Re: [I] Add pre-commit and exercise on CI [iceberg-cpp]

2024-12-30 Thread via GitHub
raulcd closed issue #15: Add pre-commit and exercise on CI URL: https://github.com/apache/iceberg-cpp/issues/15 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

Re: [I] Add pre-commit and exercise on CI [iceberg-cpp]

2024-12-30 Thread via GitHub
raulcd commented on issue #15: URL: https://github.com/apache/iceberg-cpp/issues/15#issuecomment-2565215396 Yes, it was supposed to be closed with: https://github.com/apache/iceberg-cpp/pull/16 Thanks @zhjwpku -- This is an automated message from the Apache Git Service. To respond t

Re: [PR] Integrate Test Framework [iceberg-cpp]

2024-12-30 Thread via GitHub
raulcd commented on code in PR #13: URL: https://github.com/apache/iceberg-cpp/pull/13#discussion_r1899366861 ## ci/scripts/build_iceberg.sh: ## @@ -31,6 +31,7 @@ cmake \ -DICEBERG_BUILD_SHARED=ON \ ${source_dir} cmake --build . --target install +ctest --output-on-fai

Re: [I] Iceberg has data consistency issues when processing CDC deleted records [iceberg]

2024-12-30 Thread via GitHub
beyond-up commented on issue #11888: URL: https://github.com/apache/iceberg/issues/11888#issuecomment-2565144490 Add more sinformation : I tested and found that only partitioned tables have the above problems, while non-partitioned tables can handle delete records correctly. In this non-