Re: [PR] Core: Fix typo in HadoopTableOperations [iceberg]

2024-12-27 Thread via GitHub
ebyhr commented on code in PR #11880: URL: https://github.com/apache/iceberg/pull/11880#discussion_r1898764744 ## core/src/main/java/org/apache/iceberg/hadoop/HadoopTableOperations.java: ## @@ -344,7 +344,11 @@ int findVersion() { return maxVersion; } catch (IO

[I] RUSTSEC-2024-0388: `derivative` is unmaintained; consider using an alternative [iceberg-rust]

2024-12-27 Thread via GitHub
github-actions[bot] opened a new issue, #857: URL: https://github.com/apache/iceberg-rust/issues/857 > `derivative` is unmaintained; consider using an alternative | Details || | --- | ---

Re: [I] Discussion: make DataFile Serializable && Deserializable [iceberg-rust]

2024-12-27 Thread via GitHub
liurenjie1024 commented on issue #774: URL: https://github.com/apache/iceberg-rust/issues/774#issuecomment-2564155300 Thanks @ZENOTME for raising this. I'm also +1 for adding partition schema info to make it self contained and serializable. But please be careful not to expose the data struc

Re: [PR] release_rc.sh upload artifacts to apache dist [iceberg-go]

2024-12-27 Thread via GitHub
chil-pavn commented on code in PR #237: URL: https://github.com/apache/iceberg-go/pull/237#discussion_r1898815647 ## dev/release/release_rc.sh: ## @@ -65,7 +65,7 @@ fi rc_hash="$(git rev-list --max-count=1 "${rc_tag}")" -id="apache-iceberg-go-${version}-rc${rc}" +id="apache

Re: [I] Discussion: make DataFile Serializable && Deserializable [iceberg-rust]

2024-12-27 Thread via GitHub
liurenjie1024 commented on issue #774: URL: https://github.com/apache/iceberg-rust/issues/774#issuecomment-2564156137 Thanks @ZENOTME for raising this. In your question, the typical use case is compute engine, which needs to serialized `DataFile`. But in fact, what a compute engines require

[I] ArrowTypeError: "Could not convert" Error in inspect._files method [iceberg-python]

2024-12-27 Thread via GitHub
xsfa opened a new issue, #1477: URL: https://github.com/apache/iceberg-python/issues/1477 ### Apache Iceberg version 0.8.1 (latest release) ### Please describe the bug 🐞 I think PyArrow is receiving misformatted data from the file metadata, causing me to be unable to cal

Re: [PR] fix(catalog): delete metadata file when droping table in MemoryCatalog [iceberg-rust]

2024-12-27 Thread via GitHub
Xuanwo merged PR #854: URL: https://github.com/apache/iceberg-rust/pull/854 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.a

Re: [PR] fix: fix sql catalog drop table [iceberg-rust]

2024-12-27 Thread via GitHub
Li0k commented on PR #853: URL: https://github.com/apache/iceberg-rust/pull/853#issuecomment-2563570248 cc @Xuanwo @liurenjie1024 @ZENOTME -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

Re: [I] Add s3tables catalog support [iceberg-rust]

2024-12-27 Thread via GitHub
Xuanwo closed issue #754: Add s3tables catalog support URL: https://github.com/apache/iceberg-rust/issues/754 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-

Re: [PR] fix: fix sql catalog drop table [iceberg-rust]

2024-12-27 Thread via GitHub
Xuanwo merged PR #853: URL: https://github.com/apache/iceberg-rust/pull/853 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.a

Re: [PR] feat: add s3tables catalog [iceberg-rust]

2024-12-27 Thread via GitHub
Xuanwo merged PR #807: URL: https://github.com/apache/iceberg-rust/pull/807 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.a

Re: [PR] ci: add rust-cache action [iceberg-rust]

2024-12-27 Thread via GitHub
Xuanwo merged PR #844: URL: https://github.com/apache/iceberg-rust/pull/844 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.a

Re: [PR] Bump mkdocstrings-python from 1.12.2 to 1.13.0 [iceberg-python]

2024-12-27 Thread via GitHub
Fokko merged PR #1472: URL: https://github.com/apache/iceberg-python/pull/1472 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [PR] feat: add s3tables catalog [iceberg-rust]

2024-12-27 Thread via GitHub
flaneur2020 commented on PR #807: URL: https://github.com/apache/iceberg-rust/pull/807#issuecomment-2563579324 @Xuanwo added comments on all the public apis, PTAL -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[PR] Core: Fix typo in HadoopTableOperations [iceberg]

2024-12-27 Thread via GitHub
okumin opened a new pull request, #11880: URL: https://github.com/apache/iceberg/pull/11880 `version-hint.txt` should have been `version-hint.text`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[PR] Adding new rewrite manifest spark action to accept custom partition o… [iceberg]

2024-12-27 Thread via GitHub
zachdisc opened a new pull request, #11881: URL: https://github.com/apache/iceberg/pull/11881 **Note** this is a fresh PR replacing https://github.com/apache/iceberg/pull/9731. It had too much accumulated conflicts and changes, I rebased and messed it up. This is a clean start with all pre

Re: [PR] Spark: Adding simple custom partition sort order option to RewriteManifests Spark Action [iceberg]

2024-12-27 Thread via GitHub
zachdisc commented on PR #9731: URL: https://github.com/apache/iceberg/pull/9731#issuecomment-2563840124 Created a fresh PR with feedback here incorporated - https://github.com/apache/iceberg/pull/11881 This got messy with the accumulated changes and conflicts (paired with my likely

[PR] Bump moto from 5.0.24 to 5.0.25 [iceberg-python]

2024-12-27 Thread via GitHub
dependabot[bot] opened a new pull request, #1475: URL: https://github.com/apache/iceberg-python/pull/1475 Bumps [moto](https://github.com/getmoto/moto) from 5.0.24 to 5.0.25. Changelog Sourced from https://github.com/getmoto/moto/blob/master/CHANGELOG.md";>moto's changelog.

[PR] Bump boto3 from 1.35.81 to 1.35.88 [iceberg-python]

2024-12-27 Thread via GitHub
dependabot[bot] opened a new pull request, #1476: URL: https://github.com/apache/iceberg-python/pull/1476 Bumps [boto3](https://github.com/boto/boto3) from 1.35.81 to 1.35.88. Commits https://github.com/boto/boto3/commit/2693d3b98c1bcc565482cd0bee81d1309574b380";>2693d3b Merge

Re: [PR] build(deps): bump google.golang.org/api from 0.211.0 to 0.214.0 [iceberg-go]

2024-12-27 Thread via GitHub
zeroshade merged PR #234: URL: https://github.com/apache/iceberg-go/pull/234 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] build(deps): bump github.com/aws/aws-sdk-go-v2/config from 1.28.6 to 1.28.7 [iceberg-go]

2024-12-27 Thread via GitHub
zeroshade merged PR #232: URL: https://github.com/apache/iceberg-go/pull/232 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] build(deps): bump github.com/aws/aws-sdk-go-v2/service/glue from 1.104.0 to 1.104.1 [iceberg-go]

2024-12-27 Thread via GitHub
dependabot[bot] commented on PR #233: URL: https://github.com/apache/iceberg-go/pull/233#issuecomment-2563864191 Superseded by #238. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [PR] release_rc.sh upload artifacts to apache dist [iceberg-go]

2024-12-27 Thread via GitHub
zeroshade commented on code in PR #237: URL: https://github.com/apache/iceberg-go/pull/237#discussion_r1898617319 ## dev/release/release_rc.sh: ## @@ -65,7 +65,7 @@ fi rc_hash="$(git rev-list --max-count=1 "${rc_tag}")" -id="apache-iceberg-go-${version}-rc${rc}" +id="apache

Re: [PR] build(deps): bump github.com/aws/aws-sdk-go-v2/credentials from 1.17.47 to 1.17.48 [iceberg-go]

2024-12-27 Thread via GitHub
dependabot[bot] commented on PR #235: URL: https://github.com/apache/iceberg-go/pull/235#issuecomment-2563863559 Looks like github.com/aws/aws-sdk-go-v2/credentials is up-to-date now, so this is no longer needed. -- This is an automated message from the Apache Git Service. To respond to t

Re: [PR] build(deps): bump github.com/aws/aws-sdk-go-v2/service/glue from 1.104.0 to 1.104.1 [iceberg-go]

2024-12-27 Thread via GitHub
dependabot[bot] closed pull request #233: build(deps): bump github.com/aws/aws-sdk-go-v2/service/glue from 1.104.0 to 1.104.1 URL: https://github.com/apache/iceberg-go/pull/233 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[PR] build(deps): bump github.com/aws/aws-sdk-go-v2/service/glue from 1.104.0 to 1.105.0 [iceberg-go]

2024-12-27 Thread via GitHub
dependabot[bot] opened a new pull request, #238: URL: https://github.com/apache/iceberg-go/pull/238 Bumps [github.com/aws/aws-sdk-go-v2/service/glue](https://github.com/aws/aws-sdk-go-v2) from 1.104.0 to 1.105.0. Changelog Sourced from https://github.com/aws/aws-sdk-go-v2/blob/ser

Re: [PR] build(deps): bump github.com/aws/aws-sdk-go-v2/credentials from 1.17.47 to 1.17.48 [iceberg-go]

2024-12-27 Thread via GitHub
dependabot[bot] closed pull request #235: build(deps): bump github.com/aws/aws-sdk-go-v2/credentials from 1.17.47 to 1.17.48 URL: https://github.com/apache/iceberg-go/pull/235 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

Re: [PR] build(deps): bump github.com/aws/aws-sdk-go-v2 from 1.32.6 to 1.32.7 [iceberg-go]

2024-12-27 Thread via GitHub
dependabot[bot] commented on PR #236: URL: https://github.com/apache/iceberg-go/pull/236#issuecomment-2563863599 Looks like github.com/aws/aws-sdk-go-v2 is up-to-date now, so this is no longer needed. -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] build(deps): bump github.com/aws/aws-sdk-go-v2 from 1.32.6 to 1.32.7 [iceberg-go]

2024-12-27 Thread via GitHub
dependabot[bot] closed pull request #236: build(deps): bump github.com/aws/aws-sdk-go-v2 from 1.32.6 to 1.32.7 URL: https://github.com/apache/iceberg-go/pull/236 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

Re: [PR] build(deps): bump github.com/aws/aws-sdk-go-v2/service/glue from 1.104.0 to 1.105.0 [iceberg-go]

2024-12-27 Thread via GitHub
zeroshade merged PR #238: URL: https://github.com/apache/iceberg-go/pull/238 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

[PR] API, Core: Support keeping at most N snapshots in ExpireSnapshots [iceberg]

2024-12-27 Thread via GitHub
manuzhang opened a new pull request, #11879: URL: https://github.com/apache/iceberg/pull/11879 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

[PR] fix(catalog): delete metadata file when droping table in MemoryCatalog [iceberg-rust]

2024-12-27 Thread via GitHub
lewiszlw opened a new pull request, #854: URL: https://github.com/apache/iceberg-rust/pull/854 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

Re: [PR] fix(catalog): delete metadata file when droping table in MemoryCatalog [iceberg-rust]

2024-12-27 Thread via GitHub
Xuanwo commented on PR #854: URL: https://github.com/apache/iceberg-rust/pull/854#issuecomment-2563801643 Should merge after https://github.com/apache/iceberg-rust/pull/855 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[I] How to apply partition/bloom filter to old data? Does rewrite_data_files/rewrite_manifests procedure work? [iceberg]

2024-12-27 Thread via GitHub
madeirak opened a new issue, #11878: URL: https://github.com/apache/iceberg/issues/11878 ### Query engine Spark ### Question rewrite_data_files/rewrite_manifests procedure DOC: https://iceberg.apache.org/docs/latest/spark-procedures/#rewrite_manifests How to ap

Re: [PR] Add pyspark DataFusion integration test [iceberg-rust]

2024-12-27 Thread via GitHub
gruuya commented on code in PR #850: URL: https://github.com/apache/iceberg-rust/pull/850#discussion_r1898355475 ## crates/integration_tests/tests/datafusion.rs: ## @@ -0,0 +1,138 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license a

Re: [PR] release_rc.sh upload artifacts to apache dist [iceberg-go]

2024-12-27 Thread via GitHub
chil-pavn commented on code in PR #237: URL: https://github.com/apache/iceberg-go/pull/237#discussion_r1898822266 ## dev/release/release_rc.sh: ## @@ -65,7 +65,7 @@ fi rc_hash="$(git rev-list --max-count=1 "${rc_tag}")" -id="apache-iceberg-go-${version}-rc${rc}" +id="apache

[PR] Update README.md to add link to iceberg-cpp [iceberg]

2024-12-27 Thread via GitHub
gabeiglio opened a new pull request, #11882: URL: https://github.com/apache/iceberg/pull/11882 Add the new iceberg-cpp effort to the readme -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

Re: [PR] chore: Fix cargo.lock not updated [iceberg-rust]

2024-12-27 Thread via GitHub
liurenjie1024 merged PR #855: URL: https://github.com/apache/iceberg-rust/pull/855 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [PR] Kafka Connect: Add table to topics mapping property [iceberg]

2024-12-27 Thread via GitHub
github-actions[bot] commented on PR #10422: URL: https://github.com/apache/iceberg/pull/10422#issuecomment-2564111397 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [I] SnapshotTableProcedure to migrate iceberg tables from one namespace to another [iceberg]

2024-12-27 Thread via GitHub
github-actions[bot] commented on issue #10262: URL: https://github.com/apache/iceberg/issues/10262#issuecomment-2564111395 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache

Re: [I] SnapshotTableProcedure to migrate iceberg tables from one namespace to another [iceberg]

2024-12-27 Thread via GitHub
github-actions[bot] closed issue #10262: SnapshotTableProcedure to migrate iceberg tables from one namespace to another URL: https://github.com/apache/iceberg/issues/10262 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

Re: [PR] ci: add sccache to speed up ci build [iceberg-rust]

2024-12-27 Thread via GitHub
xxchan commented on PR #824: URL: https://github.com/apache/iceberg-rust/pull/824#issuecomment-2564164172 replaced by https://github.com/apache/iceberg-rust/pull/844 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] Add orbstack guide [iceberg-rust]

2024-12-27 Thread via GitHub
xxchan commented on code in PR #856: URL: https://github.com/apache/iceberg-rust/pull/856#discussion_r1898788972 ## CONTRIBUTING.md: ## @@ -116,7 +116,7 @@ cargo 1.69.0 (6e9a83356 2023-04-12) Currently, iceberg-rust uses Docker to set up environment for integration tests. Na

Re: [PR] ci: add sccache to speed up ci build [iceberg-rust]

2024-12-27 Thread via GitHub
xxchan closed pull request #824: ci: add sccache to speed up ci build URL: https://github.com/apache/iceberg-rust/pull/824 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

Re: [PR] Add orbstack guide [iceberg-rust]

2024-12-27 Thread via GitHub
xxchan commented on code in PR #856: URL: https://github.com/apache/iceberg-rust/pull/856#discussion_r1898788972 ## CONTRIBUTING.md: ## @@ -116,7 +116,7 @@ cargo 1.69.0 (6e9a83356 2023-04-12) Currently, iceberg-rust uses Docker to set up environment for integration tests. Na

Re: [PR] Core: Fix typo in HadoopTableOperations [iceberg]

2024-12-27 Thread via GitHub
okumin commented on code in PR #11880: URL: https://github.com/apache/iceberg/pull/11880#discussion_r1898789437 ## core/src/main/java/org/apache/iceberg/hadoop/HadoopTableOperations.java: ## @@ -344,7 +344,7 @@ int findVersion() { return maxVersion; } catch (IO

Re: [PR] Core: Fix typo in HadoopTableOperations [iceberg]

2024-12-27 Thread via GitHub
okumin commented on code in PR #11880: URL: https://github.com/apache/iceberg/pull/11880#discussion_r1898789437 ## core/src/main/java/org/apache/iceberg/hadoop/HadoopTableOperations.java: ## @@ -344,7 +344,7 @@ int findVersion() { return maxVersion; } catch (IO

Re: [PR] Add GitHub cpp-linter-action [iceberg-cpp]

2024-12-27 Thread via GitHub
lidavidm commented on PR #20: URL: https://github.com/apache/iceberg-cpp/pull/20#issuecomment-2564236592 Hmm, in that screenshot, 'memory not found' seems like a spurious error? Will this be noisy? -- This is an automated message from the Apache Git Service. To respond to the message, ple

Re: [PR] Kafka Connect: Add table to topics mapping property [iceberg]

2024-12-27 Thread via GitHub
igorvoltaic commented on PR #10422: URL: https://github.com/apache/iceberg/pull/10422#issuecomment-2564185552 Ping -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsub

Re: [PR] Core: Fix typo in HadoopTableOperations [iceberg]

2024-12-27 Thread via GitHub
okumin commented on code in PR #11880: URL: https://github.com/apache/iceberg/pull/11880#discussion_r1898786693 ## core/src/main/java/org/apache/iceberg/hadoop/HadoopTableOperations.java: ## @@ -344,7 +344,11 @@ int findVersion() { return maxVersion; } catch (I

Re: [I] Dectect schema evolution or partition evolution for append DataFile [iceberg-rust]

2024-12-27 Thread via GitHub
liurenjie1024 commented on issue #777: URL: https://github.com/apache/iceberg-rust/issues/777#issuecomment-2564153339 Thanks @ZENOTME for raising this. > The partition in DataFile should include types to facilitate validation. e.g. the field name and field id Do you mean we sh