Re: [PR] Add `write_parquet` API for writing Parquet files without committing [iceberg-python]

2025-03-02 Thread via GitHub
andormarkus commented on PR #1742: URL: https://github.com/apache/iceberg-python/pull/1742#issuecomment-2693512105 Hi @Fokko We ended up to build a dynamic custom serialize / deserialize function which supports gzip and zlib compression to deal with size issue. Once the `DataFile` i

[I] Support ADD_COLUMN at particular index [iceberg-python]

2025-03-02 Thread via GitHub
dvnageshpatil opened a new issue, #1749: URL: https://github.com/apache/iceberg-python/issues/1749 ### Feature Request / Improvement There are times when I want to add a column but place it near an associated column in table ,not at the end. -- This is an automated message from the

Re: [I] Support adding parquet files to partitioned table [iceberg-rust]

2025-03-02 Thread via GitHub
liurenjie1024 commented on issue #1035: URL: https://github.com/apache/iceberg-rust/issues/1035#issuecomment-2693433181 > I can also work on this Thanks ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[I] Add an option to skip checking duplicated files when adding existing file in `FastAppendAction`. [iceberg-rust]

2025-03-02 Thread via GitHub
liurenjie1024 opened a new issue, #1031: URL: https://github.com/apache/iceberg-rust/issues/1031 ### Is your feature request related to a problem or challenge? See https://github.com/apache/iceberg-rust/pull/960#discussion_r1976556165 Checking duplicated files in fast append action m

Re: [I] Support adding parquet files to partitioned table [iceberg-rust]

2025-03-02 Thread via GitHub
jonathanc-n commented on issue #1035: URL: https://github.com/apache/iceberg-rust/issues/1035#issuecomment-2693312422 I can also work on this -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

Re: [PR] chore(deps): Bump uuid from 1.13.2 to 1.15.1 [iceberg-rust]

2025-03-02 Thread via GitHub
liurenjie1024 closed pull request #1023: chore(deps): Bump uuid from 1.13.2 to 1.15.1 URL: https://github.com/apache/iceberg-rust/pull/1023 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

[I] Support adding parquet files to partitioned table [iceberg-rust]

2025-03-02 Thread via GitHub
jonathanc-n opened a new issue, #1035: URL: https://github.com/apache/iceberg-rust/issues/1035 Looking at the `type_to_literal` function, I think this is correct. We don't want to set default values in the partition spec. I think there are two options: - Don't allow app

[PR] feat: Make duplicate check optional for adding parquet files [iceberg-rust]

2025-03-02 Thread via GitHub
jonathanc-n opened a new pull request, #1034: URL: https://github.com/apache/iceberg-rust/pull/1034 ## Which issue does this PR close? - Closes #1031 . ## What changes are included in this PR? Added option for checking duplicates when adding parquet files.

Re: [I] Iceberg-rust Write support [iceberg-rust]

2025-03-02 Thread via GitHub
takaebato commented on issue #700: URL: https://github.com/apache/iceberg-rust/issues/700#issuecomment-2693284995 @jonathanc-n Thanks!! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

Re: [I] Check parquet file schema compatibility when adding parquet file in fast append action. [iceberg-rust]

2025-03-02 Thread via GitHub
liurenjie1024 commented on issue #1030: URL: https://github.com/apache/iceberg-rust/issues/1030#issuecomment-2693266105 > Can i get this assigned to me? Done, thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] fix: Remove license from pull request template [iceberg-rust]

2025-03-02 Thread via GitHub
liurenjie1024 merged PR #1032: URL: https://github.com/apache/iceberg-rust/pull/1032 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@

[I] Consolidate methods of converting parquet file to data file builder. [iceberg-rust]

2025-03-02 Thread via GitHub
liurenjie1024 opened a new issue, #1033: URL: https://github.com/apache/iceberg-rust/issues/1033 ### Is your feature request related to a problem or challenge? In #960 , we introduce [another method](https://github.com/apache/iceberg-rust/blob/3f2e61e3de93941fff3127893297f446acf20d0e/

Re: [PR] Core: lazy init workerPool [iceberg]

2025-03-02 Thread via GitHub
abstractdog commented on PR #12427: URL: https://github.com/apache/iceberg/pull/12427#issuecomment-2691234726 > I have seen a similar issue here: #12220 thanks, it looks like the very same problem, so by using the default worker pool, we need to accept the default "exiting" behavior,

Re: [PR] feat: Add existing parquet files [iceberg-rust]

2025-03-02 Thread via GitHub
liurenjie1024 merged PR #960: URL: https://github.com/apache/iceberg-rust/pull/960 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [PR] feat: Add existing parquet files [iceberg-rust]

2025-03-02 Thread via GitHub
jonathanc-n commented on PR #960: URL: https://github.com/apache/iceberg-rust/pull/960#issuecomment-2693215983 @liurenjie1024 Thank you for all the reviews, should be good now. I will make the follow up prs after merge. -- This is an automated message from the Apache Git Service. To respo

Re: [I] Getting "offset overflow while concatenating arrays" Error when writing to iceberg [iceberg-python]

2025-03-02 Thread via GitHub
heman026 commented on issue #1733: URL: https://github.com/apache/iceberg-python/issues/1733#issuecomment-2693244579 I will check this with the 0.9.0 RC and let you know if this is working fine. -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [I] Iceberg-rust Write support [iceberg-rust]

2025-03-02 Thread via GitHub
jonathanc-n commented on issue #700: URL: https://github.com/apache/iceberg-rust/issues/700#issuecomment-2693259633 @takaebato Feel free to comment under an issue that you want to work on that hasn't been touched yet, and a maintainer can assign it to you! -- This is an automated message

[PR] fix: Remove license from pull request template [iceberg-rust]

2025-03-02 Thread via GitHub
jonathanc-n opened a new pull request, #1032: URL: https://github.com/apache/iceberg-rust/pull/1032 ## Which issue does this PR close? - Closes #. ## What changes are included in this PR? Removed the license from the pull request template. (the license is

[I] Check parquet file compatibility when adding parquet file in fast append action. [iceberg-rust]

2025-03-02 Thread via GitHub
liurenjie1024 opened a new issue, #1030: URL: https://github.com/apache/iceberg-rust/issues/1030 ### Is your feature request related to a problem or challenge? In https://github.com/apache/iceberg-rust/pull/960 we introduced a feature of adding existing parquet files in `FastAppendAct

Re: [PR] feat: Add existing parquet files [iceberg-rust]

2025-03-02 Thread via GitHub
liurenjie1024 commented on code in PR #960: URL: https://github.com/apache/iceberg-rust/pull/960#discussion_r1976845845 ## crates/iceberg/src/transaction.rs: ## @@ -205,8 +208,86 @@ impl<'a> FastAppendAction<'a> { Ok(self) } +/// Adds existing parquet files +

Re: [I] Check parquet file schema compatibility when adding parquet file in fast append action. [iceberg-rust]

2025-03-02 Thread via GitHub
jonathanc-n commented on issue #1030: URL: https://github.com/apache/iceberg-rust/issues/1030#issuecomment-2693224694 Can i get this assigned to me? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

Re: [PR] fix: fix http custom headers for rest catalog [iceberg-rust]

2025-03-02 Thread via GitHub
liurenjie1024 commented on code in PR #1010: URL: https://github.com/apache/iceberg-rust/pull/1010#discussion_r1976827642 ## crates/catalog/rest/src/catalog.rs: ## @@ -160,22 +160,28 @@ impl RestCatalogConfig { // The unwrap here is same since we are filtering the k

Re: [I] RUSTSEC-2025-0007: *ring* is unmaintained [iceberg-rust]

2025-03-02 Thread via GitHub
liurenjie1024 closed issue #985: RUSTSEC-2025-0007: *ring* is unmaintained URL: https://github.com/apache/iceberg-rust/issues/985 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] feat: support position delete writer [iceberg-rust]

2025-03-02 Thread via GitHub
liurenjie1024 commented on code in PR #704: URL: https://github.com/apache/iceberg-rust/pull/704#discussion_r1976816015 ## crates/iceberg/src/writer/base_writer/position_delete_file_writer.rs: ## @@ -0,0 +1,277 @@ +// Licensed to the Apache Software Foundation (ASF) under one +/

Re: [I] RUSTSEC-2025-0007: *ring* is unmaintained [iceberg-rust]

2025-03-02 Thread via GitHub
liurenjie1024 commented on issue #985: URL: https://github.com/apache/iceberg-rust/issues/985#issuecomment-2693163519 Thanks @xxchan , we can close this now. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

Re: [PR] chore(deps): Bump aws-sdk-s3tables from 1.10.0 to 1.11.0 [iceberg-rust]

2025-03-02 Thread via GitHub
dependabot[bot] commented on PR #1025: URL: https://github.com/apache/iceberg-rust/pull/1025#issuecomment-2693129840 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version

Re: [PR] chore(deps): Bump aws-sdk-glue from 1.82.0 to 1.83.0 [iceberg-rust]

2025-03-02 Thread via GitHub
dependabot[bot] commented on PR #1022: URL: https://github.com/apache/iceberg-rust/pull/1022#issuecomment-2693130588 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version

Re: [PR] feat: support delete if empty for parquet writer [iceberg-rust]

2025-03-02 Thread via GitHub
liurenjie1024 commented on code in PR #838: URL: https://github.com/apache/iceberg-rust/pull/838#discussion_r1976804654 ## crates/iceberg/src/io/file_io.rs: ## @@ -334,6 +334,11 @@ impl OutputFile { Ok(self.op.exists(&self.path[self.relative_path_pos..]).await?) }

Re: [PR] chore(deps): Bump either from 1.13.0 to 1.14.0 [iceberg-rust]

2025-03-02 Thread via GitHub
dependabot[bot] commented on PR #1027: URL: https://github.com/apache/iceberg-rust/pull/1027#issuecomment-2693127783 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version

Re: [PR] test: Introduce integration test framework. [iceberg-rust]

2025-03-02 Thread via GitHub
liurenjie1024 commented on PR #581: URL: https://github.com/apache/iceberg-rust/pull/581#issuecomment-2693131326 > We already have one `integration-test` (with spark) data `datafusion-integration-test`. They seem to overlap to some extent. Perhaps we can decide to all in one of them in the

Re: [PR] chore(deps): Bump uuid from 1.13.2 to 1.15.1 [iceberg-rust]

2025-03-02 Thread via GitHub
dependabot[bot] commented on PR #1023: URL: https://github.com/apache/iceberg-rust/pull/1023#issuecomment-2693130169 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version

Re: [PR] chore(deps): Bump aws-sdk-glue from 1.82.0 to 1.83.0 [iceberg-rust]

2025-03-02 Thread via GitHub
liurenjie1024 closed pull request #1022: chore(deps): Bump aws-sdk-glue from 1.82.0 to 1.83.0 URL: https://github.com/apache/iceberg-rust/pull/1022 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

Re: [PR] chore(deps): Bump aws-sdk-s3tables from 1.10.0 to 1.11.0 [iceberg-rust]

2025-03-02 Thread via GitHub
liurenjie1024 closed pull request #1025: chore(deps): Bump aws-sdk-s3tables from 1.10.0 to 1.11.0 URL: https://github.com/apache/iceberg-rust/pull/1025 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

Re: [PR] chore(deps): Bump either from 1.13.0 to 1.14.0 [iceberg-rust]

2025-03-02 Thread via GitHub
liurenjie1024 commented on PR #1027: URL: https://github.com/apache/iceberg-rust/pull/1027#issuecomment-2693127756 Security audit failed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

Re: [PR] chore(deps): Bump either from 1.13.0 to 1.14.0 [iceberg-rust]

2025-03-02 Thread via GitHub
liurenjie1024 closed pull request #1027: chore(deps): Bump either from 1.13.0 to 1.14.0 URL: https://github.com/apache/iceberg-rust/pull/1027 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

Re: [PR] feat: Add existing parquet files [iceberg-rust]

2025-03-02 Thread via GitHub
liurenjie1024 commented on code in PR #960: URL: https://github.com/apache/iceberg-rust/pull/960#discussion_r1976795860 ## crates/iceberg/src/writer/file_writer/parquet_writer.rs: ## @@ -391,6 +442,79 @@ impl ParquetWriter { ); Ok(builder) } + +///

Re: [PR] chore(deps): Bump arrow-schema from 54.2.0 to 54.2.1 [iceberg-rust]

2025-03-02 Thread via GitHub
dependabot[bot] commented on PR #1024: URL: https://github.com/apache/iceberg-rust/pull/1024#issuecomment-2693103224 Looks like arrow-schema is no longer being updated by Dependabot, so this is no longer needed. -- This is an automated message from the Apache Git Service. To respond to th

Re: [PR] chore(deps): fix bump typos to v1.30.0 [iceberg-rust]

2025-03-02 Thread via GitHub
liurenjie1024 merged PR #1029: URL: https://github.com/apache/iceberg-rust/pull/1029 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@

Re: [PR] build(deps): bump the gomod_updates group with 7 updates [iceberg-go]

2025-03-02 Thread via GitHub
zeroshade merged PR #318: URL: https://github.com/apache/iceberg-go/pull/318 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] chore(deps): Bump aws-config from 1.5.16 to 1.5.17 [iceberg-rust]

2025-03-02 Thread via GitHub
dependabot[bot] commented on PR #1026: URL: https://github.com/apache/iceberg-rust/pull/1026#issuecomment-2693103233 Looks like aws-config is no longer being updated by Dependabot, so this is no longer needed. -- This is an automated message from the Apache Git Service. To respond to the

Re: [I] Flaky test `TestHadoopCommits > testConcurrentFastAppends(File)` [iceberg]

2025-03-02 Thread via GitHub
manuzhang commented on issue #11047: URL: https://github.com/apache/iceberg/issues/11047#issuecomment-2693112328 Reopened for recent failure https://github.com/apache/iceberg/actions/runs/13618574381/job/38064821208 ``` TestHadoopCommits > testConcurrentFastAppends(File) FAILED

Re: [PR] chore(deps): Bump arrow-array from 54.2.0 to 54.2.1 [iceberg-rust]

2025-03-02 Thread via GitHub
dependabot[bot] closed pull request #1020: chore(deps): Bump arrow-array from 54.2.0 to 54.2.1 URL: https://github.com/apache/iceberg-rust/pull/1020 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

Re: [PR] chore(deps): Bump pilota from 0.11.8 to 0.11.10 [iceberg-rust]

2025-03-02 Thread via GitHub
dependabot[bot] closed pull request #1021: chore(deps): Bump pilota from 0.11.8 to 0.11.10 URL: https://github.com/apache/iceberg-rust/pull/1021 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

[I] Flaky test `TestHadoopCommits > testConcurrentFastAppends(File)` [iceberg]

2025-03-02 Thread via GitHub
manuzhang opened a new issue, #11047: URL: https://github.com/apache/iceberg/issues/11047 ### Apache Iceberg version main (development) ### Query engine None ### Please describe the bug ๐Ÿž ``` 14:26:54 TestHadoopCommits > testConcurrentFastAppends(File) FA

Re: [PR] Hive: Add Hive 4 support and remove Hive runtime [iceberg]

2025-03-02 Thread via GitHub
manuzhang commented on PR #11750: URL: https://github.com/apache/iceberg/pull/11750#issuecomment-2693109726 @deniskuzZ yes, I'm planning to work on Hive 4 support. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

Re: [PR] chore(deps): Bump chrono from 0.4.39 to 0.4.40 [iceberg-rust]

2025-03-02 Thread via GitHub
dependabot[bot] closed pull request #1019: chore(deps): Bump chrono from 0.4.39 to 0.4.40 URL: https://github.com/apache/iceberg-rust/pull/1019 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

Re: [PR] chore(deps): Bump arrow-buffer from 54.2.0 to 54.2.1 [iceberg-rust]

2025-03-02 Thread via GitHub
dependabot[bot] closed pull request #1028: chore(deps): Bump arrow-buffer from 54.2.0 to 54.2.1 URL: https://github.com/apache/iceberg-rust/pull/1028 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] ci(dependabot): Ignore all patch updates for iceberg-rust [iceberg-rust]

2025-03-02 Thread via GitHub
liurenjie1024 merged PR #1001: URL: https://github.com/apache/iceberg-rust/pull/1001 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@

Re: [PR] chore(deps): Bump pilota from 0.11.8 to 0.11.10 [iceberg-rust]

2025-03-02 Thread via GitHub
dependabot[bot] commented on PR #1021: URL: https://github.com/apache/iceberg-rust/pull/1021#issuecomment-2693103227 Looks like pilota is no longer being updated by Dependabot, so this is no longer needed. -- This is an automated message from the Apache Git Service. To respond to the mess

Re: [PR] chore(deps): Bump arrow-buffer from 54.2.0 to 54.2.1 [iceberg-rust]

2025-03-02 Thread via GitHub
dependabot[bot] commented on PR #1028: URL: https://github.com/apache/iceberg-rust/pull/1028#issuecomment-2693103231 Looks like arrow-buffer is no longer being updated by Dependabot, so this is no longer needed. -- This is an automated message from the Apache Git Service. To respond to th

Re: [PR] chore(deps): Bump aws-config from 1.5.16 to 1.5.17 [iceberg-rust]

2025-03-02 Thread via GitHub
dependabot[bot] closed pull request #1026: chore(deps): Bump aws-config from 1.5.16 to 1.5.17 URL: https://github.com/apache/iceberg-rust/pull/1026 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

Re: [PR] chore(deps): Bump arrow-array from 54.2.0 to 54.2.1 [iceberg-rust]

2025-03-02 Thread via GitHub
dependabot[bot] commented on PR #1020: URL: https://github.com/apache/iceberg-rust/pull/1020#issuecomment-2693103222 Looks like arrow-array is no longer being updated by Dependabot, so this is no longer needed. -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] chore(deps): Bump arrow-schema from 54.2.0 to 54.2.1 [iceberg-rust]

2025-03-02 Thread via GitHub
dependabot[bot] closed pull request #1024: chore(deps): Bump arrow-schema from 54.2.0 to 54.2.1 URL: https://github.com/apache/iceberg-rust/pull/1024 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] chore(deps): Bump chrono from 0.4.39 to 0.4.40 [iceberg-rust]

2025-03-02 Thread via GitHub
dependabot[bot] commented on PR #1019: URL: https://github.com/apache/iceberg-rust/pull/1019#issuecomment-2693103221 Looks like chrono is no longer being updated by Dependabot, so this is no longer needed. -- This is an automated message from the Apache Git Service. To respond to the mess

Re: [PR] chore(deps): Bump crate-ci/typos from 1.29.7 to 1.30.0 [iceberg-rust]

2025-03-02 Thread via GitHub
liurenjie1024 closed pull request #1018: chore(deps): Bump crate-ci/typos from 1.29.7 to 1.30.0 URL: https://github.com/apache/iceberg-rust/pull/1018 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] chore(deps): Bump crate-ci/typos from 1.29.7 to 1.30.0 [iceberg-rust]

2025-03-02 Thread via GitHub
dependabot[bot] commented on PR #1018: URL: https://github.com/apache/iceberg-rust/pull/1018#issuecomment-2693101002 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version

Re: [PR] chore(deps): Bump crate-ci/typos from 1.29.7 to 1.30.0 [iceberg-rust]

2025-03-02 Thread via GitHub
dependabot[bot] commented on PR #1018: URL: https://github.com/apache/iceberg-rust/pull/1018#issuecomment-2693101007 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version

Re: [PR] chore(deps): Bump crate-ci/typos from 1.29.7 to 1.30.0 [iceberg-rust]

2025-03-02 Thread via GitHub
liurenjie1024 closed pull request #1018: chore(deps): Bump crate-ci/typos from 1.29.7 to 1.30.0 URL: https://github.com/apache/iceberg-rust/pull/1018 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [I] Review new ImmutablesReferenceEquality error-prone check [iceberg]

2025-03-02 Thread via GitHub
github-actions[bot] commented on issue #10855: URL: https://github.com/apache/iceberg/issues/10855#issuecomment-2692989602 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [I] [feature request] Automatically Refreshable AWS credential [iceberg-python]

2025-03-02 Thread via GitHub
github-actions[bot] commented on issue #1129: URL: https://github.com/apache/iceberg-python/issues/1129#issuecomment-2692991568 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity

Re: [PR] ORC: Fix null map values and list elements in vectorized reads [iceberg]

2025-03-02 Thread via GitHub
github-actions[bot] closed pull request #12030: ORC: Fix null map values and list elements in vectorized reads URL: https://github.com/apache/iceberg/pull/12030 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

Re: [I] Iceberg Spark streaming skips rows of data [iceberg]

2025-03-02 Thread via GitHub
github-actions[bot] commented on issue #10156: URL: https://github.com/apache/iceberg/issues/10156#issuecomment-2692989565 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [PR] Core: Checks for Equality Delete when Row Lineage is Enabled - Using Snapshot Summary [iceberg]

2025-03-02 Thread via GitHub
github-actions[bot] closed pull request #12075: Core: Checks for Equality Delete when Row Lineage is Enabled - Using Snapshot Summary URL: https://github.com/apache/iceberg/pull/12075 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

Re: [PR] Kafka Connect: Add delta writer support [iceberg]

2025-03-02 Thread via GitHub
github-actions[bot] commented on PR #12070: URL: https://github.com/apache/iceberg/pull/12070#issuecomment-2692989754 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

Re: [I] Kafka: runtime integration test failure or flaky [iceberg]

2025-03-02 Thread via GitHub
github-actions[bot] commented on issue #11046: URL: https://github.com/apache/iceberg/issues/11046#issuecomment-2692989662 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [PR] Parquet: Fix Reader leak by removing useless copy [iceberg]

2025-03-02 Thread via GitHub
github-actions[bot] commented on PR #12079: URL: https://github.com/apache/iceberg/pull/12079#issuecomment-2692989780 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think thatโ€™s incorrect or this pul

Re: [PR] Kafka Connect: Add delta writer support [iceberg]

2025-03-02 Thread via GitHub
github-actions[bot] closed pull request #12070: Kafka Connect: Add delta writer support URL: https://github.com/apache/iceberg/pull/12070 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] Core: Checks for Equality Delete when Row Lineage is Enabled - Using Snapshot Summary [iceberg]

2025-03-02 Thread via GitHub
github-actions[bot] commented on PR #12075: URL: https://github.com/apache/iceberg/pull/12075#issuecomment-2692989773 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

Re: [PR] Ignore partition fields that are dropped from the current-schema [iceberg]

2025-03-02 Thread via GitHub
github-actions[bot] commented on PR #11868: URL: https://github.com/apache/iceberg/pull/11868#issuecomment-2692989706 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think thatโ€™s incorrect or this pul

Re: [I] Add aliyun-bundle jar [iceberg]

2025-03-02 Thread via GitHub
github-actions[bot] commented on issue #10970: URL: https://github.com/apache/iceberg/issues/10970#issuecomment-2692989624 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache

Re: [PR] ORC: Fix null map values and list elements in vectorized reads [iceberg]

2025-03-02 Thread via GitHub
github-actions[bot] commented on PR #12030: URL: https://github.com/apache/iceberg/pull/12030#issuecomment-2692989731 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

Re: [I] Add aliyun-bundle jar [iceberg]

2025-03-02 Thread via GitHub
github-actions[bot] closed issue #10970: Add aliyun-bundle jar URL: https://github.com/apache/iceberg/issues/10970 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscrib

Re: [I] Iceberg table not able to read data from S3 after few hours using Athena . [iceberg]

2025-03-02 Thread via GitHub
github-actions[bot] commented on issue #9684: URL: https://github.com/apache/iceberg/issues/9684#issuecomment-2692989547 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] Check if dependencies in libs.versions.toml are the latest supported ones for JDK11 [iceberg]

2025-03-02 Thread via GitHub
chengyitang commented on issue #10852: URL: https://github.com/apache/iceberg/issues/10852#issuecomment-2692946639 Hi team, I noticed this issue has been open for a while. I'd like to work on it if it's still available. Thanks! -- This is an automated message from the Apache Gi

Re: [PR] Support execution in Windows using Local File System and NFS [iceberg-python]

2025-03-02 Thread via GitHub
ghost closed pull request #996: Support execution in Windows using Local File System and NFS URL: https://github.com/apache/iceberg-python/pull/996 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

Re: [PR] Manual deduction of partitions [iceberg-python]

2025-03-02 Thread via GitHub
afiodorov commented on PR #1743: URL: https://github.com/apache/iceberg-python/pull/1743#issuecomment-2692884445 Hey! The regex was just an example, it's not part of the API though - the partition deduction function is. The issue I am having at a works is that our pipelines keep writi

Re: [PR] Build: Bump datamodel-code-generator from 0.28.1 to 0.28.2 [iceberg]

2025-03-02 Thread via GitHub
Fokko merged PR #12433: URL: https://github.com/apache/iceberg/pull/12433 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] Build: Bump org.openapitools:openapi-generator-gradle-plugin from 7.11.0 to 7.12.0 [iceberg]

2025-03-02 Thread via GitHub
Fokko merged PR #12435: URL: https://github.com/apache/iceberg/pull/12435 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] Build: Bump org.mongodb:bson from 4.11.0 to 4.11.5 [iceberg]

2025-03-02 Thread via GitHub
Fokko commented on PR #12438: URL: https://github.com/apache/iceberg/pull/12438#issuecomment-2692868936 We don't have to update the LICENSE since the patch version is not included: https://github.com/apache/iceberg/blob/dffce80ae15469a904fde91409deb8790b30c296/kafka-connect/kafka-connect-run

Re: [I] Support scanning a specific branch of a table [iceberg-python]

2025-03-02 Thread via GitHub
Fokko commented on issue #1746: URL: https://github.com/apache/iceberg-python/issues/1746#issuecomment-2692863452 Hey @malcolmbovey I think that would be a great addition ๐Ÿ‘ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [I] Support Arrow `string_view` and `bin_view` types. [iceberg-python]

2025-03-02 Thread via GitHub
Fokko commented on issue #1745: URL: https://github.com/apache/iceberg-python/issues/1745#issuecomment-2692863052 Thanks @ritchie46 for raising this, I took the liberty of creating a PR: https://github.com/apache/iceberg-python/pull/1748 -- This is an automated message from the Apache Gi

[PR] Support `string_view` and `binary_view` [iceberg-python]

2025-03-02 Thread via GitHub
Fokko opened a new pull request, #1748: URL: https://github.com/apache/iceberg-python/pull/1748 Resolves #1745 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscrib

[PR] chore(deps): fix bump typos to v1.30.0 [iceberg-rust]

2025-03-02 Thread via GitHub
jonathanc-n opened a new pull request, #1029: URL: https://github.com/apache/iceberg-rust/pull/1029 ## Which issue does this PR close? - Closes #. ## What changes are included in this PR? ## Are these changes tested? -- This is an automated mess

Re: [PR] Add `write_parquet` API for writing Parquet files without committing [iceberg-python]

2025-03-02 Thread via GitHub
Fokko commented on PR #1742: URL: https://github.com/apache/iceberg-python/pull/1742#issuecomment-2692859499 Thanks @andormarkus for working on this, I've just commented on #1737, would love to hear your thoughts there as well. > Passing DataFile objects between processes requires ser

Re: [I] [Feature Request] Add Writer Support for Table-Compatible Parquet Files [iceberg-python]

2025-03-02 Thread via GitHub
Fokko commented on issue #1737: URL: https://github.com/apache/iceberg-python/issues/1737#issuecomment-2692858340 Thanks @andormarkus-alcd, for raising this and the comprehensive writeup. I think a lot of this would be solved if we implement https://github.com/apache/iceberg-python/issues/

Re: [I] Push upsert changes detection to Arrow [iceberg-python]

2025-03-02 Thread via GitHub
Fokko commented on issue #1679: URL: https://github.com/apache/iceberg-python/issues/1679#issuecomment-2692857608 @kevinjqliu Yes, let's close this ๐Ÿš€ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

Re: [I] Push upsert changes detection to Arrow [iceberg-python]

2025-03-02 Thread via GitHub
Fokko closed issue #1679: Push upsert changes detection to Arrow URL: https://github.com/apache/iceberg-python/issues/1679 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

Re: [PR] Fix thrift client connection for Kerberos Hive Client [iceberg-python]

2025-03-02 Thread via GitHub
kevinjqliu commented on code in PR #1747: URL: https://github.com/apache/iceberg-python/pull/1747#discussion_r1976692000 ## pyiceberg/catalog/hive.py: ## Review Comment: I wonder if we can just move `_init_thrift_client` out of the `__init__` and into the context manager's

Re: [I] Getting "offset overflow while concatenating arrays" Error when writing to iceberg [iceberg-python]

2025-03-02 Thread via GitHub
Fokko commented on issue #1733: URL: https://github.com/apache/iceberg-python/issues/1733#issuecomment-2692857093 Thanks @heman026 for raising this issue. That's an interesting one. Could you share the full stack-trace, or even better, a way to reproduce the error? -- This is an automate

Re: [I] Python defined schema does not match created schema [iceberg-python]

2025-03-02 Thread via GitHub
Fokko commented on issue #1728: URL: https://github.com/apache/iceberg-python/issues/1728#issuecomment-2692856479 @grbinho Thanks for raising this. The IDs get re-assigned, so users don't have to worry about it. You mentioned that inserting the data fails; can you explain your process of h

Re: [PR] Fix thrift client connection for Kerberos Hive Client [iceberg-python]

2025-03-02 Thread via GitHub
kevinjqliu commented on code in PR #1747: URL: https://github.com/apache/iceberg-python/pull/1747#discussion_r1976690830 ## pyiceberg/catalog/hive.py: ## @@ -167,6 +167,10 @@ def _init_thrift_client(self) -> None: self._client = Client(protocol) def __enter__(sel

Re: [I] [Feature] - Allow Schema Overwrite [iceberg-python]

2025-03-02 Thread via GitHub
Fokko commented on issue #1726: URL: https://github.com/apache/iceberg-python/issues/1726#issuecomment-2692854196 Hey @mariotaddeucci Can you share more about the context around this functionality? Also, based on what I see in the implementation in #1727 I'm reluctant adding this. I

Re: [I] Add nanosecond timestamp type [iceberg-python]

2025-03-02 Thread via GitHub
NoldAndreas commented on issue #1226: URL: https://github.com/apache/iceberg-python/issues/1226#issuecomment-2692802219 Thank you @sungwy !! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

Re: [PR] Build: Ignore README.md/LICENSE/NOTICE in all paths on CI [iceberg]

2025-03-02 Thread via GitHub
Fokko merged PR #12429: URL: https://github.com/apache/iceberg/pull/12429 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] Build: Bump mkdocs-material from 9.6.5 to 9.6.6 [iceberg]

2025-03-02 Thread via GitHub
Fokko merged PR #12432: URL: https://github.com/apache/iceberg/pull/12432 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [I] Support scanning a specific branch of a table [iceberg-python]

2025-03-02 Thread via GitHub
malcolmbovey commented on issue #1746: URL: https://github.com/apache/iceberg-python/issues/1746#issuecomment-2692723601 Thanks, would you like me to draft a PR tp enable a branch parameter to be passed to `table.scan` ? -- This is an automated message from the Apache Git Service. To re

[PR] chore(deps): Bump arrow-buffer from 54.2.0 to 54.2.1 [iceberg-rust]

2025-03-02 Thread via GitHub
dependabot[bot] opened a new pull request, #1028: URL: https://github.com/apache/iceberg-rust/pull/1028 Bumps [arrow-buffer](https://github.com/apache/arrow-rs) from 54.2.0 to 54.2.1. Release notes Sourced from https://github.com/apache/arrow-rs/releases";>arrow-buffer's releases.

[PR] chore(deps): Bump either from 1.13.0 to 1.14.0 [iceberg-rust]

2025-03-02 Thread via GitHub
dependabot[bot] opened a new pull request, #1027: URL: https://github.com/apache/iceberg-rust/pull/1027 Bumps [either](https://github.com/rayon-rs/either) from 1.13.0 to 1.14.0. Commits https://github.com/rayon-rs/either/commit/6e6dc268282480021cc7a678ec084404aacf895e";>6e6dc26

[PR] chore(deps): Bump aws-config from 1.5.16 to 1.5.17 [iceberg-rust]

2025-03-02 Thread via GitHub
dependabot[bot] opened a new pull request, #1026: URL: https://github.com/apache/iceberg-rust/pull/1026 Bumps [aws-config](https://github.com/smithy-lang/smithy-rs) from 1.5.16 to 1.5.17. Commits See full diff in https://github.com/smithy-lang/smithy-rs/commits";>compare view

[PR] chore(deps): Bump arrow-schema from 54.2.0 to 54.2.1 [iceberg-rust]

2025-03-02 Thread via GitHub
dependabot[bot] opened a new pull request, #1024: URL: https://github.com/apache/iceberg-rust/pull/1024 Bumps [arrow-schema](https://github.com/apache/arrow-rs) from 54.2.0 to 54.2.1. Release notes Sourced from https://github.com/apache/arrow-rs/releases";>arrow-schema's releases.

  1   2   >