Re: [PR] Add properties support for HadoopTables.load() (#12251) [iceberg]

2025-03-19 Thread via GitHub
qqchang2nd commented on PR #12296: URL: https://github.com/apache/iceberg/pull/12296#issuecomment-2739378376 This PR is ready for review, @rdblue Could you please review it? Happy to address any feedback. Thanks for your time! -- This is an automated message from the Apache Git Servic

Re: [I] Status Code for `NamespaceNotEmpty` exception? [iceberg]

2025-03-19 Thread via GitHub
nastra closed issue #12502: Status Code for `NamespaceNotEmpty` exception? URL: https://github.com/apache/iceberg/issues/12502 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

Re: [PR] OpenAPI: Handle NamespaceNotEmptyException when dropping a namespace [iceberg]

2025-03-19 Thread via GitHub
nastra commented on PR #12518: URL: https://github.com/apache/iceberg/pull/12518#issuecomment-2739376143 thanks everyone for voting & reviewing -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] doc: add MSRV and dependency policy doc [iceberg-rust]

2025-03-19 Thread via GitHub
Xuanwo commented on code in PR #1114: URL: https://github.com/apache/iceberg-rust/pull/1114#discussion_r2004914808 ## CONTRIBUTING.md: ## @@ -127,6 +114,13 @@ For podman, refer to [Using Podman instead of Docker](docs/contributing/podman.m * To run unit tests only: `make unit-

Re: [PR] Spark 3.5: Add query runner in test module [iceberg]

2025-03-19 Thread via GitHub
nastra commented on PR #11758: URL: https://github.com/apache/iceberg/pull/11758#issuecomment-2739363851 @ebyhr are you still planning to work on this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

Re: [PR] Spark: Detect dangling DVs properly [iceberg]

2025-03-19 Thread via GitHub
nastra merged PR #12270: URL: https://github.com/apache/iceberg/pull/12270 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] doc: add MSRV and dependency policy doc [iceberg-rust]

2025-03-19 Thread via GitHub
Xuanwo commented on code in PR #1114: URL: https://github.com/apache/iceberg-rust/pull/1114#discussion_r2004906315 ## CONTRIBUTING.md: ## @@ -26,12 +26,16 @@ First, thank you for contributing to Iceberg Rust! The goal of this document is - [Git Branches](#git-branches) -

Re: [PR] Add JSON single value encoding [iceberg-python]

2025-03-19 Thread via GitHub
Fokko merged PR #1805: URL: https://github.com/apache/iceberg-python/pull/1805 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [I] Handling Updates on Partition Columns in Iceberg with Flink CDC [iceberg]

2025-03-19 Thread via GitHub
HoangLT2603 commented on issue #11573: URL: https://github.com/apache/iceberg/issues/11573#issuecomment-2739200202 Hi @a8356555, Do you have any solution for this issue yet? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

Re: [PR] doc: add MSRV and dependency policy doc [iceberg-rust]

2025-03-19 Thread via GitHub
liurenjie1024 commented on code in PR #1114: URL: https://github.com/apache/iceberg-rust/pull/1114#discussion_r2004868578 ## CONTRIBUTING.md: ## @@ -26,12 +26,16 @@ First, thank you for contributing to Iceberg Rust! The goal of this document is - [Git Branches](#git-branches

Re: [PR] Make `schema` and `partition_spec` optional for TableMetadataV1 [iceberg-rust]

2025-03-19 Thread via GitHub
liurenjie1024 commented on code in PR #1087: URL: https://github.com/apache/iceberg-rust/pull/1087#discussion_r2004627669 ## crates/iceberg/src/spec/table_metadata.rs: ## @@ -942,9 +944,11 @@ pub(super) mod _serde { )) }) .o

Re: [I] Commit fails because S3 checksum does match when calling PutObject [iceberg]

2025-03-19 Thread via GitHub
kien-truong commented on issue #12577: URL: https://github.com/apache/iceberg/issues/12577#issuecomment-2739178676 I've encountered this before when using Iceberg in combination with an outdate version of `msk-config-providers` that use AWS Java SDK version 2.18.8. This version of AW

Re: [I] catalog table-default and table-override properties are not supported in CREATE_OR_REPLACE operation in IRC [iceberg]

2025-03-19 Thread via GitHub
nastra closed issue #12506: catalog table-default and table-override properties are not supported in CREATE_OR_REPLACE operation in IRC URL: https://github.com/apache/iceberg/issues/12506 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] Spark: Use correct statistics file in SparkScan::estimateStatistics(Snapshot) [iceberg]

2025-03-19 Thread via GitHub
wypoon commented on PR #12482: URL: https://github.com/apache/iceberg/pull/12482#issuecomment-2739168706 @findepi can you please review this simple fix? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [I] [bug]OversizedAllocationException when query data with Spark [iceberg]

2025-03-19 Thread via GitHub
tammyxiong commented on issue #9820: URL: https://github.com/apache/iceberg/issues/9820#issuecomment-2738128729 also a quick note: the property setting is for the source table, not the target table -- This is an automated message from the Apache Git Service. To respond to the message, ple

Re: [PR] Flink: Add support for Flink 2.0 [iceberg]

2025-03-19 Thread via GitHub
stevenzwu commented on code in PR #12527: URL: https://github.com/apache/iceberg/pull/12527#discussion_r2003995287 ## gradle.properties: ## @@ -16,8 +16,8 @@ jmhOutputPath=build/reports/jmh/human-readable-output.txt jmhJsonOutputPath=build/reports/jmh/results.json jmhIncludeR

Re: [PR] doc: add MSRV and dependency policy doc [iceberg-rust]

2025-03-19 Thread via GitHub
xxchan commented on code in PR #1114: URL: https://github.com/apache/iceberg-rust/pull/1114#discussion_r2004812921 ## CONTRIBUTING.md: ## @@ -26,12 +26,16 @@ First, thank you for contributing to Iceberg Rust! The goal of this document is - [Git Branches](#git-branches) -

Re: [PR] feat: re-export name mapping [iceberg-rust]

2025-03-19 Thread via GitHub
liurenjie1024 commented on code in PR #1116: URL: https://github.com/apache/iceberg-rust/pull/1116#discussion_r2004663889 ## crates/iceberg/src/spec/name_mapping.rs: ## @@ -20,10 +20,14 @@ use serde::{Deserialize, Serialize}; use serde_with::{serde_as, DefaultOnNull}; +/// P

Re: [PR] Flink: Add support for Flink 2.0 [iceberg]

2025-03-19 Thread via GitHub
stevenzwu commented on code in PR #12527: URL: https://github.com/apache/iceberg/pull/12527#discussion_r2003995287 ## gradle.properties: ## @@ -16,8 +16,8 @@ jmhOutputPath=build/reports/jmh/human-readable-output.txt jmhJsonOutputPath=build/reports/jmh/results.json jmhIncludeR

Re: [PR] Flink: Add support for Flink 2.0 [iceberg]

2025-03-19 Thread via GitHub
stevenzwu commented on code in PR #12527: URL: https://github.com/apache/iceberg/pull/12527#discussion_r2004787419 ## gradle.properties: ## @@ -16,8 +16,8 @@ jmhOutputPath=build/reports/jmh/human-readable-output.txt jmhJsonOutputPath=build/reports/jmh/results.json jmhIncludeR

Re: [PR] Add Arrow C Data Interface and nanoarrow dependency [iceberg-cpp]

2025-03-19 Thread via GitHub
wgtmac commented on PR #44: URL: https://github.com/apache/iceberg-cpp/pull/44#issuecomment-2738966373 @Xuanwo @Fokko Could you help review this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

Re: [PR] chore(ci): use generate-notes and fix svn [iceberg-go]

2025-03-19 Thread via GitHub
zeroshade commented on code in PR #340: URL: https://github.com/apache/iceberg-go/pull/340#discussion_r2003856162 ## .github/workflows/rc.yml: ## @@ -134,6 +134,7 @@ jobs: gh release create ${GITHUB_REF_NAME} \ --prerelease \ --title "Apache

Re: [PR] Core, Parquet, ORC: Fix missing data when writing unknown [iceberg]

2025-03-19 Thread via GitHub
rdblue commented on code in PR #12581: URL: https://github.com/apache/iceberg/pull/12581#discussion_r2004250123 ## parquet/src/main/java/org/apache/iceberg/data/parquet/InternalWriter.java: ## @@ -40,11 +40,15 @@ public class InternalWriter extends BaseParquetWriter { priv

Re: [I] to_pandas(), to_arrow() fail because case_sensitive doesn't work if column in row_filter doesn't match the case even if case_sensitive is set to False in scan [iceberg-python]

2025-03-19 Thread via GitHub
github-actions[bot] commented on issue #1177: URL: https://github.com/apache/iceberg-python/issues/1177#issuecomment-2738604539 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity

Re: [PR] Add Arrow C Data Interface and nanoarrow dependency [iceberg-cpp]

2025-03-19 Thread via GitHub
zhjwpku commented on PR #44: URL: https://github.com/apache/iceberg-cpp/pull/44#issuecomment-2738871141 Agreed, nanoarrow sounds good to me. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

Re: [PR] Scan Delete Support Part 3: `ArrowReader::build_deletes_row_selection` implementation [iceberg-rust]

2025-03-19 Thread via GitHub
liurenjie1024 commented on PR #951: URL: https://github.com/apache/iceberg-rust/pull/951#issuecomment-2738957614 cc @sdd Would you help to resolve conflicts first? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

Re: [I] Introduce TimestampNs, TimestampTzNs support for Bucket transforms [iceberg-rust]

2025-03-19 Thread via GitHub
liurenjie1024 commented on issue #1110: URL: https://github.com/apache/iceberg-rust/issues/1110#issuecomment-2738885215 Thanks @sungwy for raising this. I think this should be a feature rather a bug -- This is an automated message from the Apache Git Service. To respond to the message, p

Re: [PR] doc: add MSRV and dependency policy doc [iceberg-rust]

2025-03-19 Thread via GitHub
liurenjie1024 commented on code in PR #1114: URL: https://github.com/apache/iceberg-rust/pull/1114#discussion_r2004655102 ## Cargo.toml: ## @@ -35,6 +35,7 @@ homepage = "https://rust.iceberg.apache.org/"; repository = "https://github.com/apache/iceberg-rust"; license = "Apac

Re: [PR] Scan Delete Support Part 2: introduce `DeleteFileManager` skeleton. Use in `ArrowReader` [iceberg-rust]

2025-03-19 Thread via GitHub
liurenjie1024 merged PR #950: URL: https://github.com/apache/iceberg-rust/pull/950 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [PR] Core, Parquet, ORC: Fix missing data when writing unknown [iceberg]

2025-03-19 Thread via GitHub
rdblue commented on code in PR #12581: URL: https://github.com/apache/iceberg/pull/12581#discussion_r2004256920 ## parquet/src/main/java/org/apache/iceberg/parquet/ParquetValueWriters.java: ## @@ -648,7 +656,7 @@ protected StructWriter(List> writers) { @Override publi

Re: [PR] API, Core: Add geometry and geography types support [iceberg]

2025-03-19 Thread via GitHub
Kontinuation commented on code in PR #12346: URL: https://github.com/apache/iceberg/pull/12346#discussion_r2004547525 ## api/src/main/java/org/apache/iceberg/types/Types.java: ## @@ -543,6 +561,120 @@ public int hashCode() { } } + public static class GeometryType exte

Re: [PR] API, Core: Add geometry and geography types support [iceberg]

2025-03-19 Thread via GitHub
Kontinuation commented on code in PR #12346: URL: https://github.com/apache/iceberg/pull/12346#discussion_r2004537931 ## api/src/main/java/org/apache/iceberg/types/Types.java: ## @@ -543,6 +565,134 @@ public int hashCode() { } } + public static class GeometryType exte

Re: [PR] API, Core: Add geometry and geography types support [iceberg]

2025-03-19 Thread via GitHub
Kontinuation commented on code in PR #12346: URL: https://github.com/apache/iceberg/pull/12346#discussion_r2004535984 ## api/src/main/java/org/apache/iceberg/types/Types.java: ## @@ -543,6 +565,134 @@ public int hashCode() { } } + public static class GeometryType exte

Re: [PR] feat(puffin): Add PuffinWriter [iceberg-rust]

2025-03-19 Thread via GitHub
fqaiser94 commented on PR #959: URL: https://github.com/apache/iceberg-rust/pull/959#issuecomment-2738601920 > @fqaiser94 Whats the current idea for this? I can offer help if needed. Forgot about this, let me try to revive this over the weekend -- This is an automated message from t

Re: [PR] Add properties support for HadoopTables.load() (#12251) [iceberg]

2025-03-19 Thread via GitHub
github-actions[bot] commented on PR #12296: URL: https://github.com/apache/iceberg/pull/12296#issuecomment-2738596521 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] Replace usages of Aws4Signer with AwsV4HttpSigner in REST SigV4 [iceberg]

2025-03-19 Thread via GitHub
github-actions[bot] commented on PR #12295: URL: https://github.com/apache/iceberg/pull/12295#issuecomment-2738596451 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] Core: Fix failure when reading files table with branch [iceberg]

2025-03-19 Thread via GitHub
github-actions[bot] commented on PR #11719: URL: https://github.com/apache/iceberg/pull/11719#issuecomment-2738595970 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [I] Javadoc issues [iceberg]

2025-03-19 Thread via GitHub
manuzhang closed issue #10378: Javadoc issues URL: https://github.com/apache/iceberg/issues/10378 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues

Re: [I] Javadoc issues [iceberg]

2025-03-19 Thread via GitHub
manuzhang commented on issue #10378: URL: https://github.com/apache/iceberg/issues/10378#issuecomment-2738555454 This can be closed now -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

Re: [PR] REST: HTTPRequest.baseUri() should be nullable [iceberg]

2025-03-19 Thread via GitHub
adutra commented on PR #12556: URL: https://github.com/apache/iceberg/pull/12556#issuecomment-2737282769 @nastra should this be added to 1.9.0 milestone? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] Scan Delete Support Part 2: introduce `DeleteFileManager` skeleton. Use in `ArrowReader` [iceberg-rust]

2025-03-19 Thread via GitHub
sdd commented on code in PR #950: URL: https://github.com/apache/iceberg-rust/pull/950#discussion_r2004183085 ## crates/iceberg/src/arrow/delete_file_manager.rs: ## @@ -0,0 +1,64 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license ag

Re: [I] Implement DeleteFiles operation [iceberg-go]

2025-03-19 Thread via GitHub
zeroshade commented on issue #348: URL: https://github.com/apache/iceberg-go/issues/348#issuecomment-2738339818 > Would you rather have a DeleteFiles method, or rather a MergeFiles method that does both add and delete. Looking a the current code, I'm under the impression that going the D

Re: [PR] Add Arrow C Data Interface and dependencies of nanoarrow & sparrow [iceberg-cpp]

2025-03-19 Thread via GitHub
lidavidm commented on PR #44: URL: https://github.com/apache/iceberg-cpp/pull/44#issuecomment-2738331312 Sounds good to me -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

Re: [PR] Parquet: Implement Variant metrics [iceberg]

2025-03-19 Thread via GitHub
rdblue commented on code in PR #12496: URL: https://github.com/apache/iceberg/pull/12496#discussion_r2004402908 ## parquet/src/main/java/org/apache/iceberg/parquet/ParquetVariantUtil.java: ## @@ -0,0 +1,476 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one +

Re: [PR] Parquet: Implement Variant metrics [iceberg]

2025-03-19 Thread via GitHub
rdblue commented on code in PR #12496: URL: https://github.com/apache/iceberg/pull/12496#discussion_r2004398000 ## core/src/main/java/org/apache/iceberg/variants/Variants.java: ## @@ -33,6 +37,66 @@ public static VariantMetadata metadata(ByteBuffer metadata) { return Serial

Re: [I] Implement DeleteFiles operation [iceberg-go]

2025-03-19 Thread via GitHub
arnaudbriche commented on issue #348: URL: https://github.com/apache/iceberg-go/issues/348#issuecomment-2738255768 Sorry to spam. But I see a `deletedFiles` field in snapshot producer, but it's not filling anywhere. What's the idea behind it ? -- This is an automated message from the A

Re: [PR] Docs: Site updates for 1.7.2 [iceberg]

2025-03-19 Thread via GitHub
jbonofre commented on code in PR #12576: URL: https://github.com/apache/iceberg/pull/12576#discussion_r2003558737 ## site/docs/releases.md: ## @@ -88,6 +88,20 @@ The 1.8.1 release contains bug fixes and fixes to LICENSE/NOTICE files. For full ## Past releases +### 1.7.2 re

Re: [I] Implement DeleteFiles operation [iceberg-go]

2025-03-19 Thread via GitHub
arnaudbriche commented on issue #348: URL: https://github.com/apache/iceberg-go/issues/348#issuecomment-2738252432 Regarding current API, I think it's overall great! One thing the would be useful, at least to me, would be to expose the function to get Iceberg schema from a Parque file

Re: [PR] Core, Parquet, ORC: Fix missing data when writing unknown [iceberg]

2025-03-19 Thread via GitHub
danielcweeks commented on PR #12581: URL: https://github.com/apache/iceberg/pull/12581#issuecomment-2738144462 You might want to run `gradle revapi` because I think there are a few issues. -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[I] Implement DeleteFiles operation [iceberg-go]

2025-03-19 Thread via GitHub
arnaudbriche opened a new issue, #348: URL: https://github.com/apache/iceberg-go/issues/348 ### Feature Request / Improvement Hi, I see you recently added support for Transaction and AddFiles and this is awesome. I'm building that uses this package to allow append/merges to

[PR] fix: Clickhouse does not support "null" as partition spec metadata [iceberg-go]

2025-03-19 Thread via GitHub
arnaudbriche opened a new pull request, #347: URL: https://github.com/apache/iceberg-go/pull/347 Hi, I'm working on [a tool that leverage this package](https://github.com/agnosticeng/icepq) to support adding / removing / merging Parquet files from ClickHouse with an UDF. Aft

Re: [PR] Scan Delete Support Part 2: introduce `DeleteFileManager` skeleton. Use in `ArrowReader` [iceberg-rust]

2025-03-19 Thread via GitHub
sdd commented on PR #950: URL: https://github.com/apache/iceberg-rust/pull/950#issuecomment-2737948289 Back to you @liurenjie1024 - only small changes vs when you last looked so it should be pretty quick to re-review. -- This is an automated message from the Apache Git Service. To respond

[I] Tests generate funky paths [iceberg-go]

2025-03-19 Thread via GitHub
Fokko opened a new issue, #346: URL: https://github.com/apache/iceberg-go/issues/346 ### Apache Iceberg version None ### Please describe the bug 🐞 Not sure if it is just a test-only thing. From `table_test.go` I changed the temp directory to `/tmp/v`, but it seems to

Re: [PR] Scan Delete Support Part 2: introduce `DeleteFileManager` skeleton. Use in `ArrowReader` [iceberg-rust]

2025-03-19 Thread via GitHub
sdd commented on code in PR #950: URL: https://github.com/apache/iceberg-rust/pull/950#discussion_r2004184868 ## crates/iceberg/src/arrow/delete_file_manager.rs: ## @@ -0,0 +1,64 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license ag

Re: [PR] Scan Delete Support Part 2: introduce `DeleteFileManager` skeleton. Use in `ArrowReader` [iceberg-rust]

2025-03-19 Thread via GitHub
sdd commented on code in PR #950: URL: https://github.com/apache/iceberg-rust/pull/950#discussion_r2004178851 ## crates/iceberg/src/arrow/delete_file_manager.rs: ## @@ -0,0 +1,64 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license ag

Re: [PR] Core: HadoopFileIO to support bulk delete through the Hadoop Filesystem APIs [iceberg]

2025-03-19 Thread via GitHub
steveloughran commented on code in PR #10233: URL: https://github.com/apache/iceberg/pull/10233#discussion_r2004175273 ## core/src/main/java/org/apache/iceberg/hadoop/HadoopFileIO.java: ## @@ -173,26 +203,223 @@ public void deletePrefix(String prefix) { } } + /** +

Re: [PR] Flink: Support source watermark for flink sql windows [iceberg]

2025-03-19 Thread via GitHub
swapna267 commented on PR #12191: URL: https://github.com/apache/iceberg/pull/12191#issuecomment-2737924089 Thanks all. Will submit backport PR soon. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] Spec: update to reflect lineage is required [iceberg]

2025-03-19 Thread via GitHub
RussellSpitzer commented on code in PR #12580: URL: https://github.com/apache/iceberg/pull/12580#discussion_r2004159738 ## format/spec.md: ## @@ -367,37 +367,35 @@ Iceberg tables must not use field ids greater than 2147483447 (`Integer.MAX_VALU The set of metadata columns is

Re: [PR] Spec: update to reflect lineage is required [iceberg]

2025-03-19 Thread via GitHub
RussellSpitzer commented on code in PR #12580: URL: https://github.com/apache/iceberg/pull/12580#discussion_r2004160887 ## format/spec.md: ## @@ -367,37 +367,35 @@ Iceberg tables must not use field ids greater than 2147483447 (`Integer.MAX_VALU The set of metadata columns is

[I] Inconsistencies in Iceberg metadata serialization [iceberg-go]

2025-03-19 Thread via GitHub
Fokko opened a new issue, #345: URL: https://github.com/apache/iceberg-go/issues/345 ### Apache Iceberg version None ### Please describe the bug 🐞 This is a minor thing, but I think it would be nice to get fixed anyway. While generating some V1 and V2 metadata, I n

[I] `partition-spec` is invalid while producing V1 metadata [iceberg-go]

2025-03-19 Thread via GitHub
Fokko opened a new issue, #344: URL: https://github.com/apache/iceberg-go/issues/344 ### Apache Iceberg version None ### Please describe the bug 🐞 The `partition-spec` is null, and should be equal to the spec in `partition-specs` that is being pointed at by `current-part

Re: [PR] Flink: Support source watermark for flink sql windows [iceberg]

2025-03-19 Thread via GitHub
pvary commented on PR #12191: URL: https://github.com/apache/iceberg/pull/12191#issuecomment-2737817710 Merged to main Thanks for the PR @swapna267! Thanks for the review @mxm and @stevenzwu! -- This is an automated message from the Apache Git Service. To respond to the message, plea

Re: [PR] Flink: Support source watermark for flink sql windows [iceberg]

2025-03-19 Thread via GitHub
pvary merged PR #12191: URL: https://github.com/apache/iceberg/pull/12191 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

[I] Strip the trailing slash of the warehouse path [iceberg-go]

2025-03-19 Thread via GitHub
Fokko opened a new issue, #343: URL: https://github.com/apache/iceberg-go/issues/343 ### Apache Iceberg version None ### Please describe the bug 🐞 When a table doesn't explicitly pass in a location, the location is determined based on the warehouse path. When it h

Re: [I] Issue with IsIn Predicate Formatting in Iceberg-Go [iceberg-go]

2025-03-19 Thread via GitHub
zeroshade commented on issue #335: URL: https://github.com/apache/iceberg-go/issues/335#issuecomment-2737760711 Once http://github.com/apache/arrow-go/pull/319 gets merged, I'll be able to update iceberg-go to properly utilize the `is_in` predicate. So far it works in local testing -- Th

Re: [PR] Replace `tabulario` in favor of `apache` [iceberg-go]

2025-03-19 Thread via GitHub
zeroshade merged PR #341: URL: https://github.com/apache/iceberg-go/pull/341 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

[PR] Supress version warning [iceberg-go]

2025-03-19 Thread via GitHub
Fokko opened a new pull request, #342: URL: https://github.com/apache/iceberg-go/pull/342 I'm getting this error for quite a while, I think we can remove the version: ``` WARN[] /Users/fokko.driesprong/work/iceberg-go/dev/docker-compose.yml: the attribute `version` is obsolete,

Re: [PR] chore: update denpendencies, fix ci [iceberg-rust]

2025-03-19 Thread via GitHub
Lordworms commented on code in PR #1106: URL: https://github.com/apache/iceberg-rust/pull/1106#discussion_r2004055579 ## Cargo.toml: ## @@ -94,12 +94,12 @@ serde_json = "1.0.138" serde_repr = "0.1.16" serde_with = "3.4" tempfile = "3.18" -thrift = "0.17.0" -tokio = { versio

Re: [PR] feat(iceberg): introduce remove schemas [iceberg-rust]

2025-03-19 Thread via GitHub
jonathanc-n commented on code in PR #1115: URL: https://github.com/apache/iceberg-rust/pull/1115#discussion_r2004025994 ## crates/iceberg/src/catalog/mod.rs: ## @@ -482,6 +482,12 @@ pub enum TableUpdate { /// Snapshot id to remove partition statistics for. snap

Re: [PR] ORC: Support timestamp(9), variant, and unknown in generics [iceberg]

2025-03-19 Thread via GitHub
rdblue merged PR #12567: URL: https://github.com/apache/iceberg/pull/12567 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] ORC: Support timestamp(9), variant, and unknown in generics [iceberg]

2025-03-19 Thread via GitHub
rdblue commented on PR #12567: URL: https://github.com/apache/iceberg/pull/12567#issuecomment-2737677144 Thanks for reviewing, @pvary! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [I] [feature] Investigate integrations leveraging the PyCapsule protocol [iceberg-python]

2025-03-19 Thread via GitHub
WillAyd commented on issue #1655: URL: https://github.com/apache/iceberg-python/issues/1655#issuecomment-2737663703 I think this would be a great addition to the library and open up support for integration with a large variety of dataframe tools. Given PyIceberg already has a `.to_ar

Re: [PR] Flink: Add support for Flink 2.0 [iceberg]

2025-03-19 Thread via GitHub
stevenzwu commented on code in PR #12527: URL: https://github.com/apache/iceberg/pull/12527#discussion_r2003993029 ## gradle.properties: ## @@ -16,8 +16,8 @@ jmhOutputPath=build/reports/jmh/human-readable-output.txt jmhJsonOutputPath=build/reports/jmh/results.json jmhIncludeR

Re: [PR] chore: update denpendencies, fix ci [iceberg-rust]

2025-03-19 Thread via GitHub
Lordworms commented on code in PR #1106: URL: https://github.com/apache/iceberg-rust/pull/1106#discussion_r2004010414 ## rust-toolchain.toml: ## @@ -16,5 +16,5 @@ # under the License. [toolchain] -channel = "nightly-2024-06-10" +channel = "nightly-2024-10-17" Review Comment

Re: [PR] Flink: Add support for Flink 2.0 [iceberg]

2025-03-19 Thread via GitHub
stevenzwu commented on code in PR #12527: URL: https://github.com/apache/iceberg/pull/12527#discussion_r2003995287 ## gradle.properties: ## @@ -16,8 +16,8 @@ jmhOutputPath=build/reports/jmh/human-readable-output.txt jmhJsonOutputPath=build/reports/jmh/results.json jmhIncludeR

Re: [PR] Flink: Add support for Flink 2.0 [iceberg]

2025-03-19 Thread via GitHub
stevenzwu commented on code in PR #12527: URL: https://github.com/apache/iceberg/pull/12527#discussion_r2003991315 ## gradle/libs.versions.toml: ## @@ -45,6 +45,7 @@ findbugs-jsr305 = "3.0.2" flink118 = { strictly = "1.18.1"} flink119 = { strictly = "1.19.1"} flink120 = { s

Re: [PR] Flink: Add support for Flink 2.0 [iceberg]

2025-03-19 Thread via GitHub
stevenzwu commented on code in PR #12527: URL: https://github.com/apache/iceberg/pull/12527#discussion_r2003989781 ## flink/v2.0/flink/src/test/java/org/apache/iceberg/flink/source/TestStreamingReaderOperator.java: ## @@ -275,7 +274,6 @@ private OneInputStreamOperatorTestHarness

Re: [PR] Flink: Add support for Flink 2.0 [iceberg]

2025-03-19 Thread via GitHub
stevenzwu commented on code in PR #12527: URL: https://github.com/apache/iceberg/pull/12527#discussion_r2003985770 ## flink/v2.0/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/StatisticsOrRecordTypeInformation.java: ## @@ -79,11 +78,6 @@ public TypeSerializer createS

Re: [PR] Flink: Add support for Flink 2.0 [iceberg]

2025-03-19 Thread via GitHub
stevenzwu commented on code in PR #12527: URL: https://github.com/apache/iceberg/pull/12527#discussion_r2002394034 ## flink/v2.0/flink/src/main/java/org/apache/iceberg/flink/sink/IcebergFilesCommitter.java: ## @@ -138,6 +141,12 @@ class IcebergFilesCommitter extends AbstractStr

Re: [PR] Flink: Add support for Flink 2.0 [iceberg]

2025-03-19 Thread via GitHub
stevenzwu commented on code in PR #12527: URL: https://github.com/apache/iceberg/pull/12527#discussion_r2003982416 ## flink/v2.0/flink/src/main/java/org/apache/iceberg/flink/sink/IcebergStreamWriter.java: ## @@ -46,13 +45,12 @@ class IcebergStreamWriter extends AbstractStreamOp

Re: [PR] Spark 3.4: Backport Spark actions changes in Spark rewrite_table_path procedure (#12006 #12172 #11929) [iceberg]

2025-03-19 Thread via GitHub
szehon-ho commented on PR #12568: URL: https://github.com/apache/iceberg/pull/12568#issuecomment-2737590567 should we also backport #12282? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

Re: [PR] chore(ci): use generate-notes and fix svn [iceberg-go]

2025-03-19 Thread via GitHub
zeroshade merged PR #340: URL: https://github.com/apache/iceberg-go/pull/340 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] Migrate Spark 3.4 TestBase-related remaining tests in actions [iceberg]

2025-03-19 Thread via GitHub
tomtongue commented on PR #12579: URL: https://github.com/apache/iceberg/pull/12579#issuecomment-2737577855 @nastra Could you review this PR when you have time? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

Re: [I] Error reading table after appending pyarrow table [iceberg-python]

2025-03-19 Thread via GitHub
p1c2u commented on issue #1798: URL: https://github.com/apache/iceberg-python/issues/1798#issuecomment-2737554034 @kevinjqliu after investigation I found out it happens after I append pyarrow table without list field specified in schema as optional. Example table with schema

Re: [PR] API, Core: Add geometry and geography types support [iceberg]

2025-03-19 Thread via GitHub
szehon-ho commented on code in PR #12346: URL: https://github.com/apache/iceberg/pull/12346#discussion_r2003929143 ## api/src/main/java/org/apache/iceberg/types/Types.java: ## @@ -543,6 +565,134 @@ public int hashCode() { } } + public static class GeometryType extends

Re: [PR] chore(ci): use generate-notes and fix svn [iceberg-go]

2025-03-19 Thread via GitHub
kevinjqliu commented on code in PR #340: URL: https://github.com/apache/iceberg-go/pull/340#discussion_r2003909901 ## dev/release/release_rc.sh: ## @@ -89,40 +89,38 @@ if [ "${RELEASE_SIGN}" -gt 0 ]; then gh run watch --repo "${repository}" --exit-status "${run_id}" # Cr

Re: [PR] Docs: Site updates for 1.7.2 [iceberg]

2025-03-19 Thread via GitHub
nastra merged PR #12576: URL: https://github.com/apache/iceberg/pull/12576 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] chore(ci): use generate-notes and fix svn [iceberg-go]

2025-03-19 Thread via GitHub
zeroshade commented on code in PR #340: URL: https://github.com/apache/iceberg-go/pull/340#discussion_r2003857562 ## dev/release/release_rc.sh: ## @@ -89,7 +89,6 @@ if [ "${RELEASE_SIGN}" -gt 0 ]; then gh run watch --repo "${repository}" --exit-status "${run_id}" # Creat

Re: [I] Pyiceberg leaks memory on table write [iceberg-python]

2025-03-19 Thread via GitHub
gabor-one closed issue #1809: Pyiceberg leaks memory on table write URL: https://github.com/apache/iceberg-python/issues/1809 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [I] Pyiceberg leaks memory on table write [iceberg-python]

2025-03-19 Thread via GitHub
gabor-one commented on issue #1809: URL: https://github.com/apache/iceberg-python/issues/1809#issuecomment-2737434606 I've traced back the issue to PyArrow. Sorry, my mistake. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

Re: [PR] Core: JDBCCatalog's dropView() should purge metadata files if GC is enabled [iceberg]

2025-03-19 Thread via GitHub
hsiang-c commented on code in PR #12511: URL: https://github.com/apache/iceberg/pull/12511#discussion_r2003819845 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcCatalog.java: ## @@ -612,6 +613,14 @@ public boolean dropView(TableIdentifier identifier) { throw new Unsupp

Re: [PR] V3: Introduce `timestamp_ns` and `timestamptz_ns` [iceberg-python]

2025-03-19 Thread via GitHub
kevinjqliu commented on code in PR #1632: URL: https://github.com/apache/iceberg-python/pull/1632#discussion_r2003800525 ## pyiceberg/types.py: ## @@ -62,6 +63,12 @@ FIXED_PARSER = ParseNumberFromBrackets(FIXED) +class TableVersion(IntEnum): +ONE = 1 +TWO = 2 +T

Re: [PR] V3: Introduce `timestamp_ns` and `timestamptz_ns` [iceberg-python]

2025-03-19 Thread via GitHub
kevinjqliu commented on code in PR #1632: URL: https://github.com/apache/iceberg-python/pull/1632#discussion_r2003796102 ## pyiceberg/types.py: ## @@ -62,6 +63,12 @@ FIXED_PARSER = ParseNumberFromBrackets(FIXED) +class TableVersion(IntEnum): +ONE = 1 +TWO = 2 +T

Re: [I] Rust doc is broken. [iceberg-rust]

2025-03-19 Thread via GitHub
kevinjqliu commented on issue #1112: URL: https://github.com/apache/iceberg-rust/issues/1112#issuecomment-273713 Yea i couldn't find any ways to do this using mdbook -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] REST: HTTPRequest.baseUri() should be nullable [iceberg]

2025-03-19 Thread via GitHub
nastra commented on PR #12556: URL: https://github.com/apache/iceberg/pull/12556#issuecomment-2737322494 no need to add it, since it's already merged and the 1.9.0 RC will take whatever is on `main` when the RC is created -- This is an automated message from the Apache Git Service. To res

Re: [PR] Upsert: Don't produce empty snapshots [iceberg-python]

2025-03-19 Thread via GitHub
Fokko merged PR #1810: URL: https://github.com/apache/iceberg-python/pull/1810 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [PR] feat: check whether table ops conflict when committing [iceberg-python]

2025-03-19 Thread via GitHub
YinZheng-Sun commented on code in PR #1816: URL: https://github.com/apache/iceberg-python/pull/1816#discussion_r2003271142 ## pyiceberg/table/update/snapshot.py: ## @@ -239,7 +240,21 @@ def _summary(self, snapshot_properties: Dict[str, str] = EMPTY_DICT) -> Summary:

Re: [PR] chore(ci): use generate-notes and fix svn [iceberg-go]

2025-03-19 Thread via GitHub
kevinjqliu commented on code in PR #340: URL: https://github.com/apache/iceberg-go/pull/340#discussion_r2003750801 ## dev/release/release_rc.sh: ## @@ -89,7 +89,6 @@ if [ "${RELEASE_SIGN}" -gt 0 ]; then gh run watch --repo "${repository}" --exit-status "${run_id}" # Crea

Re: [PR] Core: Use InternalData when reading manifests in FileCleanupStrategy [iceberg]

2025-03-19 Thread via GitHub
amogh-jahagirdar merged PR #12575: URL: https://github.com/apache/iceberg/pull/12575 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@

Re: [PR] docs: update go impl status [iceberg]

2025-03-19 Thread via GitHub
nastra merged PR #12578: URL: https://github.com/apache/iceberg/pull/12578 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

  1   2   >