Re: [PR] AWS: fix GlueCatalog name validation [iceberg]

2025-03-23 Thread via GitHub
wombatu-kun commented on PR #12367: URL: https://github.com/apache/iceberg/pull/12367#issuecomment-2746681717 not a bug? or just no one wants to review? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] backport #11301(rowconverter) to Flink 1.19 and 1.18 [iceberg]

2025-03-23 Thread via GitHub
pvary commented on PR #11826: URL: https://github.com/apache/iceberg/pull/11826#issuecomment-2747017155 @abharath9: Do we need this backport? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

Re: [I] Table corruption using lock-free Hive commits [iceberg]

2025-03-23 Thread via GitHub
lirui-apache commented on issue #11814: URL: https://github.com/apache/iceberg/issues/11814#issuecomment-2746961218 Is anyone working on this issue? If not, I'd like to submit a PR for it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [I] Support RowDeltaAction [iceberg-rust]

2025-03-23 Thread via GitHub
jonathanc-n commented on issue #1104: URL: https://github.com/apache/iceberg-rust/issues/1104#issuecomment-2746763897 One thing I would like to implement before the filter manager are the residual predicates mentioned by @Fokko [here](https://github.com/apache/iceberg-rust/pull/377#discussi

Re: [I] AWS: Glue ETL Job fails to create a table using lakeformation [iceberg]

2025-03-23 Thread via GitHub
xiaoxuandev commented on issue #11126: URL: https://github.com/apache/iceberg/issues/11126#issuecomment-2746922015 It seems like a setup issue. Are the glue.id and glue.account-id the same? -- This is an automated message from the Apache Git Service. To respond to the message, please log o

Re: [I] [DISCUSS] We have some C++ code to share for Iceberg read/write [iceberg-cpp]

2025-03-23 Thread via GitHub
jovezhong commented on issue #46: URL: https://github.com/apache/iceberg-cpp/issues/46#issuecomment-2746753342 Sure, loop in @zliang-min who works on the actual implementation from our team. We found a lot of tricky issues in Glue/Athena/Spark/S3Table. For example, some data written by Time

Re: [I] Subfolder with no name under /data folder [iceberg]

2025-03-23 Thread via GitHub
xiaoxuandev commented on issue #12065: URL: https://github.com/apache/iceberg/issues/12065#issuecomment-2746905992 Could you please provide the setup and steps to reproduce the issue? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to G

Re: [I] [DISCUSS] We have some C++ code to share for Iceberg read/write [iceberg-cpp]

2025-03-23 Thread via GitHub
wgtmac commented on issue #46: URL: https://github.com/apache/iceberg-cpp/issues/46#issuecomment-2746785851 I agree with @lidavidm. We need to finalize core APIs (especially on how to integrate with different engines and I/O approaches) before concrete implementations. -- This is an auto

Re: [PR] fix: do not write ICEBERG_INCLUDES everywhere [iceberg-cpp]

2025-03-23 Thread via GitHub
wgtmac commented on PR #49: URL: https://github.com/apache/iceberg-cpp/pull/49#issuecomment-2746714269 Thank you, @lidavidm and @Xuanwo! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

Re: [I] Investigation about tracing, logging, and metrics support. [iceberg-rust]

2025-03-23 Thread via GitHub
Xuanwo commented on issue #482: URL: https://github.com/apache/iceberg-rust/issues/482#issuecomment-2746862981 There is a post for why I prefer `fastrace` + `log`: https://fast.github.io/blog/fastrace-a-modern-approach-to-distributed-tracing-in-rust/ -- This is an automated message from t

Re: [PR] chore(catalog/rest): Add response headers in error for debug [iceberg-rust]

2025-03-23 Thread via GitHub
Xuanwo merged PR #1129: URL: https://github.com/apache/iceberg-rust/pull/1129 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

[I] HDFS support in iceberg-rust [iceberg-rust]

2025-03-23 Thread via GitHub
Xuanwo opened a new issue, #1130: URL: https://github.com/apache/iceberg-rust/issues/1130 There are several ways to access HDFS in remote areas: - libhdfs + JNI: needs java runtime and libhdfs - WebHDFS: http based, may have performance issues - [HDFS Native](https://github.com/K

Re: [I] [DISCUSS] We have some C++ code to share for Iceberg read/write [iceberg-cpp]

2025-03-23 Thread via GitHub
lidavidm commented on issue #46: URL: https://github.com/apache/iceberg-cpp/issues/46#issuecomment-2746771163 Also @wgtmac any thoughts? It will probably be some time before we're prepared to tackle this though (iceberg-cpp is still sketching out the core APIs) -- This is an automated me

Re: [PR] V3: Introduce `timestamp_ns` and `timestamptz_ns` [iceberg-python]

2025-03-23 Thread via GitHub
Fokko merged PR #1632: URL: https://github.com/apache/iceberg-python/pull/1632 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [PR] chore(deps): Bump typed-builder from 0.20.0 to 0.20.1 [iceberg-rust]

2025-03-23 Thread via GitHub
Xuanwo merged PR #1125: URL: https://github.com/apache/iceberg-rust/pull/1125 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [I] Support RowDeltaAction [iceberg-rust]

2025-03-23 Thread via GitHub
jonathanc-n commented on issue #1104: URL: https://github.com/apache/iceberg-rust/issues/1104#issuecomment-2746763712 @ZENOTME Sorry for the delay, i was a little busy with family. In the example you showed in the python implementation, the filtering for `OverwriteFiles` and `DeleteFiles` s

Re: [PR] feat: add file_io and local fs impl [iceberg-cpp]

2025-03-23 Thread via GitHub
lidavidm commented on code in PR #30: URL: https://github.com/apache/iceberg-cpp/pull/30#discussion_r2009380801 ## src/iceberg/file_io.h: ## @@ -0,0 +1,112 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the

Re: [I] [DISCUSS] We have some C++ code to share for Iceberg read/write [iceberg-cpp]

2025-03-23 Thread via GitHub
lidavidm commented on issue #46: URL: https://github.com/apache/iceberg-cpp/issues/46#issuecomment-2746737834 Thanks! It would be interesting to find a way to integrate this here -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [I] Kafka Connect: auto create with lowercase columns [iceberg]

2025-03-23 Thread via GitHub
github-actions[bot] commented on issue #11091: URL: https://github.com/apache/iceberg/issues/11091#issuecomment-2746574946 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache

Re: [PR] AWS: Move docker-based tests to integrationTest [iceberg]

2025-03-23 Thread via GitHub
andreanerla closed pull request #12625: AWS: Move docker-based tests to integrationTest URL: https://github.com/apache/iceberg/pull/12625 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

Re: [PR] SPARK: Remove dependency on hadoop's filesystem class from remove orphan files [iceberg]

2025-03-23 Thread via GitHub
liziyan-lzy commented on PR #12254: URL: https://github.com/apache/iceberg/pull/12254#issuecomment-2746725592 > @liziyan-lzy i think there is a problem with listprefix as is while removing orphan files for table invoices it also removed files from table invoices_other @eshishki Thank

Re: [PR] chore(deps): Bump rust_decimal from 1.36.0 to 1.37.1 [iceberg-rust]

2025-03-23 Thread via GitHub
liurenjie1024 merged PR #1124: URL: https://github.com/apache/iceberg-rust/pull/1124 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@

Re: [PR] V3: Introduce `timestamp_ns` and `timestamptz_ns` [iceberg-python]

2025-03-23 Thread via GitHub
Fokko commented on PR #1632: URL: https://github.com/apache/iceberg-python/pull/1632#issuecomment-2746373183 Thanks for working on this @sungwy, and thanks for the reviews @smaheshwar-pltr and @kevinjqliu đŸ™Œ -- This is an automated message from the Apache Git Service. To respond to the me

Re: [PR] chore(deps): Bump arrow-schema from 54.2.0 to 54.3.0 [iceberg-rust]

2025-03-23 Thread via GitHub
liurenjie1024 merged PR #1126: URL: https://github.com/apache/iceberg-rust/pull/1126 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@

Re: [PR] fix: safety ci using static check zizmor [iceberg-rust]

2025-03-23 Thread via GitHub
Xuanwo merged PR #1123: URL: https://github.com/apache/iceberg-rust/pull/1123 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [PR] Fix rounding of negative hour transform [iceberg-rust]

2025-03-23 Thread via GitHub
Xuanwo merged PR #1128: URL: https://github.com/apache/iceberg-rust/pull/1128 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [PR] chore(deps): Bump arrow-buffer from 54.2.0 to 54.3.0 [iceberg-rust]

2025-03-23 Thread via GitHub
liurenjie1024 merged PR #1127: URL: https://github.com/apache/iceberg-rust/pull/1127 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@

Re: [PR] fix: do not write ICEBERG_INCLUDES everywhere [iceberg-cpp]

2025-03-23 Thread via GitHub
Xuanwo merged PR #49: URL: https://github.com/apache/iceberg-cpp/pull/49 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

Re: [PR] add missing iceberg_export.h [iceberg-cpp]

2025-03-23 Thread via GitHub
Xuanwo merged PR #48: URL: https://github.com/apache/iceberg-cpp/pull/48 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

[PR] Add negative example [iceberg-rust]

2025-03-23 Thread via GitHub
Fokko opened a new pull request, #1128: URL: https://github.com/apache/iceberg-rust/pull/1128 ## Which issue does this PR close? - Closes #. ## What changes are included in this PR? ## Are these changes tested? -- This is an automated message fr

Re: [PR] Support In and notIn operators in ParquetFilters.ConvertFilterToParquet [iceberg]

2025-03-23 Thread via GitHub
github-actions[bot] commented on PR #12240: URL: https://github.com/apache/iceberg/pull/12240#issuecomment-2746575055 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

Re: [PR] AWS: fix GlueCatalog name validation [iceberg]

2025-03-23 Thread via GitHub
github-actions[bot] commented on PR #12367: URL: https://github.com/apache/iceberg/pull/12367#issuecomment-2746575088 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] Spec: Geo spec simplifications [iceberg]

2025-03-23 Thread via GitHub
szehon-ho commented on PR #12533: URL: https://github.com/apache/iceberg/pull/12533#issuecomment-2746621760 Thanks all for voting and review! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

Re: [PR] Spec: Geo spec simplifications [iceberg]

2025-03-23 Thread via GitHub
szehon-ho commented on PR #12533: URL: https://github.com/apache/iceberg/pull/12533#issuecomment-2746621407 The vote has passed: https://lists.apache.org/thread/cl177z548ts8vj7s7fmwo5k8tbl7fd9k -- This is an automated message from the Apache Git Service. To respond to the message, pleas

Re: [PR] Spec: Geo spec simplifications [iceberg]

2025-03-23 Thread via GitHub
szehon-ho merged PR #12533: URL: https://github.com/apache/iceberg/pull/12533 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [PR] Materialized View Spec [iceberg]

2025-03-23 Thread via GitHub
github-actions[bot] commented on PR #11041: URL: https://github.com/apache/iceberg/pull/11041#issuecomment-2746574923 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [I] Support data files compaction [iceberg-python]

2025-03-23 Thread via GitHub
github-actions[bot] commented on issue #1092: URL: https://github.com/apache/iceberg-python/issues/1092#issuecomment-2746577303 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity

Re: [I] Kafka Connect: auto create with lowercase columns [iceberg]

2025-03-23 Thread via GitHub
github-actions[bot] closed issue #11091: Kafka Connect: auto create with lowercase columns URL: https://github.com/apache/iceberg/issues/11091 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

Re: [PR] backport #11301(rowconverter) to Flink 1.19 and 1.18 [iceberg]

2025-03-23 Thread via GitHub
github-actions[bot] closed pull request #11826: backport #11301(rowconverter) to Flink 1.19 and 1.18 URL: https://github.com/apache/iceberg/pull/11826 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Support In and notIn operators in ParquetFilters.ConvertFilterToParquet [iceberg]

2025-03-23 Thread via GitHub
github-actions[bot] closed pull request #12240: Support In and notIn operators in ParquetFilters.ConvertFilterToParquet URL: https://github.com/apache/iceberg/pull/12240 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] Use Snapshot's statistics file in SparkScan [iceberg]

2025-03-23 Thread via GitHub
github-actions[bot] commented on PR #11040: URL: https://github.com/apache/iceberg/pull/11040#issuecomment-2746574904 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] backport #11301(rowconverter) to Flink 1.19 and 1.18 [iceberg]

2025-03-23 Thread via GitHub
github-actions[bot] commented on PR #11826: URL: https://github.com/apache/iceberg/pull/11826#issuecomment-2746575017 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

Re: [I] DatasourceV2 does not prune columns after V2ScanRelationPushDown [iceberg]

2025-03-23 Thread via GitHub
github-actions[bot] commented on issue #9268: URL: https://github.com/apache/iceberg/issues/9268#issuecomment-2746574876 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

[PR] Support Snowflake-Managed Iceberg Tables via SnowflakeCatalog [iceberg-python]

2025-03-23 Thread via GitHub
srilman opened a new pull request, #1834: URL: https://github.com/apache/iceberg-python/pull/1834 Closes #685. # Rationale for this change Reopens PR #687 that adds a Snowflake Catalog that was closed. I addressed comments and applied some additional changes based on errors fou

Re: [PR] Build: Bump nessie from 0.103.0 to 0.103.2 [iceberg]

2025-03-23 Thread via GitHub
Fokko merged PR #12615: URL: https://github.com/apache/iceberg/pull/12615 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] Build: Bump mkdocs-material from 9.6.8 to 9.6.9 [iceberg]

2025-03-23 Thread via GitHub
Fokko merged PR #12614: URL: https://github.com/apache/iceberg/pull/12614 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] Build: Bump getdaft from 0.4.7 to 0.4.8 [iceberg-python]

2025-03-23 Thread via GitHub
Fokko merged PR #1826: URL: https://github.com/apache/iceberg-python/pull/1826 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [I] Iceberg Kafka Connect Sink has AVRO 1.12.0, but Kafka Connect AVRO Converter 7.9.0 wants 1.11.4 and therefore Avro Converter does not work [iceberg]

2025-03-23 Thread via GitHub
lk-1984 closed issue #12571: Iceberg Kafka Connect Sink has AVRO 1.12.0, but Kafka Connect AVRO Converter 7.9.0 wants 1.11.4 and therefore Avro Converter does not work URL: https://github.com/apache/iceberg/issues/12571 -- This is an automated message from the Apache Git Service. To respond

[PR] fix: do not write ICEBERG_INCLUDES everywhere [iceberg-cpp]

2025-03-23 Thread via GitHub
wgtmac opened a new pull request, #49: URL: https://github.com/apache/iceberg-cpp/pull/49 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mai

Re: [PR] [Do not merge] Iterative `bind` with a stack instead of recursion [iceberg-python]

2025-03-23 Thread via GitHub
smaheshwar-pltr commented on PR #1783: URL: https://github.com/apache/iceberg-python/pull/1783#issuecomment-2746204012 Thanks for taking a look folks, and apologies for the delayed response. > are there any reasons we don't want to do this? This approach serves only to circumven

[PR] chore(deps): Bump arrow-schema from 54.2.0 to 54.3.0 [iceberg-rust]

2025-03-23 Thread via GitHub
dependabot[bot] opened a new pull request, #1126: URL: https://github.com/apache/iceberg-rust/pull/1126 Bumps [arrow-schema](https://github.com/apache/arrow-rs) from 54.2.0 to 54.3.0. Release notes Sourced from https://github.com/apache/arrow-rs/releases";>arrow-schema's releases.

[PR] chore(deps): Bump typed-builder from 0.20.0 to 0.20.1 [iceberg-rust]

2025-03-23 Thread via GitHub
dependabot[bot] opened a new pull request, #1125: URL: https://github.com/apache/iceberg-rust/pull/1125 Bumps [typed-builder](https://github.com/idanarye/rust-typed-builder) from 0.20.0 to 0.20.1. Release notes Sourced from https://github.com/idanarye/rust-typed-builder/releases";>

[PR] chore(deps): Bump rust_decimal from 1.36.0 to 1.37.1 [iceberg-rust]

2025-03-23 Thread via GitHub
dependabot[bot] opened a new pull request, #1124: URL: https://github.com/apache/iceberg-rust/pull/1124 Bumps [rust_decimal](https://github.com/paupino/rust-decimal) from 1.36.0 to 1.37.1. Release notes Sourced from https://github.com/paupino/rust-decimal/releases";>rust_decimal's

Re: [PR] Add pure virtual classes for Catalog, Table, etc. [iceberg-cpp]

2025-03-23 Thread via GitHub
wgtmac commented on code in PR #47: URL: https://github.com/apache/iceberg-cpp/pull/47#discussion_r2007098812 ## src/iceberg/table_identifier.h: ## @@ -0,0 +1,43 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. Se

Re: [PR] Build: Bump coverage from 7.7.0 to 7.7.1 [iceberg-python]

2025-03-23 Thread via GitHub
Fokko merged PR #1827: URL: https://github.com/apache/iceberg-python/pull/1827 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

[PR] build(deps): bump google.golang.org/api from 0.226.0 to 0.227.0 in the gomod_updates group [iceberg-go]

2025-03-23 Thread via GitHub
dependabot[bot] opened a new pull request, #358: URL: https://github.com/apache/iceberg-go/pull/358 Bumps the gomod_updates group with 1 update: [google.golang.org/api](https://github.com/googleapis/google-api-go-client). Updates `google.golang.org/api` from 0.226.0 to 0.227.0

Re: [PR] Spark 3.4: Backport Spark actions changes in Spark rewrite_table_path procedure (#12006 #12172 #11929 #12282 #12569) [iceberg]

2025-03-23 Thread via GitHub
slfan1989 commented on PR #12568: URL: https://github.com/apache/iceberg/pull/12568#issuecomment-2746075887 @nastra @szehon-ho Could you please help review this PR again? Thank you very much. The latest commit includes #12282 and #12569. -- This is an automated message from the Apache Git

[PR] [Build]: Enforce error message check on Exception assertions [iceberg]

2025-03-23 Thread via GitHub
lliangyu-lin opened a new pull request, #12623: URL: https://github.com/apache/iceberg/pull/12623 ### Description * Added regex to check usage of `assertThatThrownBy` and `assertThatExceptionOfType` need to include check on the error message content. * Resolve all current files that di

[PR] Build: Enforce error message check on Exception assertions [iceberg]

2025-03-23 Thread via GitHub
lliangyu-lin opened a new pull request, #12624: URL: https://github.com/apache/iceberg/pull/12624 ### Description * Added regex to check usage of `assertThatThrownBy` and `assertThatExceptionOfType` need to include check on the error message content. * Resolve all current files that di

Re: [PR] Build: Enforce error message check on Exception assertions [iceberg]

2025-03-23 Thread via GitHub
lliangyu-lin closed pull request #12623: Build: Enforce error message check on Exception assertions URL: https://github.com/apache/iceberg/pull/12623 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [I] com.esotericsoftware.kryo.KryoException: java.lang.ClassCastException: java.lang.Integer cannot be cast to java.nio.ByteBuffer [iceberg]

2025-03-23 Thread via GitHub
github-actions[bot] commented on issue #9738: URL: https://github.com/apache/iceberg/issues/9738#issuecomment-2745929556 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi