[PR] Core: Move `iceberg-parquet` files to `iceberg-core` [iceberg]

2025-01-29 Thread via GitHub
ajantha-bhat opened a new pull request, #8500: URL: https://github.com/apache/iceberg/pull/8500 - Since core module need to write stats in parquet format, to avoid circular dependency, move all the files from iceberg-parquet module to iceberg code. Fixes #8453 -- This is an automa

[PR] Add Azure support [iceberg-go]

2025-01-29 Thread via GitHub
jbonofre opened a new pull request, #278: URL: https://github.com/apache/iceberg-go/pull/278 @zeroshade That's a first attend to support Azure storage. I will ping you for guidance πŸ˜„ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] Auth Manager API part 4: RESTClient, HTTPClient [iceberg]

2025-01-29 Thread via GitHub
adutra commented on code in PR #11992: URL: https://github.com/apache/iceberg/pull/11992#discussion_r1933705991 ## aws/src/main/java/org/apache/iceberg/aws/s3/signer/S3V4RestSignerClient.java: ## @@ -192,7 +192,8 @@ private RESTClient httpClient() { HTTPClient.bui

[PR] Minor: typo in url [iceberg]

2025-01-29 Thread via GitHub
Fokko opened a new pull request, #12126: URL: https://github.com/apache/iceberg/pull/12126 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-ma

Re: [PR] Core: Support removing keys from EnvironmentContext [iceberg]

2025-01-29 Thread via GitHub
nastra commented on code in PR #12103: URL: https://github.com/apache/iceberg/pull/12103#discussion_r1933780261 ## core/src/test/java/org/apache/iceberg/TestEnvironmentContext.java: ## @@ -19,6 +19,7 @@ package org.apache.iceberg; import static org.assertj.core.api.Assertion

Re: [I] Support for timestamp downcasting when loading data to iceberg tables [iceberg-python]

2025-01-29 Thread via GitHub
fusion commented on issue #1045: URL: https://github.com/apache/iceberg-python/issues/1045#issuecomment-2621499506 I will close my PR. The codebase seems not to be newcomer friendly and it seems like @lloyd-EA already has some experience with pyiceberg library. -- This is an automat

Re: [PR] Enable pyiceberg.table.Table.add_files ns downcasting [iceberg-python]

2025-01-29 Thread via GitHub
fusion closed pull request #1572: Enable pyiceberg.table.Table.add_files ns downcasting URL: https://github.com/apache/iceberg-python/pull/1572 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

Re: [PR] Core: Add InternalData read and write builders [iceberg]

2025-01-29 Thread via GitHub
pvary commented on code in PR #12060: URL: https://github.com/apache/iceberg/pull/12060#discussion_r1933980314 ## core/src/main/java/org/apache/iceberg/InternalData.java: ## @@ -0,0 +1,159 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contribu

Re: [PR] Minor: typo in url [iceberg]

2025-01-29 Thread via GitHub
nastra merged PR #12126: URL: https://github.com/apache/iceberg/pull/12126 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

[PR] Fixed linter error [iceberg-python]

2025-01-29 Thread via GitHub
gayatrikate04 opened a new pull request, #1590: URL: https://github.com/apache/iceberg-python/pull/1590 This pull request updates the docstrings and fixes the linter errors that were identified in the previous PR (#1533) Changes: Updated docstrings for improved documentation and r

Re: [PR] Docs: Fix latest and nightly link on javadoc (according to site README.md) [iceberg]

2025-01-29 Thread via GitHub
jbonofre commented on PR #12023: URL: https://github.com/apache/iceberg/pull/12023#issuecomment-2621557542 @Fokko gentle reminder πŸ˜„ Thanks ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

[PR] Add "clean" NOTICE/LICENSE in jar files [iceberg]

2025-01-29 Thread via GitHub
jbonofre opened a new pull request, #12127: URL: https://github.com/apache/iceberg/pull/12127 Our distributed jar artifacts use the `LICENSE` and `NOTICE` from the root directory. These `LICENSE` and `NOTICE` files are valid for source distribution, but it's not correct for the distributed

Re: [PR] Issue 1045 - Allow downcasting of parquet timestamp[ns] to timestamp[us] for Table.add_files() [iceberg-python]

2025-01-29 Thread via GitHub
sungwy commented on PR #1569: URL: https://github.com/apache/iceberg-python/pull/1569#issuecomment-2621569571 Thank you for following up on the suggestion @lloyd-EA . I'm glad we included this integration test to find out the issue with Spark's timestamp conversion. I think we will h

Re: [I] Support for timestamp downcasting when loading data to iceberg tables [iceberg-python]

2025-01-29 Thread via GitHub
sungwy commented on issue #1045: URL: https://github.com/apache/iceberg-python/issues/1045#issuecomment-2621578812 Sorry you had that experience @fusion ! There's of course a lot of Iceberg specific context here on this repository, and I'm hoping we can continue to work to build a libr

[I] In Test for iceberg + Hive Catalog jackson library shade [iceberg]

2025-01-29 Thread via GitHub
2MD opened a new issue, #12130: URL: https://github.com/apache/iceberg/issues/12130 ### Apache Iceberg version 1.6.0 ### Query engine Spark ### Please describe the bug 🐞 val icebergVersion = "1.6.0" scalaVersion := "2.12.15" val sparkVersion = "3.3.2"

[I] Class TestHiveMetastore use getSystemClassLoader instead of getClass.getClassLoader in setupMetastoreDB [iceberg]

2025-01-29 Thread via GitHub
2MD opened a new issue, #12131: URL: https://github.com/apache/iceberg/issues/12131 ### Apache Iceberg version 1.7.1 (latest release) ### Query engine Spark ### Please describe the bug 🐞 ``` val icebergVersion = "1.7.1" val sparkVersion = "3.3.2"

Re: [PR] Make view metadata path configurable by `write.metadata.path` [iceberg]

2025-01-29 Thread via GitHub
tomtongue commented on code in PR #12017: URL: https://github.com/apache/iceberg/pull/12017#discussion_r1933677900 ## core/src/test/java/org/apache/iceberg/view/ViewCatalogTests.java: ## @@ -239,6 +239,36 @@ public void completeCreateView() { assertThat(catalog().viewExists

Re: [PR] Core: don't override `LoggingMetricsReporter` [iceberg]

2025-01-29 Thread via GitHub
nastra commented on PR #12092: URL: https://github.com/apache/iceberg/pull/12092#issuecomment-2621884278 > Hi @mst , I was looking at this code myself recently too. Initially I also thought that we shouldn't swap LoggingMetricsReporter here and we should combine it with the one received as

Re: [PR] Core: don't override `LoggingMetricsReporter` [iceberg]

2025-01-29 Thread via GitHub
nastra commented on PR #12092: URL: https://github.com/apache/iceberg/pull/12092#issuecomment-2621887691 > Hi @mst , I was looking at this code myself recently too. Initially I also thought that we shouldn't swap LoggingMetricsReporter here and we should combine it with the one received as

Re: [PR] Core: don't override `LoggingMetricsReporter` [iceberg]

2025-01-29 Thread via GitHub
nastra closed pull request #12092: Core: don't override `LoggingMetricsReporter` URL: https://github.com/apache/iceberg/pull/12092 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

Re: [PR] feat(datafusion): Expose DataFusion statistics on an IcebergTableScan [iceberg-rust]

2025-01-29 Thread via GitHub
gruuya commented on PR #880: URL: https://github.com/apache/iceberg-rust/pull/880#issuecomment-2621261175 I think this PR is ready for review again; @ZENOTME @liurenjie1024 can you take a look and share additional feedback? If this proposal makes sense I can start adding more tests. I

Re: [PR] Core: Move `iceberg-parquet` files to `iceberg-core` [iceberg]

2025-01-29 Thread via GitHub
ajantha-bhat commented on code in PR #8500: URL: https://github.com/apache/iceberg/pull/8500#discussion_r1933732242 ## build.gradle: ## @@ -897,7 +874,9 @@ project(':iceberg-snowflake') { } dependencies { -implementation project(':iceberg-core') +implementation(p

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-01-29 Thread via GitHub
mattmartin14 commented on code in PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#discussion_r1933957191 ## pyiceberg/table/__init__.py: ## @@ -1064,6 +1064,125 @@ def name_mapping(self) -> Optional[NameMapping]: """Return the table's field-id NameMapp

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-01-29 Thread via GitHub
Fokko commented on code in PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#discussion_r1933527728 ## pyiceberg/table/__init__.py: ## @@ -1064,6 +1064,125 @@ def name_mapping(self) -> Optional[NameMapping]: """Return the table's field-id NameMapping."""

Re: [PR] Core: don't override `LoggingMetricsReporter` [iceberg]

2025-01-29 Thread via GitHub
gaborkaszab commented on PR #12092: URL: https://github.com/apache/iceberg/pull/12092#issuecomment-2621101631 Hi @mst , I was looking at this code myself recently too. Initially I also thought that we shouldn't swap LoggingMetricsReporter here and we should combine it with the one receiv

[I] FileIO Implementation Configuration Priority Question [iceberg-python]

2025-01-29 Thread via GitHub
bigluck opened a new issue, #1589: URL: https://github.com/apache/iceberg-python/issues/1589 ### Apache Iceberg version None ### Please describe the bug 🐞 Hey team! πŸ‘‹ Hope you're doing well! I've been working with PyIceberg and ran into an interesting situation regard

Re: [I] Spark rewrite_data_files failing with java.lang.IllegalStateException: Connection pool shut down [iceberg]

2025-01-29 Thread via GitHub
mgmarino commented on issue #12046: URL: https://github.com/apache/iceberg/issues/12046#issuecomment-2621663034 Ok, I finally have a full explanation. The issue is that Spark is cleaning up memory, moving broadcast variables to disk and this results in the closure of the I/O even if it's cu

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-01-29 Thread via GitHub
mattmartin14 commented on code in PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#discussion_r1933902417 ## pyiceberg/table/__init__.py: ## @@ -1064,6 +1064,125 @@ def name_mapping(self) -> Optional[NameMapping]: """Return the table's field-id NameMapp

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-01-29 Thread via GitHub
mattmartin14 commented on code in PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#discussion_r1933904478 ## pyiceberg/table/__init__.py: ## @@ -1064,6 +1064,125 @@ def name_mapping(self) -> Optional[NameMapping]: """Return the table's field-id NameMapp

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-01-29 Thread via GitHub
mattmartin14 commented on code in PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#discussion_r1933907086 ## pyiceberg/table/__init__.py: ## @@ -1064,6 +1064,125 @@ def name_mapping(self) -> Optional[NameMapping]: """Return the table's field-id NameMapp

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-01-29 Thread via GitHub
mattmartin14 commented on code in PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#discussion_r1933908403 ## pyiceberg/table/__init__.py: ## @@ -1064,6 +1064,125 @@ def name_mapping(self) -> Optional[NameMapping]: """Return the table's field-id NameMapp

Re: [PR] Spec: Fix minor typo in `_last_updated_sequence_number` docs [iceberg]

2025-01-29 Thread via GitHub
smaheshwar-pltr commented on PR #12128: URL: https://github.com/apache/iceberg/pull/12128#issuecomment-2621696714 cc @RussellSpitzer maybe, mind taking a look? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

[PR] Spec: Fix minor typo in `_last_updated_sequence_number` docs [iceberg]

2025-01-29 Thread via GitHub
smaheshwar-pltr opened a new pull request, #12128: URL: https://github.com/apache/iceberg/pull/12128 The docs for `_row_id` has "see Row Lineage" but `_last_updated_sequence_number` has "Row Lineage" by itself, which reads like a small typo. -- This is an automated message from the Apach

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-01-29 Thread via GitHub
mattmartin14 commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2621699461 Hi All, I've pushed a new update through and asked for some clarrification from @Fokko on some of his suggestions. In terms of the call we had yesterday (1/28/25), I think

[PR] Spark: Remove closing of IO in SerializableTable* [iceberg]

2025-01-29 Thread via GitHub
mgmarino opened a new pull request, #12129: URL: https://github.com/apache/iceberg/pull/12129 This is to fix: #12046 To summarize, the issue is that Spark can remove broadcast variables from memory and persist them to disk in case that memory needs to be freed. In the case that

Re: [PR] Spark: Remove closing of IO in SerializableTable* [iceberg]

2025-01-29 Thread via GitHub
mgmarino commented on PR #12129: URL: https://github.com/apache/iceberg/pull/12129#issuecomment-2621728020 I am happy to get input here as to whether or not this is the correct way to solve this issue and am happy to adapt as necessary. Thanks! This effectively reverts: #8924 -- Th

Re: [PR] Add relevant NOTICE portions from ALv2 bundled dependencies [iceberg]

2025-01-29 Thread via GitHub
jbonofre commented on PR #12095: URL: https://github.com/apache/iceberg/pull/12095#issuecomment-2622082894 @amogh-jahagirdar @rdblue @Fokko I fixed the versions in the bundle jar files. I added fixed on flink-runtime. I'm checking/updating LICENSE/NOTICE in spark-runtime now. -- This is

Re: [PR] Build: Bump scala-collection-compat from 2.12.0 to 2.13.0 [iceberg]

2025-01-29 Thread via GitHub
findepi merged PR #12121: URL: https://github.com/apache/iceberg/pull/12121 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.a

Re: [PR] Reduce 'Scanning table' log verbosity for long IN list [iceberg]

2025-01-29 Thread via GitHub
findepi commented on code in PR #5908: URL: https://github.com/apache/iceberg/pull/5908#discussion_r1933444518 ## api/src/main/java/org/apache/iceberg/expressions/ExpressionUtil.java: ## @@ -258,6 +270,23 @@ public String predicate(UnboundPredicate pred) { } } + priv

Re: [PR] Hive metastore register table [iceberg-python]

2025-01-29 Thread via GitHub
JoniKet commented on code in PR #1580: URL: https://github.com/apache/iceberg-python/pull/1580#discussion_r1933478098 ## tests/catalog/test_hive.py: ## @@ -204,6 +204,87 @@ def test_check_number_of_namespaces(table_schema_simple: Schema) -> None: catalog.create_table("

Re: [I] Support for timestamp downcasting when loading data to iceberg tables [iceberg-python]

2025-01-29 Thread via GitHub
adaudeh commented on issue #1045: URL: https://github.com/apache/iceberg-python/issues/1045#issuecomment-2621039944 please merge already..this is a huge blocker... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

Re: [PR] support create table like in flink catalog and watermark in windows [iceberg]

2025-01-29 Thread via GitHub
pvary commented on code in PR #12116: URL: https://github.com/apache/iceberg/pull/12116#discussion_r1933513558 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/source/IcebergTableSource.java: ## @@ -53,7 +54,8 @@ public class IcebergTableSource implements ScanTab

Re: [PR] support create table like in flink catalog and watermark in windows [iceberg]

2025-01-29 Thread via GitHub
pvary commented on code in PR #12116: URL: https://github.com/apache/iceberg/pull/12116#discussion_r1933510999 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/FlinkCatalog.java: ## @@ -384,13 +395,6 @@ public void renameTable(ObjectPath tablePath, String newTableNam

Re: [PR] Spec: add variant type [iceberg]

2025-01-29 Thread via GitHub
emkornfield commented on code in PR #10831: URL: https://github.com/apache/iceberg/pull/10831#discussion_r1934723602 ## format/spec.md: ## @@ -1208,6 +1223,7 @@ Lists must use the [3-level representation](https://github.com/apache/parquet-fo | **`struct`** | `group`

Re: [PR] #12081: "Add deleteFileThreshold parameter to SizeBasedDataRewriter, update logic, and include tests" [iceberg]

2025-01-29 Thread via GitHub
jangalasriramd7 commented on PR #12133: URL: https://github.com/apache/iceberg/pull/12133#issuecomment-2622966642 I have changed the PR title. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] Add "clean" NOTICE/LICENSE in jar files [iceberg]

2025-01-29 Thread via GitHub
amogh-jahagirdar commented on code in PR #12127: URL: https://github.com/apache/iceberg/pull/12127#discussion_r1934724011 ## licenses/LICENSE: ## @@ -0,0 +1,202 @@ + + Apache License + Version 2.0, January 2004 +

Re: [PR] Add "clean" NOTICE/LICENSE in jar files [iceberg]

2025-01-29 Thread via GitHub
amogh-jahagirdar commented on code in PR #12127: URL: https://github.com/apache/iceberg/pull/12127#discussion_r1934728523 ## licenses/LICENSE: ## @@ -0,0 +1,202 @@ + + Apache License + Version 2.0, January 2004 +

Re: [PR] Docs: Add rewrite-table-path in spark procedure [iceberg]

2025-01-29 Thread via GitHub
dramaticlly commented on PR #12115: URL: https://github.com/apache/iceberg/pull/12115#issuecomment-2622983101 thank you Russell for the detailed review! This is getting closer and I did a local check to ensure the info and warning section rendered as expected like in following screenshot. h

Re: [PR] Core: Relocate parquet to core [iceberg]

2025-01-29 Thread via GitHub
danielcweeks commented on PR #11716: URL: https://github.com/apache/iceberg/pull/11716#issuecomment-2622982192 Closing in favor of #12060 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

Re: [PR] Core: Relocate parquet to core [iceberg]

2025-01-29 Thread via GitHub
danielcweeks closed pull request #11716: Core: Relocate parquet to core URL: https://github.com/apache/iceberg/pull/11716 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To uns

Re: [PR] Spec: add variant type [iceberg]

2025-01-29 Thread via GitHub
aihuaxu commented on code in PR #10831: URL: https://github.com/apache/iceberg/pull/10831#discussion_r1934733942 ## format/spec.md: ## @@ -1208,6 +1223,7 @@ Lists must use the [3-level representation](https://github.com/apache/parquet-fo | **`struct`** | `group`

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2025-01-29 Thread via GitHub
emkornfield commented on PR #11039: URL: https://github.com/apache/iceberg/pull/11039#issuecomment-2623021449 > > @hesham-medhat @rdblue - could you pls give an update on this PR? It seems it would massively simplify the Iceberg table management in GCP for non-spark usecases. > > thanks!

Re: [PR] #12081: "Add deleteFileThreshold parameter to SizeBasedDataRewriter, update logic, and include tests" [iceberg]

2025-01-29 Thread via GitHub
RussellSpitzer commented on code in PR #12133: URL: https://github.com/apache/iceberg/pull/12133#discussion_r1934752632 ## core/src/main/java/org/apache/iceberg/actions/SizeBasedDataRewriter.java: ## @@ -47,12 +47,19 @@ public abstract class SizeBasedDataRewriter extends SizeBa

Re: [PR] #12081: "Add deleteFileThreshold parameter to SizeBasedDataRewriter, update logic, and include tests" [iceberg]

2025-01-29 Thread via GitHub
RussellSpitzer commented on code in PR #12133: URL: https://github.com/apache/iceberg/pull/12133#discussion_r1934753900 ## core/src/main/java/org/apache/iceberg/actions/SizeBasedDataRewriter.java: ## @@ -47,12 +47,19 @@ public abstract class SizeBasedDataRewriter extends SizeBa

Re: [PR] #12081: "Add deleteFileThreshold parameter to SizeBasedDataRewriter, update logic, and include tests" [iceberg]

2025-01-29 Thread via GitHub
RussellSpitzer commented on code in PR #12133: URL: https://github.com/apache/iceberg/pull/12133#discussion_r1934753417 ## core/src/main/java/org/apache/iceberg/actions/SizeBasedDataRewriter.java: ## @@ -47,12 +47,19 @@ public abstract class SizeBasedDataRewriter extends SizeBa

Re: [PR] Core: Avro writers use BlockingBinaryEncoder to enable array/map size calculations. [iceberg]

2025-01-29 Thread via GitHub
github-actions[bot] commented on PR #8625: URL: https://github.com/apache/iceberg/pull/8625#issuecomment-2623222605 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Encryption integration and test [iceberg]

2025-01-29 Thread via GitHub
github-actions[bot] closed pull request #5544: Encryption integration and test URL: https://github.com/apache/iceberg/pull/5544 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] API,Core: Introduce metrics for data files by file format [iceberg]

2025-01-29 Thread via GitHub
github-actions[bot] commented on PR #5837: URL: https://github.com/apache/iceberg/pull/5837#issuecomment-2623222418 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] feat(catalog): Propagate ctx from catalog interface through call stack [iceberg-go]

2025-01-29 Thread via GitHub
zeroshade merged PR #276: URL: https://github.com/apache/iceberg-go/pull/276 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] Core: Make metrics reporter serializable (alternative impl) [iceberg]

2025-01-29 Thread via GitHub
github-actions[bot] commented on PR #8032: URL: https://github.com/apache/iceberg/pull/8032#issuecomment-2623222552 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [I] The "Emitting watermarks" feature can't be used in flink sql? [iceberg]

2025-01-29 Thread via GitHub
github-actions[bot] commented on issue #10219: URL: https://github.com/apache/iceberg/issues/10219#issuecomment-2623222735 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [I] Running MERGE INTO with more than one WHEN condition fails if the number of columns in the target table is > 321 [iceberg]

2025-01-29 Thread via GitHub
github-actions[bot] commented on issue #10294: URL: https://github.com/apache/iceberg/issues/10294#issuecomment-2623222767 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [I] Please remove old releases [iceberg]

2025-01-29 Thread via GitHub
github-actions[bot] commented on issue #2414: URL: https://github.com/apache/iceberg/issues/2414#issuecomment-262393 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [PR] Encryption integration and test [iceberg]

2025-01-29 Thread via GitHub
github-actions[bot] commented on PR #5544: URL: https://github.com/apache/iceberg/pull/5544#issuecomment-2623222370 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Spec: add variant type [iceberg]

2025-01-29 Thread via GitHub
sfc-gh-aixu commented on PR #10831: URL: https://github.com/apache/iceberg/pull/10831#issuecomment-2623219808 > Merged, thanks everyone for your thoughtful feedback. Thanks @aihuaxu for the pr. Thanks to @rdblue , @emkornfield , @findepi , @XBaith , and @flyrain for your contributions revie

Re: [PR] API,Core: Introduce metrics for data files by file format [iceberg]

2025-01-29 Thread via GitHub
github-actions[bot] closed pull request #5837: API,Core: Introduce metrics for data files by file format URL: https://github.com/apache/iceberg/pull/5837 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] Core: Make metrics reporter serializable (alternative impl) [iceberg]

2025-01-29 Thread via GitHub
github-actions[bot] closed pull request #8032: Core: Make metrics reporter serializable (alternative impl) URL: https://github.com/apache/iceberg/pull/8032 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] API: New API For sequential / streaming updates [iceberg]

2025-01-29 Thread via GitHub
github-actions[bot] closed pull request #9323: API: New API For sequential / streaming updates URL: https://github.com/apache/iceberg/pull/9323 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

Re: [PR] Kafka Connect: Add config to route to tables using topic name [iceberg]

2025-01-29 Thread via GitHub
github-actions[bot] commented on PR #11313: URL: https://github.com/apache/iceberg/pull/11313#issuecomment-2623223008 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] Spark: support rewrite on specified target branch [iceberg]

2025-01-29 Thread via GitHub
github-actions[bot] commented on PR #8797: URL: https://github.com/apache/iceberg/pull/8797#issuecomment-2623222633 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] API: New API For sequential / streaming updates [iceberg]

2025-01-29 Thread via GitHub
github-actions[bot] commented on PR #9323: URL: https://github.com/apache/iceberg/pull/9323#issuecomment-2623222682 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] JDBC: Escape table names when checking the existence [iceberg]

2025-01-29 Thread via GitHub
github-actions[bot] commented on PR #11863: URL: https://github.com/apache/iceberg/pull/11863#issuecomment-2623223185 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

Re: [PR] Parquet: add variant type support [iceberg]

2025-01-29 Thread via GitHub
github-actions[bot] commented on PR #11653: URL: https://github.com/apache/iceberg/pull/11653#issuecomment-2623223077 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

Re: [PR] API,Core: Support Conditional Commits [iceberg]

2025-01-29 Thread via GitHub
github-actions[bot] closed pull request #6513: API,Core: Support Conditional Commits URL: https://github.com/apache/iceberg/pull/6513 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

Re: [PR] Parquet: add variant type support [iceberg]

2025-01-29 Thread via GitHub
github-actions[bot] closed pull request #11653: Parquet: add variant type support URL: https://github.com/apache/iceberg/pull/11653 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

Re: [I] A move after a rename fails [iceberg]

2025-01-29 Thread via GitHub
github-actions[bot] commented on issue #10830: URL: https://github.com/apache/iceberg/issues/10830#issuecomment-2623222951 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [PR] Core: Avro writers use BlockingBinaryEncoder to enable array/map size calculations. [iceberg]

2025-01-29 Thread via GitHub
github-actions[bot] closed pull request #8625: Core: Avro writers use BlockingBinaryEncoder to enable array/map size calculations. URL: https://github.com/apache/iceberg/pull/8625 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

Re: [I] Support Conditional Transaction Commits [iceberg]

2025-01-29 Thread via GitHub
github-actions[bot] commented on issue #6514: URL: https://github.com/apache/iceberg/issues/6514#issuecomment-2623222467 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [PR] Fix Hive FileIO closing with FileIOTracker [iceberg]

2025-01-29 Thread via GitHub
github-actions[bot] commented on PR #11782: URL: https://github.com/apache/iceberg/pull/11782#issuecomment-2623223116 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

Re: [PR] JDBC: Escape table names when checking the existence [iceberg]

2025-01-29 Thread via GitHub
github-actions[bot] closed pull request #11863: JDBC: Escape table names when checking the existence URL: https://github.com/apache/iceberg/pull/11863 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Fix Hive FileIO closing with FileIOTracker [iceberg]

2025-01-29 Thread via GitHub
github-actions[bot] closed pull request #11782: Fix Hive FileIO closing with FileIOTracker URL: https://github.com/apache/iceberg/pull/11782 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

Re: [PR] Spark: support rewrite on specified target branch [iceberg]

2025-01-29 Thread via GitHub
github-actions[bot] closed pull request #8797: Spark: support rewrite on specified target branch URL: https://github.com/apache/iceberg/pull/8797 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] API,Core: Support Conditional Commits [iceberg]

2025-01-29 Thread via GitHub
github-actions[bot] commented on PR #6513: URL: https://github.com/apache/iceberg/pull/6513#issuecomment-2623222450 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Core: Fix cleanup of orphaned statistics files in dropTableData [iceberg]

2025-01-29 Thread via GitHub
ebyhr commented on code in PR #12132: URL: https://github.com/apache/iceberg/pull/12132#discussion_r1934925978 ## core/src/test/java/org/apache/iceberg/hadoop/TestCatalogUtilDropTable.java: ## @@ -129,6 +130,81 @@ public void dropTableDataDeletesExpectedFiles() throws IOExcepti

Re: [PR] Core: Fix cleanup of orphaned statistics files in dropTableData [iceberg]

2025-01-29 Thread via GitHub
ebyhr commented on code in PR #12132: URL: https://github.com/apache/iceberg/pull/12132#discussion_r1934925978 ## core/src/test/java/org/apache/iceberg/hadoop/TestCatalogUtilDropTable.java: ## @@ -129,6 +130,81 @@ public void dropTableDataDeletesExpectedFiles() throws IOExcepti

Re: [PR] Core: Fix cleanup of orphaned statistics files in dropTableData [iceberg]

2025-01-29 Thread via GitHub
ebyhr commented on code in PR #12132: URL: https://github.com/apache/iceberg/pull/12132#discussion_r1934928382 ## core/src/test/java/org/apache/iceberg/hadoop/TestCatalogUtilDropTable.java: ## @@ -129,6 +130,81 @@ public void dropTableDataDeletesExpectedFiles() throws IOExcepti

Re: [PR] Spark: Support singular form of years, months, days, and hours functions [iceberg]

2025-01-29 Thread via GitHub
ebyhr commented on code in PR #12117: URL: https://github.com/apache/iceberg/pull/12117#discussion_r1934934393 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/sql/TestSparkDaysFunction.java: ## @@ -39,6 +39,9 @@ public void testDates() { assertThat(scalarSql("SEL

Re: [PR] Spark: Support singular form of years, months, days, and hours functions [iceberg]

2025-01-29 Thread via GitHub
ebyhr commented on code in PR #12117: URL: https://github.com/apache/iceberg/pull/12117#discussion_r1934934393 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/sql/TestSparkDaysFunction.java: ## @@ -39,6 +39,9 @@ public void testDates() { assertThat(scalarSql("SEL

Re: [PR] API: Define RepairManifests action interface [iceberg]

2025-01-29 Thread via GitHub
amogh-jahagirdar commented on PR #10784: URL: https://github.com/apache/iceberg/pull/10784#issuecomment-2623396968 > For my edification, can someone please explain how duplicate file entries in manifests can arise? Can two entries for the same file occur in a single manifest? Can even two

Re: [PR] Iceberg/Comet integration POC [iceberg]

2025-01-29 Thread via GitHub
parthchandra commented on code in PR #9841: URL: https://github.com/apache/iceberg/pull/9841#discussion_r1934545952 ## spark/v3.4/spark/src/main/java/org/apache/iceberg/spark/data/vectorized/CometColumnReader.java: ## @@ -0,0 +1,149 @@ +/* + * Licensed to the Apache Software Fou

Re: [PR] feat(catalog): Initial implementation of sql catalog [iceberg-go]

2025-01-29 Thread via GitHub
zeroshade commented on code in PR #246: URL: https://github.com/apache/iceberg-go/pull/246#discussion_r1934571592 ## catalog/sql/sql.go: ## @@ -0,0 +1,721 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTIC

[PR] Core: Fix cleanup of orphaned statistics files in dropTableData [iceberg]

2025-01-29 Thread via GitHub
lliangyu-lin opened a new pull request, #12132: URL: https://github.com/apache/iceberg/pull/12132 ### Description Currently, Iceberg ```dropTableData()``` does not properly delete statistics files (```.stats```) that are replaced by newer statistics files. When ```updateStatistics()``` i

Re: [PR] feat(catalog): Initial implementation of sql catalog [iceberg-go]

2025-01-29 Thread via GitHub
kevinjqliu commented on code in PR #246: URL: https://github.com/apache/iceberg-go/pull/246#discussion_r1934685789 ## catalog/sql/sql.go: ## @@ -0,0 +1,721 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTI

Re: [PR] feat(catalog): Propagate ctx from catalog interface through call stack [iceberg-go]

2025-01-29 Thread via GitHub
curtisr7 commented on code in PR #276: URL: https://github.com/apache/iceberg-go/pull/276#discussion_r1934689581 ## catalog/registry.go: ## @@ -64,13 +65,13 @@ var ( // Registrar is a factory for creating Catalog instances, used for registering to use // with LoadCatalog. ty

Re: [PR] Update flink docs with alter column support [iceberg]

2025-01-29 Thread via GitHub
elrob commented on PR #9756: URL: https://github.com/apache/iceberg/pull/9756#issuecomment-2623534131 This was auto-closed but maybe should be re-opened? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] Parquet: Fix Reader leak by removing useless copy [iceberg]

2025-01-29 Thread via GitHub
zizon commented on code in PR #12079: URL: https://github.com/apache/iceberg/pull/12079#discussion_r1935015216 ## parquet/src/main/java/org/apache/iceberg/parquet/ParquetIO.java: ## @@ -82,22 +75,10 @@ static OutputFile file(org.apache.iceberg.io.OutputFile file, Configuration

Re: [PR] Parquet: Fix Reader leak by removing useless copy [iceberg]

2025-01-29 Thread via GitHub
zizon commented on code in PR #12079: URL: https://github.com/apache/iceberg/pull/12079#discussion_r1935008359 ## parquet/src/test/java/org/apache/iceberg/parquet/TestParquet.java: ## @@ -219,6 +228,191 @@ public void testTwoLevelList() throws IOException { assertThat(recor

Re: [PR] Core: Fix cleanup of orphaned statistics files in dropTableData [iceberg]

2025-01-29 Thread via GitHub
ebyhr commented on code in PR #12132: URL: https://github.com/apache/iceberg/pull/12132#discussion_r1934928382 ## core/src/test/java/org/apache/iceberg/hadoop/TestCatalogUtilDropTable.java: ## @@ -129,6 +130,81 @@ public void dropTableDataDeletesExpectedFiles() throws IOExcepti

Re: [I] Variant Data Type Support [iceberg]

2025-01-29 Thread via GitHub
aihuaxu commented on issue #10392: URL: https://github.com/apache/iceberg/issues/10392#issuecomment-2623548847 @RussellSpitzer Can you help reopen this? It's automatically closed when one task #10831 (https://github.com/apache/iceberg/pull/10831) mentioned "fixed this issue" . Thanks. --

[I] Variant Data Type Support [iceberg]

2025-01-29 Thread via GitHub
sfc-gh-aixu opened a new issue, #10392: URL: https://github.com/apache/iceberg/issues/10392 ### Proposed Change We would like to propose to add Variant type to Iceberg data types. Variant data types allow for the efficient binary encoding of dynamic semi-structured data such a

  1   2   >