[I] Variant Data Type Support [iceberg]

sfc-gh-aixu opened a new issue, #10392: URL: https://github.com/apache/iceberg/issues/10392 ### Proposed Change We would like to propose to add Variant type to Iceberg data types. Variant data types allow for the efficient binary encoding of dynamic semi-structured data such a

Re: [I] Using tables created in Hive Catalog in Hadoop catalog [iceberg]

ajantha-bhat commented on issue #12125: URL: https://github.com/apache/iceberg/issues/12125#issuecomment-2623589630 reason for different naming conventions is based on the spec (based on how they operate). https://iceberg.apache.org/spec/#file-system-tables https://iceberg.apache.o

Re: [PR] API: Define RepairManifests action interface [iceberg]

wypoon commented on PR #10784: URL: https://github.com/apache/iceberg/pull/10784#issuecomment-2623572958 Another question I have about duplicate entries (in the manifests) is: do their presence make the table unreadable? Or is the table still readable and it is a valid state, although undes

Re: [I] Variant Data Type Support [iceberg]

aihuaxu commented on issue #10392: URL: https://github.com/apache/iceberg/issues/10392#issuecomment-2623548847 @RussellSpitzer Can you help reopen this? It's automatically closed when one task #10831 (https://github.com/apache/iceberg/pull/10831) mentioned "fixed this issue" . Thanks. --

Re: [PR] Parquet: Fix Reader leak by removing useless copy [iceberg]

zizon commented on code in PR #12079: URL: https://github.com/apache/iceberg/pull/12079#discussion_r1935015216 ## parquet/src/main/java/org/apache/iceberg/parquet/ParquetIO.java: ## @@ -82,22 +75,10 @@ static OutputFile file(org.apache.iceberg.io.OutputFile file, Configuration

Re: [PR] Update flink docs with alter column support [iceberg]

elrob commented on PR #9756: URL: https://github.com/apache/iceberg/pull/9756#issuecomment-2623534131 This was auto-closed but maybe should be re-opened? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] Core: Fix cleanup of orphaned statistics files in dropTableData [iceberg]

ebyhr commented on code in PR #12132: URL: https://github.com/apache/iceberg/pull/12132#discussion_r1934928382 ## core/src/test/java/org/apache/iceberg/hadoop/TestCatalogUtilDropTable.java: ## @@ -129,6 +130,81 @@ public void dropTableDataDeletesExpectedFiles() throws IOExcepti

Re: [PR] Parquet: Fix Reader leak by removing useless copy [iceberg]

zizon commented on code in PR #12079: URL: https://github.com/apache/iceberg/pull/12079#discussion_r1935008359 ## parquet/src/test/java/org/apache/iceberg/parquet/TestParquet.java: ## @@ -219,6 +228,191 @@ public void testTwoLevelList() throws IOException { assertThat(recor

Re: [PR] Make view metadata path configurable by `write.metadata.path` [iceberg]

tomtongue commented on PR #12017: URL: https://github.com/apache/iceberg/pull/12017#issuecomment-2623514157 Thanks so much for the quick review! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

Re: [PR] Core: Fix cleanup of orphaned statistics files in dropTableData [iceberg]

ebyhr commented on code in PR #12132: URL: https://github.com/apache/iceberg/pull/12132#discussion_r1934925978 ## core/src/test/java/org/apache/iceberg/hadoop/TestCatalogUtilDropTable.java: ## @@ -129,6 +130,81 @@ public void dropTableDataDeletesExpectedFiles() throws IOExcepti

Re: [PR] Core: Fix cleanup of orphaned statistics files in dropTableData [iceberg]

lliangyu-lin commented on code in PR #12132: URL: https://github.com/apache/iceberg/pull/12132#discussion_r1934986266 ## core/src/test/java/org/apache/iceberg/hadoop/TestCatalogUtilDropTable.java: ## @@ -129,6 +130,81 @@ public void dropTableDataDeletesExpectedFiles() throws IO

Re: [PR] API: Define RepairManifests action interface [iceberg]

amogh-jahagirdar commented on PR #10784: URL: https://github.com/apache/iceberg/pull/10784#issuecomment-2623396968 > For my edification, can someone please explain how duplicate file entries in manifests can arise? Can two entries for the same file occur in a single manifest? Can even two

Re: [PR] Spark: Support singular form of years, months, days, and hours functions [iceberg]

ebyhr commented on code in PR #12117: URL: https://github.com/apache/iceberg/pull/12117#discussion_r1934934393 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/sql/TestSparkDaysFunction.java: ## @@ -39,6 +39,9 @@ public void testDates() { assertThat(scalarSql("SEL

Re: [PR] Spark: Support singular form of years, months, days, and hours functions [iceberg]

ebyhr commented on code in PR #12117: URL: https://github.com/apache/iceberg/pull/12117#discussion_r1934934393 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/sql/TestSparkDaysFunction.java: ## @@ -39,6 +39,9 @@ public void testDates() { assertThat(scalarSql("SEL

Re: [PR] Core: Fix cleanup of orphaned statistics files in dropTableData [iceberg]

ebyhr commented on code in PR #12132: URL: https://github.com/apache/iceberg/pull/12132#discussion_r1934928382 ## core/src/test/java/org/apache/iceberg/hadoop/TestCatalogUtilDropTable.java: ## @@ -129,6 +130,81 @@ public void dropTableDataDeletesExpectedFiles() throws IOExcepti

Re: [PR] Core: Fix cleanup of orphaned statistics files in dropTableData [iceberg]

ebyhr commented on code in PR #12132: URL: https://github.com/apache/iceberg/pull/12132#discussion_r1934925978 ## core/src/test/java/org/apache/iceberg/hadoop/TestCatalogUtilDropTable.java: ## @@ -129,6 +130,81 @@ public void dropTableDataDeletesExpectedFiles() throws IOExcepti

Re: [PR] Core: Fix cleanup of orphaned statistics files in dropTableData [iceberg]

ebyhr commented on code in PR #12132: URL: https://github.com/apache/iceberg/pull/12132#discussion_r1934925978 ## core/src/test/java/org/apache/iceberg/hadoop/TestCatalogUtilDropTable.java: ## @@ -129,6 +130,81 @@ public void dropTableDataDeletesExpectedFiles() throws IOExcepti

Re: [PR] API: Define RepairManifests action interface [iceberg]

wypoon commented on PR #10784: URL: https://github.com/apache/iceberg/pull/10784#issuecomment-2623353054 For my edification, can someone please explain how duplicate file entries in manifests can arise? Can two entries for the same file occur in a single manifest? Can even two manifests be

Re: [PR] Iceberg/Comet integration POC [iceberg]

parthchandra commented on code in PR #9841: URL: https://github.com/apache/iceberg/pull/9841#discussion_r1934904760 ## .baseline/checkstyle/checkstyle-suppressions.xml: ## @@ -48,4 +48,7 @@ + Review Comment: Thank you! -- This is an automated message from t

Re: [PR] Iceberg/Comet integration POC [iceberg]

huaxingao commented on code in PR #9841: URL: https://github.com/apache/iceberg/pull/9841#discussion_r1934900306 ## spark/v3.4/spark/src/main/java/org/apache/iceberg/spark/data/vectorized/CometColumnReader.java: ## @@ -0,0 +1,146 @@ +/* + * Licensed to the Apache Software Founda

Re: [PR] Iceberg/Comet integration POC [iceberg]

parthchandra commented on code in PR #9841: URL: https://github.com/apache/iceberg/pull/9841#discussion_r1934889396 ## spark/v3.4/spark/src/main/java/org/apache/iceberg/spark/data/vectorized/CometColumnReader.java: ## @@ -0,0 +1,146 @@ +/* + * Licensed to the Apache Software Fou

Re: [PR] Iceberg/Comet integration POC [iceberg]

huaxingao commented on code in PR #9841: URL: https://github.com/apache/iceberg/pull/9841#discussion_r1934890980 ## .baseline/checkstyle/checkstyle-suppressions.xml: ## @@ -48,4 +48,7 @@ + Review Comment: I have created https://github.com/apache/datafusion-come

Re: [PR] Iceberg/Comet integration POC [iceberg]

huaxingao commented on code in PR #9841: URL: https://github.com/apache/iceberg/pull/9841#discussion_r1934882377 ## spark/v3.4/spark/src/main/java/org/apache/iceberg/spark/data/vectorized/CometColumnReader.java: ## @@ -0,0 +1,149 @@ +/* + * Licensed to the Apache Software Founda

Re: [PR] Iceberg/Comet integration POC [iceberg]

huaxingao commented on code in PR #9841: URL: https://github.com/apache/iceberg/pull/9841#discussion_r1934882503 ## spark/v3.4/spark/src/main/java/org/apache/iceberg/spark/data/vectorized/CometColumnReader.java: ## @@ -0,0 +1,149 @@ +/* + * Licensed to the Apache Software Founda

Re: [PR] Iceberg/Comet integration POC [iceberg]

huaxingao commented on code in PR #9841: URL: https://github.com/apache/iceberg/pull/9841#discussion_r1934881973 ## spark/v3.4/spark/src/main/java/org/apache/iceberg/spark/data/vectorized/CometColumnReader.java: ## @@ -0,0 +1,149 @@ +/* + * Licensed to the Apache Software Founda

Re: [PR] Iceberg/Comet integration POC [iceberg]

huaxingao commented on code in PR #9841: URL: https://github.com/apache/iceberg/pull/9841#discussion_r1934875937 ## spark/v3.4/spark/src/main/java/org/apache/iceberg/spark/data/vectorized/CometColumnReader.java: ## @@ -0,0 +1,149 @@ +/* + * Licensed to the Apache Software Founda

Re: [PR] Iceberg/Comet integration POC [iceberg]

huaxingao commented on code in PR #9841: URL: https://github.com/apache/iceberg/pull/9841#discussion_r1934875784 ## spark/v3.4/spark/src/main/java/org/apache/iceberg/spark/data/vectorized/CometVector.java: ## @@ -0,0 +1,117 @@ +/* + * Licensed to the Apache Software Foundation (

Re: [I] RESTTableOperations does not support table metadata swap like others TableOperations did [iceberg]

dramaticlly commented on issue #12134: URL: https://github.com/apache/iceberg/issues/12134#issuecomment-2623289459 @rdblue @nastra appreciate your insight on this! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

Re: [I] Please remove old releases [iceberg]

sebbASF commented on issue #2414: URL: https://github.com/apache/iceberg/issues/2414#issuecomment-2623282578 https://dist.apache.org/repos/dist/release/iceberg/apache-iceberg-1.7.0/ appears to be stale; please remove -- This is an automated message from the Apache Git Service. To respond

[I] RESTTableOperations does not support table metadata swap like others TableOperations did [iceberg]

dramaticlly opened a new issue, #12134: URL: https://github.com/apache/iceberg/issues/12134 ### Apache Iceberg version 1.7.1 (latest release) ### Query engine Spark ### Please describe the bug 🐞 Before migrate to REST catalog, we rely on following `TableOpe

Re: [I] The "Emitting watermarks" feature can't be used in flink sql? [iceberg]

github-actions[bot] commented on issue #10219: URL: https://github.com/apache/iceberg/issues/10219#issuecomment-2623222735 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [PR] Core: Make metrics reporter serializable (alternative impl) [iceberg]

github-actions[bot] commented on PR #8032: URL: https://github.com/apache/iceberg/pull/8032#issuecomment-2623222552 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] API,Core: Introduce metrics for data files by file format [iceberg]

github-actions[bot] commented on PR #5837: URL: https://github.com/apache/iceberg/pull/5837#issuecomment-2623222418 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] feat(catalog): Propagate ctx from catalog interface through call stack [iceberg-go]

zeroshade merged PR #276: URL: https://github.com/apache/iceberg-go/pull/276 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] JDBC: Escape table names when checking the existence [iceberg]

github-actions[bot] closed pull request #11863: JDBC: Escape table names when checking the existence URL: https://github.com/apache/iceberg/pull/11863 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Fix Hive FileIO closing with FileIOTracker [iceberg]

github-actions[bot] commented on PR #11782: URL: https://github.com/apache/iceberg/pull/11782#issuecomment-2623223116 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

Re: [I] Support Conditional Transaction Commits [iceberg]

github-actions[bot] commented on issue #6514: URL: https://github.com/apache/iceberg/issues/6514#issuecomment-2623222467 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [PR] Fix Hive FileIO closing with FileIOTracker [iceberg]

github-actions[bot] closed pull request #11782: Fix Hive FileIO closing with FileIOTracker URL: https://github.com/apache/iceberg/pull/11782 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

Re: [I] A move after a rename fails [iceberg]

github-actions[bot] commented on issue #10830: URL: https://github.com/apache/iceberg/issues/10830#issuecomment-2623222951 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [PR] Core: Avro writers use BlockingBinaryEncoder to enable array/map size calculations. [iceberg]

github-actions[bot] closed pull request #8625: Core: Avro writers use BlockingBinaryEncoder to enable array/map size calculations. URL: https://github.com/apache/iceberg/pull/8625 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

Re: [PR] API,Core: Support Conditional Commits [iceberg]

github-actions[bot] closed pull request #6513: API,Core: Support Conditional Commits URL: https://github.com/apache/iceberg/pull/6513 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

Re: [PR] Parquet: add variant type support [iceberg]

github-actions[bot] closed pull request #11653: Parquet: add variant type support URL: https://github.com/apache/iceberg/pull/11653 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

Re: [PR] Parquet: add variant type support [iceberg]

github-actions[bot] commented on PR #11653: URL: https://github.com/apache/iceberg/pull/11653#issuecomment-2623223077 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

Re: [PR] Spark: support rewrite on specified target branch [iceberg]

github-actions[bot] closed pull request #8797: Spark: support rewrite on specified target branch URL: https://github.com/apache/iceberg/pull/8797 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] API,Core: Support Conditional Commits [iceberg]

github-actions[bot] commented on PR #6513: URL: https://github.com/apache/iceberg/pull/6513#issuecomment-2623222450 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] JDBC: Escape table names when checking the existence [iceberg]

github-actions[bot] commented on PR #11863: URL: https://github.com/apache/iceberg/pull/11863#issuecomment-2623223185 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

Re: [PR] API: New API For sequential / streaming updates [iceberg]

github-actions[bot] commented on PR #9323: URL: https://github.com/apache/iceberg/pull/9323#issuecomment-2623222682 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Kafka Connect: Add config to route to tables using topic name [iceberg]

github-actions[bot] commented on PR #11313: URL: https://github.com/apache/iceberg/pull/11313#issuecomment-2623223008 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] API: New API For sequential / streaming updates [iceberg]

github-actions[bot] closed pull request #9323: API: New API For sequential / streaming updates URL: https://github.com/apache/iceberg/pull/9323 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

Re: [PR] Encryption integration and test [iceberg]

github-actions[bot] closed pull request #5544: Encryption integration and test URL: https://github.com/apache/iceberg/pull/5544 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] Core: Avro writers use BlockingBinaryEncoder to enable array/map size calculations. [iceberg]

github-actions[bot] commented on PR #8625: URL: https://github.com/apache/iceberg/pull/8625#issuecomment-2623222605 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [I] Running MERGE INTO with more than one WHEN condition fails if the number of columns in the target table is > 321 [iceberg]

github-actions[bot] commented on issue #10294: URL: https://github.com/apache/iceberg/issues/10294#issuecomment-2623222767 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [PR] Spark: support rewrite on specified target branch [iceberg]

github-actions[bot] commented on PR #8797: URL: https://github.com/apache/iceberg/pull/8797#issuecomment-2623222633 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Core: Make metrics reporter serializable (alternative impl) [iceberg]

github-actions[bot] closed pull request #8032: Core: Make metrics reporter serializable (alternative impl) URL: https://github.com/apache/iceberg/pull/8032 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] API,Core: Introduce metrics for data files by file format [iceberg]

github-actions[bot] closed pull request #5837: API,Core: Introduce metrics for data files by file format URL: https://github.com/apache/iceberg/pull/5837 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] Spec: add variant type [iceberg]

sfc-gh-aixu commented on PR #10831: URL: https://github.com/apache/iceberg/pull/10831#issuecomment-2623219808 > Merged, thanks everyone for your thoughtful feedback. Thanks @aihuaxu for the pr. Thanks to @rdblue , @emkornfield , @findepi , @XBaith , and @flyrain for your contributions revie

Re: [PR] Encryption integration and test [iceberg]

github-actions[bot] commented on PR #5544: URL: https://github.com/apache/iceberg/pull/5544#issuecomment-2623222370 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [I] Please remove old releases [iceberg]

github-actions[bot] commented on issue #2414: URL: https://github.com/apache/iceberg/issues/2414#issuecomment-262393 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [PR] Kafka Connect: Add kerberos authentication option [iceberg]

Dawnpool commented on PR #12119: URL: https://github.com/apache/iceberg/pull/12119#issuecomment-2623203039 Hi @bryanck , As mentioned in the original [PR](https://github.com/apache/iceberg/pull/10173#issuecomment-2418414725), Flink has its own security configuration for Kerberos authenti

Re: [PR] Spec: Update partition stats for V3 [iceberg]

amogh-jahagirdar commented on code in PR #12098: URL: https://github.com/apache/iceberg/pull/12098#discussion_r1934780048 ## format/spec.md: ## @@ -927,20 +927,21 @@ These rows must be sorted (in ascending manner with NULL FIRST) by `partition` f The schema of the partition

Re: [PR] Spec: Update partition stats for V3 [iceberg]

amogh-jahagirdar commented on code in PR #12098: URL: https://github.com/apache/iceberg/pull/12098#discussion_r1934775737 ## format/spec.md: ## @@ -927,20 +927,21 @@ These rows must be sorted (in ascending manner with NULL FIRST) by `partition` f The schema of the partition

Re: [PR] Spec: Update partition stats for V3 [iceberg]

amogh-jahagirdar commented on code in PR #12098: URL: https://github.com/apache/iceberg/pull/12098#discussion_r1934775737 ## format/spec.md: ## @@ -927,20 +927,21 @@ These rows must be sorted (in ascending manner with NULL FIRST) by `partition` f The schema of the partition

Re: [PR] Spec: Update partition stats for V3 [iceberg]

amogh-jahagirdar commented on code in PR #12098: URL: https://github.com/apache/iceberg/pull/12098#discussion_r1934775737 ## format/spec.md: ## @@ -927,20 +927,21 @@ These rows must be sorted (in ascending manner with NULL FIRST) by `partition` f The schema of the partition

Re: [PR] fix(catalog/sql): change isolation to LevelDefault [iceberg-go]

zeroshade merged PR #279: URL: https://github.com/apache/iceberg-go/pull/279 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] Spec: add variant type [iceberg]

RussellSpitzer commented on PR #10831: URL: https://github.com/apache/iceberg/pull/10831#issuecomment-2623055587 Merged, thanks everyone for your thoughtful feedback. Thanks @aihuaxu for the pr. Thanks to @rdblue , @emkornfield , @findepi , @XBaith , and @flyrain for your contributions revi

Re: [I] Variant Data Type Support [iceberg]

RussellSpitzer closed issue #10392: Variant Data Type Support URL: https://github.com/apache/iceberg/issues/10392 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe

Re: [PR] Spec: add variant type [iceberg]

RussellSpitzer merged PR #10831: URL: https://github.com/apache/iceberg/pull/10831 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [PR] #12081: "Add deleteFileThreshold parameter to SizeBasedDataRewriter, update logic, and include tests" [iceberg]

RussellSpitzer commented on code in PR #12133: URL: https://github.com/apache/iceberg/pull/12133#discussion_r1934753900 ## core/src/main/java/org/apache/iceberg/actions/SizeBasedDataRewriter.java: ## @@ -47,12 +47,19 @@ public abstract class SizeBasedDataRewriter extends SizeBa

Re: [PR] #12081: "Add deleteFileThreshold parameter to SizeBasedDataRewriter, update logic, and include tests" [iceberg]

RussellSpitzer commented on code in PR #12133: URL: https://github.com/apache/iceberg/pull/12133#discussion_r1934753417 ## core/src/main/java/org/apache/iceberg/actions/SizeBasedDataRewriter.java: ## @@ -47,12 +47,19 @@ public abstract class SizeBasedDataRewriter extends SizeBa

Re: [PR] #12081: "Add deleteFileThreshold parameter to SizeBasedDataRewriter, update logic, and include tests" [iceberg]

RussellSpitzer commented on code in PR #12133: URL: https://github.com/apache/iceberg/pull/12133#discussion_r1934752632 ## core/src/main/java/org/apache/iceberg/actions/SizeBasedDataRewriter.java: ## @@ -47,12 +47,19 @@ public abstract class SizeBasedDataRewriter extends SizeBa

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

emkornfield commented on PR #11039: URL: https://github.com/apache/iceberg/pull/11039#issuecomment-2623021449 > > @hesham-medhat @rdblue - could you pls give an update on this PR? It seems it would massively simplify the Iceberg table management in GCP for non-spark usecases. > > thanks!

Re: [PR] Add "clean" NOTICE/LICENSE in jar files [iceberg]

amogh-jahagirdar commented on code in PR #12127: URL: https://github.com/apache/iceberg/pull/12127#discussion_r1934737242 ## licenses/LICENSE: ## @@ -0,0 +1,202 @@ + + Apache License + Version 2.0, January 2004 +

Re: [PR] Spec: add variant type [iceberg]

aihuaxu commented on code in PR #10831: URL: https://github.com/apache/iceberg/pull/10831#discussion_r1934733942 ## format/spec.md: ## @@ -1208,6 +1223,7 @@ Lists must use the [3-level representation](https://github.com/apache/parquet-fo | **`struct`** | `group`

Re: [PR] Docs: Add rewrite-table-path in spark procedure [iceberg]

dramaticlly commented on PR #12115: URL: https://github.com/apache/iceberg/pull/12115#issuecomment-2622983101 thank you Russell for the detailed review! This is getting closer and I did a local check to ensure the info and warning section rendered as expected like in following screenshot. h

Re: [PR] Core: Relocate parquet to core [iceberg]

danielcweeks closed pull request #11716: Core: Relocate parquet to core URL: https://github.com/apache/iceberg/pull/11716 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To uns

Re: [PR] Core: Relocate parquet to core [iceberg]

danielcweeks commented on PR #11716: URL: https://github.com/apache/iceberg/pull/11716#issuecomment-2622982192 Closing in favor of #12060 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

Re: [PR] Add "clean" NOTICE/LICENSE in jar files [iceberg]

amogh-jahagirdar commented on code in PR #12127: URL: https://github.com/apache/iceberg/pull/12127#discussion_r1934728523 ## licenses/LICENSE: ## @@ -0,0 +1,202 @@ + + Apache License + Version 2.0, January 2004 +

Re: [PR] Add "clean" NOTICE/LICENSE in jar files [iceberg]

amogh-jahagirdar commented on code in PR #12127: URL: https://github.com/apache/iceberg/pull/12127#discussion_r1934724011 ## licenses/LICENSE: ## @@ -0,0 +1,202 @@ + + Apache License + Version 2.0, January 2004 +

Re: [PR] #12081: "Add deleteFileThreshold parameter to SizeBasedDataRewriter, update logic, and include tests" [iceberg]

jangalasriramd7 commented on PR #12133: URL: https://github.com/apache/iceberg/pull/12133#issuecomment-2622966642 I have changed the PR title. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] Spec: add variant type [iceberg]

emkornfield commented on code in PR #10831: URL: https://github.com/apache/iceberg/pull/10831#discussion_r1934723602 ## format/spec.md: ## @@ -1208,6 +1223,7 @@ Lists must use the [3-level representation](https://github.com/apache/parquet-fo | **`struct`** | `group`

Re: [PR] Fixing #12081: Fix SizeBasedDataRewriter constructors and related logic [iceberg]

RussellSpitzer commented on PR #12133: URL: https://github.com/apache/iceberg/pull/12133#issuecomment-2622950480 Title of this pr looks incorrect? This is a new feature and not a fix. The PR also would require some tests to prove the new parameter is working as expected. -- This is an a

[PR] fix(catalog/sql): change isolation to LevelDefault [iceberg-go]

zeroshade opened a new pull request, #279: URL: https://github.com/apache/iceberg-go/pull/279 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e

Re: [PR] feat(catalog): Propagate ctx from catalog interface through call stack [iceberg-go]

curtisr7 commented on code in PR #276: URL: https://github.com/apache/iceberg-go/pull/276#discussion_r1934689581 ## catalog/registry.go: ## @@ -64,13 +65,13 @@ var ( // Registrar is a factory for creating Catalog instances, used for registering to use // with LoadCatalog. ty

Re: [PR] feat(catalog): Initial implementation of sql catalog [iceberg-go]

kevinjqliu commented on code in PR #246: URL: https://github.com/apache/iceberg-go/pull/246#discussion_r1934685789 ## catalog/sql/sql.go: ## @@ -0,0 +1,721 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTI

Re: [PR] feat(catalog): Initial implementation of sql catalog [iceberg-go]

zeroshade merged PR #246: URL: https://github.com/apache/iceberg-go/pull/246 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] feat(catalog): Initial implementation of sql catalog [iceberg-go]

kevinjqliu commented on code in PR #246: URL: https://github.com/apache/iceberg-go/pull/246#discussion_r1934674526 ## catalog/sql/sql.go: ## @@ -0,0 +1,721 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTI

[PR] Fixing #12081: Fix SizeBasedDataRewriter constructors and related logic [iceberg]

jangalasriramd7 opened a new pull request, #12133: URL: https://github.com/apache/iceberg/pull/12133 This PR refines the SizeBasedDataRewriter class in the Apache Iceberg project, addressing issues related to the constructor initialization and logic for delete file threshold handling. It ma

Re: [PR] Iceberg/Comet integration POC [iceberg]

parthchandra commented on code in PR #9841: URL: https://github.com/apache/iceberg/pull/9841#discussion_r1934545952 ## spark/v3.4/spark/src/main/java/org/apache/iceberg/spark/data/vectorized/CometColumnReader.java: ## @@ -0,0 +1,149 @@ +/* + * Licensed to the Apache Software Fou

[PR] Core: Fix cleanup of orphaned statistics files in dropTableData [iceberg]

lliangyu-lin opened a new pull request, #12132: URL: https://github.com/apache/iceberg/pull/12132 ### Description Currently, Iceberg ```dropTableData()``` does not properly delete statistics files (```.stats```) that are replaced by newer statistics files. When ```updateStatistics()``` i

Re: [PR] feat(catalog): Initial implementation of sql catalog [iceberg-go]

zeroshade commented on code in PR #246: URL: https://github.com/apache/iceberg-go/pull/246#discussion_r1934573327 ## catalog/sql/sql.go: ## @@ -0,0 +1,721 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTIC

Re: [PR] feat(catalog): Initial implementation of sql catalog [iceberg-go]

zeroshade commented on code in PR #246: URL: https://github.com/apache/iceberg-go/pull/246#discussion_r1934571592 ## catalog/sql/sql.go: ## @@ -0,0 +1,721 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTIC

Re: [PR] Propagate ctx from catalog interface through call stack [iceberg-go]

zeroshade commented on code in PR #276: URL: https://github.com/apache/iceberg-go/pull/276#discussion_r1934565005 ## catalog/registry.go: ## @@ -64,13 +65,13 @@ var ( // Registrar is a factory for creating Catalog instances, used for registering to use // with LoadCatalog. t

Re: [PR] Docs: Add rewrite-table-path in spark procedure [iceberg]

RussellSpitzer commented on code in PR #12115: URL: https://github.com/apache/iceberg/pull/12115#discussion_r1934566388 ## docs/docs/spark-procedures.md: ## @@ -972,4 +972,91 @@ CALL catalog_name.system.compute_table_stats(table => 'my_table', snapshot_id => Collect statistics

Re: [PR] Add CheckTableExists method for rest.go [iceberg-go]

zeroshade merged PR #277: URL: https://github.com/apache/iceberg-go/pull/277 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] Add relevant NOTICE portions from ALv2 bundled dependencies [iceberg]

jbonofre commented on code in PR #12095: URL: https://github.com/apache/iceberg/pull/12095#discussion_r1934558928 ## spark/v3.5/spark-runtime/LICENSE: ## @@ -456,95 +380,49 @@ License text: -T

Re: [PR] Docs: Add rewrite-table-path in spark procedure [iceberg]

RussellSpitzer commented on code in PR #12115: URL: https://github.com/apache/iceberg/pull/12115#discussion_r1934550850 ## docs/docs/spark-procedures.md: ## @@ -972,4 +972,91 @@ CALL catalog_name.system.compute_table_stats(table => 'my_table', snapshot_id => Collect statistics

Re: [PR] Docs: Add rewrite-table-path in spark procedure [iceberg]

RussellSpitzer commented on code in PR #12115: URL: https://github.com/apache/iceberg/pull/12115#discussion_r1934549172 ## docs/docs/spark-procedures.md: ## @@ -972,4 +972,91 @@ CALL catalog_name.system.compute_table_stats(table => 'my_table', snapshot_id => Collect statistics

Re: [PR] Docs: Add rewrite-table-path in spark procedure [iceberg]

RussellSpitzer commented on code in PR #12115: URL: https://github.com/apache/iceberg/pull/12115#discussion_r1934547493 ## docs/docs/spark-procedures.md: ## @@ -972,4 +972,91 @@ CALL catalog_name.system.compute_table_stats(table => 'my_table', snapshot_id => Collect statistics

Re: [PR] Docs: Add rewrite-table-path in spark procedure [iceberg]

RussellSpitzer commented on code in PR #12115: URL: https://github.com/apache/iceberg/pull/12115#discussion_r1934547087 ## docs/docs/spark-procedures.md: ## @@ -972,4 +972,91 @@ CALL catalog_name.system.compute_table_stats(table => 'my_table', snapshot_id => Collect statistics

Re: [PR] Spec: Fix minor typo in `_last_updated_sequence_number` docs [iceberg]

RussellSpitzer commented on PR #12128: URL: https://github.com/apache/iceberg/pull/12128#issuecomment-2622782683 Thanks for the cleanup! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

  1   2   >