Re: [PR] feat: add file reader interface [iceberg-cpp]

wgtmac commented on PR #88: URL: https://github.com/apache/iceberg-cpp/pull/88#issuecomment-2831908914 @lidavidm @zhjwpku Could you help review this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [I] [Bug] Error HTTP status handling [iceberg-rust]

Xuanwo closed issue #1247: [Bug] Error HTTP status handling URL: https://github.com/apache/iceberg-rust/issues/1247 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscri

Re: [I] [Bug] Error HTTP status handling [iceberg-rust]

Xuanwo closed issue #1247: [Bug] Error HTTP status handling URL: https://github.com/apache/iceberg-rust/issues/1247 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscri

Re: [PR] [catalog] Fix namespace creation error status [iceberg-rust]

Xuanwo merged PR #1248: URL: https://github.com/apache/iceberg-rust/pull/1248 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [PR] Core: Implement source-ids to deal with multi arguments transforms [iceberg]

jbonofre commented on code in PR #12897: URL: https://github.com/apache/iceberg/pull/12897#discussion_r2061155133 ## core/src/main/java/org/apache/iceberg/PartitionSpecParser.java: ## @@ -132,15 +147,29 @@ private static void buildFromJsonFields(UnboundPartitionSpec.Builder bui

Re: [PR] Core: Implement source-ids to deal with multi arguments transforms [iceberg]

jbonofre commented on code in PR #12897: URL: https://github.com/apache/iceberg/pull/12897#discussion_r2061154925 ## api/src/main/java/org/apache/iceberg/PartitionField.java: ## @@ -19,26 +19,49 @@ package org.apache.iceberg; import java.io.Serializable; +import java.util.Li

Re: [PR] Core: Implement source-ids to deal with multi arguments transforms [iceberg]

jbonofre commented on code in PR #12897: URL: https://github.com/apache/iceberg/pull/12897#discussion_r2061143136 ## core/src/test/java/org/apache/iceberg/TestPartitionSpecParser.java: ## @@ -95,6 +96,56 @@ public void testFromJsonWithFieldId() { assertThat(spec.fields().ge

Re: [PR] Core: Implement source-ids to deal with multi arguments transforms [iceberg]

jbonofre commented on code in PR #12897: URL: https://github.com/apache/iceberg/pull/12897#discussion_r2061141617 ## api/src/main/java/org/apache/iceberg/PartitionSpec.java: ## @@ -601,17 +601,26 @@ public Builder alwaysNull(String sourceName) { // add a partition field w

Re: [PR] Core: Implement source-ids to deal with multi arguments transforms [iceberg]

jbonofre commented on code in PR #12897: URL: https://github.com/apache/iceberg/pull/12897#discussion_r2061143136 ## core/src/test/java/org/apache/iceberg/TestPartitionSpecParser.java: ## @@ -95,6 +96,56 @@ public void testFromJsonWithFieldId() { assertThat(spec.fields().ge

Re: [PR] Core: Implement source-ids to deal with multi arguments transforms [iceberg]

jbonofre commented on code in PR #12897: URL: https://github.com/apache/iceberg/pull/12897#discussion_r2061143410 ## core/src/main/java/org/apache/iceberg/PartitionSpecParser.java: ## @@ -132,15 +147,29 @@ private static void buildFromJsonFields(UnboundPartitionSpec.Builder bui

Re: [PR] Core: Implement source-ids to deal with multi arguments transforms [iceberg]

jbonofre commented on code in PR #12897: URL: https://github.com/apache/iceberg/pull/12897#discussion_r2061143136 ## core/src/test/java/org/apache/iceberg/TestPartitionSpecParser.java: ## @@ -95,6 +96,56 @@ public void testFromJsonWithFieldId() { assertThat(spec.fields().ge

Re: [PR] Spark 3.5: Update MERGE and UPDATE for row lineage [iceberg]

amogh-jahagirdar commented on code in PR #12736: URL: https://github.com/apache/iceberg/pull/12736#discussion_r2061132912 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestRowLevelOperationsWithLineage.java: ## @@ -0,0 +1,492 @@ +/* + * License

Re: [PR] Spark 3.5: Update MERGE and UPDATE for row lineage [iceberg]

amogh-jahagirdar commented on code in PR #12736: URL: https://github.com/apache/iceberg/pull/12736#discussion_r2061132781 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestRowLevelOperationsWithLineage.java: ## @@ -0,0 +1,492 @@ +/* + * License

Re: [PR] Parquet variant array write [iceberg]

rdblue commented on code in PR #12847: URL: https://github.com/apache/iceberg/pull/12847#discussion_r2061126632 ## parquet/src/main/java/org/apache/iceberg/parquet/ParquetVariantWriters.java: ## @@ -98,6 +99,26 @@ static ParquetValueWriter objects( builder.build());

Re: [PR] Parquet variant array write [iceberg]

rdblue commented on code in PR #12847: URL: https://github.com/apache/iceberg/pull/12847#discussion_r2061126510 ## parquet/src/main/java/org/apache/iceberg/parquet/ParquetVariantWriters.java: ## @@ -360,6 +381,92 @@ public void setColumnStore(ColumnWriteStore columnStore) {

Re: [PR] Use Snapshot's statistics file in SparkScan [iceberg]

github-actions[bot] commented on PR #11040: URL: https://github.com/apache/iceberg/pull/11040#issuecomment-2831647937 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] Parquet variant array write [iceberg]

rdblue commented on code in PR #12847: URL: https://github.com/apache/iceberg/pull/12847#discussion_r2061123843 ## core/src/main/java/org/apache/iceberg/variants/ValueArray.java: ## @@ -127,4 +127,9 @@ private int writeTo(ByteBuffer buffer, int offset) { return (dataOffse

Re: [PR] Parquet variant array write [iceberg]

rdblue commented on code in PR #12847: URL: https://github.com/apache/iceberg/pull/12847#discussion_r2061123548 ## api/src/main/java/org/apache/iceberg/variants/VariantArray.java: ## @@ -20,6 +20,25 @@ /** An variant array value. */ public interface VariantArray extends Vari

Re: [PR] Spark 3.5: Update MERGE and UPDATE for row lineage [iceberg]

rdblue commented on code in PR #12736: URL: https://github.com/apache/iceberg/pull/12736#discussion_r2061122782 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestRowLevelOperationsWithLineage.java: ## @@ -0,0 +1,492 @@ +/* + * Licensed to the A

Re: [PR] Spark 3.5 row lineage [iceberg]

rdblue commented on code in PR #12736: URL: https://github.com/apache/iceberg/pull/12736#discussion_r2061122165 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestRowLevelOperationsWithLineage.java: ## @@ -0,0 +1,492 @@ +/* + * Licensed to the A

Re: [PR] Core: Fix numeric overflow of timestamp nano literal [iceberg]

github-actions[bot] commented on PR #11775: URL: https://github.com/apache/iceberg/pull/11775#issuecomment-2831648013 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [I] Spark procedure to compute partition stats. [iceberg]

github-actions[bot] commented on issue #10106: URL: https://github.com/apache/iceberg/issues/10106#issuecomment-2831647884 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [I] Flink: Add RowConverter for Iceberg Source [iceberg]

github-actions[bot] commented on issue #11312: URL: https://github.com/apache/iceberg/issues/11312#issuecomment-2831647959 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache

Re: [PR] Spark-3.5: Add spark action to compute partition stats [iceberg]

github-actions[bot] commented on PR #12450: URL: https://github.com/apache/iceberg/pull/12450#issuecomment-2831648087 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

Re: [I] Flink: Add RowConverter for Iceberg Source [iceberg]

github-actions[bot] closed issue #11312: Flink: Add RowConverter for Iceberg Source URL: https://github.com/apache/iceberg/issues/11312 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [PR] OpenAPI: Use more clear language in recommending error responses [iceberg]

github-actions[bot] commented on PR #12376: URL: https://github.com/apache/iceberg/pull/12376#issuecomment-2831648069 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] Spark-3.5: Add spark action to compute partition stats [iceberg]

github-actions[bot] closed pull request #12450: Spark-3.5: Add spark action to compute partition stats URL: https://github.com/apache/iceberg/pull/12450 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] Parquet: Fix column pruning for deeply nested fields [iceberg]

github-actions[bot] commented on PR #12634: URL: https://github.com/apache/iceberg/pull/12634#issuecomment-2831648160 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] Spark-3.5: Add procedure to compute partition stats [iceberg]

github-actions[bot] commented on PR #12451: URL: https://github.com/apache/iceberg/pull/12451#issuecomment-2831648106 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] [Spark]Add max files rewrite option for RewriteAction [iceberg]

coderfender commented on code in PR #12824: URL: https://github.com/apache/iceberg/pull/12824#discussion_r2061052784 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/RewriteDataFilesSparkAction.java: ## @@ -407,15 +409,37 @@ private Builder doExecuteWithPartial

Re: [PR] [Spark]Add max files rewrite option for RewriteAction [iceberg]

coderfender commented on code in PR #12824: URL: https://github.com/apache/iceberg/pull/12824#discussion_r2061051874 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/actions/TestRewriteDataFilesAction.java: ## @@ -2011,6 +2011,40 @@ public void testZOrderRewriteWithSpe

Re: [PR] Spark 3.5 row lineage [iceberg]

amogh-jahagirdar commented on code in PR #12736: URL: https://github.com/apache/iceberg/pull/12736#discussion_r2061029326 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestRowLevelOperationsWithLineage.java: ## @@ -0,0 +1,490 @@ +/* + * License

Re: [PR] Spark 3.5 row lineage [iceberg]

amogh-jahagirdar commented on code in PR #12736: URL: https://github.com/apache/iceberg/pull/12736#discussion_r2061028456 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestRowLevelOperationsWithLineage.java: ## @@ -0,0 +1,490 @@ +/* + * License

Re: [PR] Spark 3.5 row lineage [iceberg]

amogh-jahagirdar commented on code in PR #12736: URL: https://github.com/apache/iceberg/pull/12736#discussion_r2061003382 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestRowLevelOperationsWithLineage.java: ## @@ -0,0 +1,490 @@ +/* + * License

Re: [PR] Parquet: Add variant array reader in Parquet [iceberg]

rdblue commented on PR #12512: URL: https://github.com/apache/iceberg/pull/12512#issuecomment-2831591018 Merged. Thanks, @aihuaxu! Nice work. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

Re: [PR] Parquet: Add variant array reader in Parquet [iceberg]

rdblue merged PR #12512: URL: https://github.com/apache/iceberg/pull/12512 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Parquet: Add variant array reader in Parquet [iceberg]

aihuaxu commented on PR #12512: URL: https://github.com/apache/iceberg/pull/12512#issuecomment-2831585772 > I'm re-running the failed test run because I think it may have just been a flaky test. Thanks. Yeah. This is flaky test. It was passing locally and seems sometimes it may take

Re: [PR] Parquet: Add variant array reader in Parquet [iceberg]

aihuaxu commented on code in PR #12512: URL: https://github.com/apache/iceberg/pull/12512#discussion_r2060868714 ## parquet/src/test/java/org/apache/iceberg/parquet/TestVariantReaders.java: ## @@ -885,6 +897,460 @@ public void testMixedRecords() throws IOException { Variant

Re: [PR] Parquet: Add variant array reader in Parquet [iceberg]

aihuaxu commented on code in PR #12512: URL: https://github.com/apache/iceberg/pull/12512#discussion_r2060868714 ## parquet/src/test/java/org/apache/iceberg/parquet/TestVariantReaders.java: ## @@ -885,6 +897,460 @@ public void testMixedRecords() throws IOException { Variant

Re: [PR] Parquet: Add variant array reader in Parquet [iceberg]

aihuaxu commented on code in PR #12512: URL: https://github.com/apache/iceberg/pull/12512#discussion_r2060868714 ## parquet/src/test/java/org/apache/iceberg/parquet/TestVariantReaders.java: ## @@ -885,6 +897,460 @@ public void testMixedRecords() throws IOException { Variant

Re: [PR] Parquet: Add variant array reader in Parquet [iceberg]

rdblue commented on PR #12512: URL: https://github.com/apache/iceberg/pull/12512#issuecomment-2831574843 I'm re-running the failed test run because I think it may have just been a flaky test. -- This is an automated message from the Apache Git Service. To respond to the message, please lo

Re: [PR] Parquet: Add variant array reader in Parquet [iceberg]

rdblue commented on code in PR #12512: URL: https://github.com/apache/iceberg/pull/12512#discussion_r2060986753 ## parquet/src/test/java/org/apache/iceberg/parquet/TestVariantReaders.java: ## @@ -1327,7 +1327,7 @@ public void testArrayWithElementNullValueAndNullTypedValue() thr

Re: [PR] Spark: Support rewrite file with z-order for nested Struct type [iceberg]

geruh commented on PR #9818: URL: https://github.com/apache/iceberg/pull/9818#issuecomment-2831571036 Awesome, LGTM! cc: @amogh-jahagirdar @kevinjqliu -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

Re: [PR] Spark: Support rewrite file with z-order for nested Struct type [iceberg]

geruh commented on code in PR #9818: URL: https://github.com/apache/iceberg/pull/9818#discussion_r2060981381 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/SparkZOrderDataRewriter.java: ## @@ -135,15 +140,52 @@ private Column zValue(Dataset df) { SparkZO

Re: [PR] Spark 3.5 row lineage [iceberg]

rdblue commented on code in PR #12736: URL: https://github.com/apache/iceberg/pull/12736#discussion_r2060979914 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestRowLevelOperationsWithLineage.java: ## @@ -0,0 +1,490 @@ +/* + * Licensed to the A

Re: [PR] Spark 3.5 row lineage [iceberg]

rdblue commented on code in PR #12736: URL: https://github.com/apache/iceberg/pull/12736#discussion_r2060975885 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestRowLevelOperationsWithLineage.java: ## @@ -0,0 +1,490 @@ +/* + * Licensed to the A

Re: [PR] Spark 3.5 row lineage [iceberg]

rdblue commented on code in PR #12736: URL: https://github.com/apache/iceberg/pull/12736#discussion_r2060974460 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestRowLevelOperationsWithLineage.java: ## @@ -0,0 +1,490 @@ +/* + * Licensed to the A

Re: [PR] Flink: Backport add StreamingStartingStrategy.INCREMENTAL_FROM_LATEST_SNAPSHOT_EXCLUSIVE [iceberg]

morhidi commented on PR #12899: URL: https://github.com/apache/iceberg/pull/12899#issuecomment-2831551244 cc @rodmeneses @mxm -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

Re: [PR] Spark 3.5 row lineage [iceberg]

rdblue commented on code in PR #12736: URL: https://github.com/apache/iceberg/pull/12736#discussion_r2060973380 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestRowLevelOperationsWithLineage.java: ## @@ -0,0 +1,490 @@ +/* + * Licensed to the A

Re: [PR] Spark 3.5 row lineage [iceberg]

rdblue commented on code in PR #12736: URL: https://github.com/apache/iceberg/pull/12736#discussion_r2060972410 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestRowLevelOperationsWithLineage.java: ## @@ -0,0 +1,490 @@ +/* + * Licensed to the A

Re: [PR] Spark 3.5 row lineage [iceberg]

rdblue commented on code in PR #12736: URL: https://github.com/apache/iceberg/pull/12736#discussion_r2060969325 ## spark/v3.5/spark-extensions/src/main/scala/org/apache/spark/sql/catalyst/analysis/RewriteMergeIntoTableForRowLineage.scala: ## @@ -0,0 +1,75 @@ +/* + * Licensed to

Re: [PR] Fallback for upsert when arrow cannot compare source rows with target rows [iceberg-python]

kevinjqliu commented on PR #1878: URL: https://github.com/apache/iceberg-python/pull/1878#issuecomment-2830724552 Leaving this snippet from my stash here for when pyarrow supports comparison of complex types. This is a pyarrow compute only implementation ``` # filters the s

Re: [PR] Spark 3.5 row lineage [iceberg]

rdblue commented on code in PR #12736: URL: https://github.com/apache/iceberg/pull/12736#discussion_r2060952937 ## core/src/main/java/org/apache/iceberg/MetadataColumns.java: ## @@ -152,4 +154,19 @@ public static boolean isMetadataColumn(int id) { public static boolean nonMet

Re: [PR] Spark 3.5 row lineage [iceberg]

rdblue commented on code in PR #12736: URL: https://github.com/apache/iceberg/pull/12736#discussion_r2060856557 ## core/src/main/java/org/apache/iceberg/MetadataColumns.java: ## @@ -152,4 +154,19 @@ public static boolean isMetadataColumn(int id) { public static boolean nonMet

Re: [I] Support native Pydantic schemas [iceberg-python]

potatochipcoconut commented on issue #1934: URL: https://github.com/apache/iceberg-python/issues/1934#issuecomment-2831447185 @choucavalier started playing with your lib and seems like generally what I was looking for, ran into issues around UUID (opened issues there and duckdb). Loc

Re: [PR] Spark 3.5 row lineage [iceberg]

amogh-jahagirdar commented on code in PR #12736: URL: https://github.com/apache/iceberg/pull/12736#discussion_r2060895675 ## core/src/main/java/org/apache/iceberg/MetadataColumns.java: ## @@ -152,4 +154,19 @@ public static boolean isMetadataColumn(int id) { public static bool

Re: [PR] Parquet: Add variant array reader in Parquet [iceberg]

aihuaxu commented on code in PR #12512: URL: https://github.com/apache/iceberg/pull/12512#discussion_r2060868714 ## parquet/src/test/java/org/apache/iceberg/parquet/TestVariantReaders.java: ## @@ -885,6 +897,460 @@ public void testMixedRecords() throws IOException { Variant

Re: [PR] Spark 3.5 row lineage [iceberg]

amogh-jahagirdar commented on code in PR #12736: URL: https://github.com/apache/iceberg/pull/12736#discussion_r2060873913 ## core/src/main/java/org/apache/iceberg/MetadataColumns.java: ## @@ -152,4 +154,19 @@ public static boolean isMetadataColumn(int id) { public static bool

Re: [PR] Spark 3.5 row lineage [iceberg]

amogh-jahagirdar commented on code in PR #12736: URL: https://github.com/apache/iceberg/pull/12736#discussion_r2060887711 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/SparkPositionDeltaWriteBuilder.java: ## @@ -104,6 +109,16 @@ private void validateMetadataS

Re: [PR] fix(catalog/glue): Fix glue integration test failures [iceberg-go]

lliangyu-lin commented on code in PR #400: URL: https://github.com/apache/iceberg-go/pull/400#discussion_r2060881839 ## catalog/glue/glue_test.go: ## @@ -791,7 +791,7 @@ func TestGlueListTablesIntegration(t *testing.T) { } assert.NoError(lastErr) - assert

Re: [PR] Spark 3.5 row lineage [iceberg]

amogh-jahagirdar commented on code in PR #12736: URL: https://github.com/apache/iceberg/pull/12736#discussion_r2060881490 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/SparkCopyOnWriteOperation.java: ## @@ -95,13 +98,18 @@ public WriteBuilder newWriteBuilder(

Re: [PR] Spark 3.5 row lineage [iceberg]

amogh-jahagirdar commented on code in PR #12736: URL: https://github.com/apache/iceberg/pull/12736#discussion_r2060879933 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/SparkCopyOnWriteOperation.java: ## @@ -95,13 +98,18 @@ public WriteBuilder newWriteBuilder(

Re: [PR] Spark 3.5 row lineage [iceberg]

amogh-jahagirdar commented on code in PR #12736: URL: https://github.com/apache/iceberg/pull/12736#discussion_r2060870333 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/SparkTable.java: ## @@ -258,23 +256,16 @@ public Set capabilities() { @Override publi

Re: [PR] Spark 3.5 row lineage [iceberg]

rdblue commented on code in PR #12736: URL: https://github.com/apache/iceberg/pull/12736#discussion_r2060855402 ## core/src/main/java/org/apache/iceberg/MetadataColumns.java: ## @@ -152,4 +154,19 @@ public static boolean isMetadataColumn(int id) { public static boolean nonMet

Re: [PR] Catalog: Add BigQuery Metastore Catalog Support [iceberg]

talatuyarer commented on code in PR #12808: URL: https://github.com/apache/iceberg/pull/12808#discussion_r2060849882 ## bigquery/src/test/java/org/apache/iceberg/gcp/bigquery/BigQueryMetastoreTestUtils.java: ## @@ -0,0 +1,87 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] Parquet: Add variant array reader in Parquet [iceberg]

rdblue commented on code in PR #12512: URL: https://github.com/apache/iceberg/pull/12512#discussion_r2060849017 ## parquet/src/main/java/org/apache/iceberg/parquet/ParquetVariantReaders.java: ## @@ -332,6 +341,57 @@ public void setPageSource(PageReadStore pageStore) { }

Re: [PR] Parquet: Add variant array reader in Parquet [iceberg]

rdblue commented on code in PR #12512: URL: https://github.com/apache/iceberg/pull/12512#discussion_r2060845248 ## parquet/src/test/java/org/apache/iceberg/parquet/TestVariantReaders.java: ## @@ -885,6 +897,460 @@ public void testMixedRecords() throws IOException { VariantT

Re: [PR] Parquet: Add variant array reader in Parquet [iceberg]

rdblue commented on code in PR #12512: URL: https://github.com/apache/iceberg/pull/12512#discussion_r2060840753 ## parquet/src/test/java/org/apache/iceberg/parquet/TestVariantReaders.java: ## @@ -879,6 +896,405 @@ public void testMixedRecords() throws IOException { VariantT

Re: [I] remove redundant ci tests [iceberg-python]

FerminPaez commented on issue #1942: URL: https://github.com/apache/iceberg-python/issues/1942#issuecomment-2831337693 Hi! How can I help you? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] Add test to verify that no avro schema cache is used with manifest files [iceberg-go]

zeroshade merged PR #402: URL: https://github.com/apache/iceberg-go/pull/402 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] feat: implement initial MemoryCatalog functionality with namespace and table support [iceberg-cpp]

Fokko commented on code in PR #80: URL: https://github.com/apache/iceberg-cpp/pull/80#discussion_r2060705584 ## src/iceberg/catalog/memory_catalog.h: ## @@ -0,0 +1,204 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreement

Re: [PR] Core: Implement source-ids to deal with multi arguments transforms [iceberg]

jbonofre commented on PR #12897: URL: https://github.com/apache/iceberg/pull/12897#issuecomment-2830855584 Thanks @RussellSpitzer ! Let me fix the util methods and add tests. Thanks again ! -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [I] Manifest entry partition filtering on struct fields [iceberg-go]

GabrielM98 commented on issue #404: URL: https://github.com/apache/iceberg-go/issues/404#issuecomment-2830909201 Thanks for the quick response @zeroshade! > can you share the full schema The schema looks like so... ```json { "type": "struct", "schema-id": 0,

Re: [PR] [SPARK] Fix add_files type conversion exception and incorrect partition value when handling null partitions [iceberg]

lliangyu-lin commented on PR #12886: URL: https://github.com/apache/iceberg/pull/12886#issuecomment-2830873779 Usually what I see is, make sure the issue is fixed and approved in `v3.5`. Then port the changes to `v3.4` and earlier versions in the same PR or separate PRs. In this way, you do

[I] table.ParseMetadataBytes fails if table has no columns [iceberg-go]

mikhasd opened a new issue, #405: URL: https://github.com/apache/iceberg-go/issues/405 ### Apache Iceberg version main (development) ### Please describe the bug 🐞 `table.ParseMetadataBytes` fails with error `missing last-column-id` if table has no columns. --

Re: [PR] Core: Implement source-ids to deal with multi arguments transforms [iceberg]

RussellSpitzer commented on code in PR #12897: URL: https://github.com/apache/iceberg/pull/12897#discussion_r2060514306 ## core/src/main/java/org/apache/iceberg/PartitionSpecParser.java: ## @@ -132,15 +147,29 @@ private static void buildFromJsonFields(UnboundPartitionSpec.Build

Re: [PR] Core: Implement source-ids to deal with multi arguments transforms [iceberg]

jbonofre commented on code in PR #12897: URL: https://github.com/apache/iceberg/pull/12897#discussion_r2060507585 ## api/src/main/java/org/apache/iceberg/PartitionField.java: ## @@ -19,26 +19,49 @@ package org.apache.iceberg; import java.io.Serializable; +import java.util.Li

Re: [PR] Core: Implement source-ids to deal with multi arguments transforms [iceberg]

RussellSpitzer commented on code in PR #12897: URL: https://github.com/apache/iceberg/pull/12897#discussion_r2060514306 ## core/src/main/java/org/apache/iceberg/PartitionSpecParser.java: ## @@ -132,15 +147,29 @@ private static void buildFromJsonFields(UnboundPartitionSpec.Build

Re: [PR] Core: Implement source-ids to deal with multi arguments transforms [iceberg]

RussellSpitzer commented on code in PR #12897: URL: https://github.com/apache/iceberg/pull/12897#discussion_r2060518830 ## core/src/test/java/org/apache/iceberg/TestPartitionSpecParser.java: ## @@ -95,6 +96,56 @@ public void testFromJsonWithFieldId() { assertThat(spec.field

Re: [PR] Core: Implement source-ids to deal with multi arguments transforms [iceberg]

RussellSpitzer commented on code in PR #12897: URL: https://github.com/apache/iceberg/pull/12897#discussion_r2060457399 ## api/src/main/java/org/apache/iceberg/PartitionField.java: ## @@ -19,26 +19,49 @@ package org.apache.iceberg; import java.io.Serializable; +import java.u

Re: [PR] Core: Implement source-ids to deal with multi arguments transforms [iceberg]

RussellSpitzer commented on code in PR #12897: URL: https://github.com/apache/iceberg/pull/12897#discussion_r2060511995 ## api/src/main/java/org/apache/iceberg/UnboundPartitionSpec.java: ## @@ -118,7 +128,12 @@ public String transformAsString() { } public int sourceI

Re: [PR] Core: Implement source-ids to deal with multi arguments transforms [iceberg]

jbonofre commented on code in PR #12897: URL: https://github.com/apache/iceberg/pull/12897#discussion_r2060508764 ## api/src/main/java/org/apache/iceberg/UnboundPartitionSpec.java: ## @@ -54,17 +55,17 @@ private PartitionSpec.Builder copyToBuilder(Schema schema) { Partition

Re: [PR] Core: Implement source-ids to deal with multi arguments transforms [iceberg]

RussellSpitzer commented on code in PR #12897: URL: https://github.com/apache/iceberg/pull/12897#discussion_r2060504659 ## api/src/main/java/org/apache/iceberg/UnboundPartitionSpec.java: ## @@ -54,17 +55,17 @@ private PartitionSpec.Builder copyToBuilder(Schema schema) { Par

Re: [PR] Core: Implement source-ids to deal with multi arguments transforms [iceberg]

RussellSpitzer commented on code in PR #12897: URL: https://github.com/apache/iceberg/pull/12897#discussion_r2060493653 ## api/src/main/java/org/apache/iceberg/PartitionSpec.java: ## @@ -601,17 +601,26 @@ public Builder alwaysNull(String sourceName) { // add a partition f

Re: [PR] fix(catalog/glue): Fix glue integration test failures [iceberg-go]

zeroshade commented on code in PR #400: URL: https://github.com/apache/iceberg-go/pull/400#discussion_r2060481292 ## catalog/glue/glue_test.go: ## @@ -791,7 +791,7 @@ func TestGlueListTablesIntegration(t *testing.T) { } assert.NoError(lastErr) - assert.Eq

Re: [PR] Core: Implement source-ids to deal with multi arguments transforms [iceberg]

RussellSpitzer commented on code in PR #12897: URL: https://github.com/apache/iceberg/pull/12897#discussion_r2060447565 ## .palantir/revapi.yml: ## @@ -1177,6 +1177,10 @@ acceptedBreaks: old: "class org.apache.iceberg.Metrics" new: "class org.apache.iceberg.Metrics

Re: [PR] Core: Implement source-ids to deal with multi arguments transforms [iceberg]

jbonofre commented on code in PR #12897: URL: https://github.com/apache/iceberg/pull/12897#discussion_r2060454170 ## api/src/main/java/org/apache/iceberg/PartitionField.java: ## @@ -19,26 +19,49 @@ package org.apache.iceberg; import java.io.Serializable; +import java.util.Li

Re: [PR] Core: Implement source-ids to deal with multi arguments transforms [iceberg]

RussellSpitzer commented on code in PR #12897: URL: https://github.com/apache/iceberg/pull/12897#discussion_r2060449197 ## api/src/main/java/org/apache/iceberg/PartitionField.java: ## @@ -19,26 +19,49 @@ package org.apache.iceberg; import java.io.Serializable; +import java.u

Re: [PR] Core: Implement source-ids to deal with multi arguments transforms [iceberg]

jbonofre commented on code in PR #12897: URL: https://github.com/apache/iceberg/pull/12897#discussion_r2060368165 ## .palantir/revapi.yml: ## @@ -1177,6 +1177,10 @@ acceptedBreaks: old: "class org.apache.iceberg.Metrics" new: "class org.apache.iceberg.Metrics"

Re: [PR] Core: Implement source-ids to deal with multi arguments transforms [iceberg]

jbonofre commented on code in PR #12897: URL: https://github.com/apache/iceberg/pull/12897#discussion_r2060368165 ## .palantir/revapi.yml: ## @@ -1177,6 +1177,10 @@ acceptedBreaks: old: "class org.apache.iceberg.Metrics" new: "class org.apache.iceberg.Metrics"

Re: [PR] Core: Implement source-ids to deal with multi arguments transforms [iceberg]

ajantha-bhat commented on code in PR #12897: URL: https://github.com/apache/iceberg/pull/12897#discussion_r2060354754 ## .palantir/revapi.yml: ## @@ -1177,6 +1177,10 @@ acceptedBreaks: old: "class org.apache.iceberg.Metrics" new: "class org.apache.iceberg.Metrics"

Re: [PR] Bump to 0.9.1 [iceberg-python]

Fokko merged PR #1950: URL: https://github.com/apache/iceberg-python/pull/1950 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [PR] Docs: Incorrect property in CREATE CATALOG for Flink [iceberg]

ebyhr commented on code in PR #12894: URL: https://github.com/apache/iceberg/pull/12894#discussion_r2059691717 ## docs/docs/aws.md: ## @@ -84,7 +84,7 @@ With those dependencies, you can create a Flink catalog like the following: CREATE CATALOG my_catalog WITH ( 'type'='iceb

Re: [PR] Core: Implement source-ids to deal with multi arguments transforms [iceberg]

jbonofre commented on code in PR #12897: URL: https://github.com/apache/iceberg/pull/12897#discussion_r2060237024 ## .palantir/revapi.yml: ## @@ -1177,6 +1177,10 @@ acceptedBreaks: old: "class org.apache.iceberg.Metrics" new: "class org.apache.iceberg.Metrics"

Re: [I] Rough indication for a release date with Spark 4 support [iceberg]

jensborrmann commented on issue #12896: URL: https://github.com/apache/iceberg/issues/12896#issuecomment-2830422333 Thanks for the immediate answer! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

Re: [PR] Docs: Incorrect property in CREATE CATALOG for Flink [iceberg]

ebyhr commented on code in PR #12894: URL: https://github.com/apache/iceberg/pull/12894#discussion_r2059691717 ## docs/docs/aws.md: ## @@ -84,7 +84,7 @@ With those dependencies, you can create a Flink catalog like the following: CREATE CATALOG my_catalog WITH ( 'type'='iceb

Re: [PR] Core: Implement source-ids to deal with multi arguments transforms [iceberg]

jbonofre commented on code in PR #12897: URL: https://github.com/apache/iceberg/pull/12897#discussion_r2060214278 ## api/src/main/java/org/apache/iceberg/PartitionSpec.java: ## @@ -601,17 +601,26 @@ public Builder alwaysNull(String sourceName) { // add a partition field w

Re: [PR] Core: Implement source-ids to deal with multi arguments transforms [iceberg]

jbonofre commented on code in PR #12897: URL: https://github.com/apache/iceberg/pull/12897#discussion_r2060214278 ## api/src/main/java/org/apache/iceberg/PartitionSpec.java: ## @@ -601,17 +601,26 @@ public Builder alwaysNull(String sourceName) { // add a partition field w

[I] Manifest entry partition filtering on struct fields [iceberg-go]

GabrielM98 opened a new issue, #404: URL: https://github.com/apache/iceberg-go/issues/404 ### Apache Iceberg version main (development) ### Please describe the bug 🐞 There appears to be an issue with the way in which partition filtering is applied to manifest entries whe

Re: [PR] Core: Implement source-ids to deal with multi arguments transforms [iceberg]

jbonofre commented on code in PR #12897: URL: https://github.com/apache/iceberg/pull/12897#discussion_r2060216106 ## core/src/main/java/org/apache/iceberg/PartitionSpecParser.java: ## @@ -98,7 +101,16 @@ static void toJsonFields(UnboundPartitionSpec spec, JsonGenerator generato

  1   2   >