Re: [PR] API, Core: Add geometry and geography types support [iceberg]

2025-03-14 Thread via GitHub
szehon-ho commented on code in PR #12346: URL: https://github.com/apache/iceberg/pull/12346#discussion_r1996622869 ## core/src/main/java/org/apache/iceberg/SchemaParser.java: ## @@ -141,11 +145,39 @@ static void toJson(Types.MapType map, JsonGenerator generator) throws IOExcept

Re: [PR] API, Core: Add geometry and geography types support [iceberg]

2025-03-14 Thread via GitHub
Kontinuation commented on code in PR #12346: URL: https://github.com/apache/iceberg/pull/12346#discussion_r1996621912 ## api/src/test/java/org/apache/iceberg/types/TestSerializableTypes.java: ## @@ -63,7 +63,14 @@ public void testEqualTypes() throws Exception { Types.

Re: [PR] Core: HadoopCatalog root level warehouse path bug: missing slash [iceberg]

2025-03-14 Thread via GitHub
github-actions[bot] commented on PR #12233: URL: https://github.com/apache/iceberg/pull/12233#issuecomment-2726047471 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [I] Make reporter() function public from BaseTable public so that it can be used in Trino for creating BaseTable [iceberg]

2025-03-14 Thread via GitHub
ebyhr commented on issue #12513: URL: https://github.com/apache/iceberg/issues/12513#issuecomment-2726255913 @puchengy @nastra Can we close this issue? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] Parquet: Add variant array reader in Parquet [iceberg]

2025-03-14 Thread via GitHub
rdblue commented on code in PR #12512: URL: https://github.com/apache/iceberg/pull/12512#discussion_r1996392277 ## core/src/test/java/org/apache/iceberg/variants/TestShreddedArray.java: ## @@ -0,0 +1,190 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * o

Re: [PR] API, Core: Add geometry and geography types support [iceberg]

2025-03-14 Thread via GitHub
rdblue commented on code in PR #12346: URL: https://github.com/apache/iceberg/pull/12346#discussion_r1996298767 ## api/src/main/java/org/apache/iceberg/types/Types.java: ## @@ -543,6 +568,124 @@ public int hashCode() { } } + public static class GeometryType extends Pr

Re: [PR] Flink 1.20: Support Avro and Parquet timestamp(9), unknown, and defaults [iceberg]

2025-03-14 Thread via GitHub
rdblue merged PR #12470: URL: https://github.com/apache/iceberg/pull/12470 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [I] catalog table-default and table-override properties are not supported in CREATE_OR_REPLACE operation in IRC [iceberg]

2025-03-14 Thread via GitHub
puchengy commented on issue #12506: URL: https://github.com/apache/iceberg/issues/12506#issuecomment-2726179325 @nastra Thanks, I will it still don't work for my use case. In my use case, default table properties are set on IRC server side, not on client side. And based on my understanding

Re: [I] Create table format version constants [iceberg-python]

2025-03-14 Thread via GitHub
iyad-f commented on issue #851: URL: https://github.com/apache/iceberg-python/issues/851#issuecomment-2726145120 Hello @Fokko, The original PR statement addresses your question. It was noted that in numerous locations, we refer to the constants 1 and 2 as table versions, which will b

Re: [PR] Parquet: Add variant array reader in Parquet [iceberg]

2025-03-14 Thread via GitHub
rdblue commented on code in PR #12512: URL: https://github.com/apache/iceberg/pull/12512#discussion_r1996431272 ## parquet/src/main/java/org/apache/iceberg/parquet/ParquetVariantReaders.java: ## @@ -332,6 +347,91 @@ public void setPageSource(PageReadStore pageStore) { }

Re: [PR] Core: JDBCCatalog's dropView() should purge metadata files if GC is enabled [iceberg]

2025-03-14 Thread via GitHub
hsiang-c commented on code in PR #12511: URL: https://github.com/apache/iceberg/pull/12511#discussion_r1996433987 ## core/src/test/java/org/apache/iceberg/jdbc/TestJdbcViewCatalog.java: ## @@ -138,4 +142,50 @@ public void testCommitExceptionWithMessage() { .hasMessage

Re: [PR] Spark 3.4: Backport partition spec inference in spark ADD_FILES procedure [iceberg]

2025-03-14 Thread via GitHub
RussellSpitzer merged PR #12508: URL: https://github.com/apache/iceberg/pull/12508 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [PR] Parquet: Add variant array reader in Parquet [iceberg]

2025-03-14 Thread via GitHub
rdblue commented on code in PR #12512: URL: https://github.com/apache/iceberg/pull/12512#discussion_r1996431848 ## parquet/src/main/java/org/apache/iceberg/parquet/VariantReaderBuilder.java: ## @@ -159,8 +159,13 @@ public VariantValueReader object( @Override public Varia

Re: [PR] Core: JDBCCatalog's dropView() should purge metadata files if GC is enabled [iceberg]

2025-03-14 Thread via GitHub
hsiang-c commented on code in PR #12511: URL: https://github.com/apache/iceberg/pull/12511#discussion_r1996505373 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcCatalog.java: ## @@ -612,6 +613,14 @@ public boolean dropView(TableIdentifier identifier) { throw new Unsupp

Re: [PR] WIP: Interface based DataFile reader and writer API [iceberg]

2025-03-14 Thread via GitHub
rdblue commented on code in PR #12298: URL: https://github.com/apache/iceberg/pull/12298#discussion_r1996046263 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/data/vectorized/CometColumnarBatchReader.java: ## @@ -43,7 +43,7 @@ * populated via delegated read calls t

Re: [PR] Update documentation / add missing Iceberg table read properties [iceberg]

2025-03-14 Thread via GitHub
github-actions[bot] commented on PR #12163: URL: https://github.com/apache/iceberg/pull/12163#issuecomment-2726047234 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] Core: JDBCCatalog's dropView() should purge metadata files if GC is enabled [iceberg]

2025-03-14 Thread via GitHub
hsiang-c commented on code in PR #12511: URL: https://github.com/apache/iceberg/pull/12511#discussion_r1996443096 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcCatalog.java: ## @@ -612,6 +613,14 @@ public boolean dropView(TableIdentifier identifier) { throw new Unsupp

Re: [PR] WIP: Interface based DataFile reader and writer API [iceberg]

2025-03-14 Thread via GitHub
rdblue commented on code in PR #12298: URL: https://github.com/apache/iceberg/pull/12298#discussion_r1996104527 ## core/src/main/java/org/apache/iceberg/io/datafile/WriteBuilder.java: ## @@ -0,0 +1,340 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or

Re: [PR] API, Core: Add geometry and geography types support [iceberg]

2025-03-14 Thread via GitHub
rdblue commented on code in PR #12346: URL: https://github.com/apache/iceberg/pull/12346#discussion_r1995894075 ## api/src/test/java/org/apache/iceberg/types/TestSerializableTypes.java: ## @@ -63,7 +63,14 @@ public void testEqualTypes() throws Exception { Types.Decima

Re: [PR] WIP File format write [iceberg]

2025-03-14 Thread via GitHub
github-actions[bot] closed pull request #12164: WIP File format write URL: https://github.com/apache/iceberg/pull/12164 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsub

Re: [PR] Core: Fix numeric overflow of timestamp nano literal [iceberg]

2025-03-14 Thread via GitHub
ebyhr commented on code in PR #11775: URL: https://github.com/apache/iceberg/pull/11775#discussion_r1996456626 ## api/src/main/java/org/apache/iceberg/expressions/Literals.java: ## @@ -300,8 +300,7 @@ public Literal to(Type type) { case TIMESTAMP: return (Li

Re: [PR] WIP File format write [iceberg]

2025-03-14 Thread via GitHub
github-actions[bot] commented on PR #12164: URL: https://github.com/apache/iceberg/pull/12164#issuecomment-2726047279 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

Re: [I] Kafka Connect: Add delta writer support [iceberg]

2025-03-14 Thread via GitHub
github-actions[bot] closed issue #10842: Kafka Connect: Add delta writer support URL: https://github.com/apache/iceberg/issues/10842 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

Re: [PR] Core: Fix numeric overflow of timestamp nano literal [iceberg]

2025-03-14 Thread via GitHub
github-actions[bot] commented on PR #11775: URL: https://github.com/apache/iceberg/pull/11775#issuecomment-2726047205 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] Spark 3.5: Add Comet tests [iceberg]

2025-03-14 Thread via GitHub
github-actions[bot] closed pull request #12176: Spark 3.5: Add Comet tests URL: https://github.com/apache/iceberg/pull/12176 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] Core: JDBCCatalog's dropView() should purge metadata files if GC is enabled [iceberg]

2025-03-14 Thread via GitHub
hsiang-c commented on code in PR #12511: URL: https://github.com/apache/iceberg/pull/12511#discussion_r1996443096 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcCatalog.java: ## @@ -612,6 +613,14 @@ public boolean dropView(TableIdentifier identifier) { throw new Unsupp

Re: [PR] API, Core: Add geometry and geography types support [iceberg]

2025-03-14 Thread via GitHub
rdblue commented on code in PR #12346: URL: https://github.com/apache/iceberg/pull/12346#discussion_r1996302892 ## api/src/test/java/org/apache/iceberg/types/TestTypes.java: ## @@ -81,6 +81,56 @@ public void fromPrimitiveString() { .withMessage("Cannot parse type string

Re: [PR] Core: PartitionsTable#partitions returns incomplete list in case of partition evolution and null partition values [iceberg]

2025-03-14 Thread via GitHub
RussellSpitzer commented on code in PR #12528: URL: https://github.com/apache/iceberg/pull/12528#discussion_r1996373199 ## api/src/main/java/org/apache/iceberg/types/Comparators.java: ## @@ -108,6 +109,15 @@ public int compare(StructLike o1, StructLike o2) { return 0;

Re: [PR] Core: JDBCCatalog's dropView() should purge metadata files if GC is enabled [iceberg]

2025-03-14 Thread via GitHub
hsiang-c commented on code in PR #12511: URL: https://github.com/apache/iceberg/pull/12511#discussion_r1996433987 ## core/src/test/java/org/apache/iceberg/jdbc/TestJdbcViewCatalog.java: ## @@ -138,4 +142,50 @@ public void testCommitExceptionWithMessage() { .hasMessage

Re: [PR] build(deps): bump golang.org/x/net from 0.35.0 to 0.36.0 [iceberg-go]

2025-03-14 Thread via GitHub
zeroshade merged PR #331: URL: https://github.com/apache/iceberg-go/pull/331 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] Parquet: Add variant array reader in Parquet [iceberg]

2025-03-14 Thread via GitHub
rdblue commented on code in PR #12512: URL: https://github.com/apache/iceberg/pull/12512#discussion_r1996429723 ## parquet/src/main/java/org/apache/iceberg/parquet/ParquetVariantReaders.java: ## @@ -332,6 +347,91 @@ public void setPageSource(PageReadStore pageStore) { }

Re: [I] improve pyiceberg CLI [iceberg-python]

2025-03-14 Thread via GitHub
kevinjqliu commented on issue #1784: URL: https://github.com/apache/iceberg-python/issues/1784#issuecomment-2725994258 sure @iting0321 happy to help review :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

Re: [PR] Parquet: Add variant array reader in Parquet [iceberg]

2025-03-14 Thread via GitHub
rdblue commented on code in PR #12512: URL: https://github.com/apache/iceberg/pull/12512#discussion_r1996400677 ## core/src/test/java/org/apache/iceberg/variants/TestShreddedArray.java: ## @@ -0,0 +1,190 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * o

Re: [PR] Parquet: Add variant array reader in Parquet [iceberg]

2025-03-14 Thread via GitHub
rdblue commented on code in PR #12512: URL: https://github.com/apache/iceberg/pull/12512#discussion_r1996400299 ## core/src/test/java/org/apache/iceberg/variants/TestShreddedArray.java: ## @@ -0,0 +1,190 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * o

Re: [PR] Azure: Support vended credentials refresh in ADLSFileIO. [iceberg]

2025-03-14 Thread via GitHub
danielcweeks commented on code in PR #11577: URL: https://github.com/apache/iceberg/pull/11577#discussion_r1996385021 ## azure/src/main/java/org/apache/iceberg/azure/adlsv2/VendedAdlsCredentialProvider.java: ## @@ -0,0 +1,157 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] Core: PartitionsTable#partitions returns incomplete list in case of partition evolution and null partition values [iceberg]

2025-03-14 Thread via GitHub
deniskuzZ commented on code in PR #12528: URL: https://github.com/apache/iceberg/pull/12528#discussion_r1996398329 ## api/src/main/java/org/apache/iceberg/types/Comparators.java: ## @@ -108,6 +109,15 @@ public int compare(StructLike o1, StructLike o2) { return 0;

Re: [PR] Core: PartitionsTable#partitions returns incomplete list in case of partition evolution and null partition values [iceberg]

2025-03-14 Thread via GitHub
deniskuzZ commented on code in PR #12528: URL: https://github.com/apache/iceberg/pull/12528#discussion_r1996398329 ## api/src/main/java/org/apache/iceberg/types/Comparators.java: ## @@ -108,6 +109,15 @@ public int compare(StructLike o1, StructLike o2) { return 0;

Re: [PR] Parquet: Add variant array reader in Parquet [iceberg]

2025-03-14 Thread via GitHub
rdblue commented on code in PR #12512: URL: https://github.com/apache/iceberg/pull/12512#discussion_r1996398148 ## core/src/test/java/org/apache/iceberg/variants/TestShreddedArray.java: ## @@ -0,0 +1,190 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * o

Re: [PR] Azure: Support vended credentials refresh in ADLSFileIO. [iceberg]

2025-03-14 Thread via GitHub
danielcweeks commented on code in PR #11577: URL: https://github.com/apache/iceberg/pull/11577#discussion_r1996399213 ## azure/src/main/java/org/apache/iceberg/azure/adlsv2/VendedAdlsCredentialProvider.java: ## @@ -0,0 +1,155 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] Parquet: Add variant array reader in Parquet [iceberg]

2025-03-14 Thread via GitHub
rdblue commented on code in PR #12512: URL: https://github.com/apache/iceberg/pull/12512#discussion_r1996391841 ## core/src/test/java/org/apache/iceberg/variants/TestShreddedArray.java: ## @@ -0,0 +1,190 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * o

Re: [PR] API, Core: Add geometry and geography types support [iceberg]

2025-03-14 Thread via GitHub
rdblue commented on code in PR #12346: URL: https://github.com/apache/iceberg/pull/12346#discussion_r1996309678 ## core/src/test/java/org/apache/iceberg/data/DataTest.java: ## @@ -129,28 +135,37 @@ protected boolean supportsVariant() { return false; } + protected bool

Re: [PR] Parquet: Add variant array reader in Parquet [iceberg]

2025-03-14 Thread via GitHub
rdblue commented on code in PR #12512: URL: https://github.com/apache/iceberg/pull/12512#discussion_r1996387522 ## core/src/test/java/org/apache/iceberg/variants/TestShreddedArray.java: ## @@ -0,0 +1,190 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * o

Re: [PR] Core: PartitionsTable#partitions returns incomplete list in case of partition evolution and null partition values [iceberg]

2025-03-14 Thread via GitHub
deniskuzZ commented on code in PR #12528: URL: https://github.com/apache/iceberg/pull/12528#discussion_r1996347542 ## api/src/main/java/org/apache/iceberg/types/Comparators.java: ## @@ -108,6 +109,15 @@ public int compare(StructLike o1, StructLike o2) { return 0;

Re: [PR] Parquet: Add variant array reader in Parquet [iceberg]

2025-03-14 Thread via GitHub
rdblue commented on code in PR #12512: URL: https://github.com/apache/iceberg/pull/12512#discussion_r1996384210 ## core/src/test/java/org/apache/iceberg/variants/TestShreddedArray.java: ## @@ -0,0 +1,190 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * o

[PR] Build: Bump mypy-boto3-glue from 1.37.0 to 1.37.13 [iceberg-python]

2025-03-14 Thread via GitHub
dependabot[bot] opened a new pull request, #1795: URL: https://github.com/apache/iceberg-python/pull/1795 Bumps [mypy-boto3-glue](https://github.com/youtype/mypy_boto3_builder) from 1.37.0 to 1.37.13. Release notes Sourced from https://github.com/youtype/mypy_boto3_builder/releases

[PR] Build: Bump polars from 1.24.0 to 1.25.0 [iceberg-python]

2025-03-14 Thread via GitHub
dependabot[bot] opened a new pull request, #1794: URL: https://github.com/apache/iceberg-python/pull/1794 Bumps [polars](https://github.com/pola-rs/polars) from 1.24.0 to 1.25.0. Commits See full diff in https://github.com/pola-rs/polars/commits";>compare view

Re: [PR] Azure: Support vended credentials refresh in ADLSFileIO. [iceberg]

2025-03-14 Thread via GitHub
danielcweeks commented on code in PR #11577: URL: https://github.com/apache/iceberg/pull/11577#discussion_r1996376630 ## azure/src/main/java/org/apache/iceberg/azure/adlsv2/VendedAdlsCredentialProvider.java: ## @@ -0,0 +1,157 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] API, Core: Add geometry and geography types support [iceberg]

2025-03-14 Thread via GitHub
szehon-ho commented on code in PR #12346: URL: https://github.com/apache/iceberg/pull/12346#discussion_r1996376344 ## core/src/main/java/org/apache/iceberg/SingleValueParser.java: ## @@ -335,6 +349,13 @@ public static void toJson(Type type, Object defaultValue, JsonGenerator ge

Re: [PR] Spark 3.4: Backport partition spec inference in spark ADD_FILES procedure [iceberg]

2025-03-14 Thread via GitHub
RussellSpitzer commented on PR #12508: URL: https://github.com/apache/iceberg/pull/12508#issuecomment-2725917531 Thanks @bharos ! Merged -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

Re: [PR] Azure: Support vended credentials refresh in ADLSFileIO. [iceberg]

2025-03-14 Thread via GitHub
danielcweeks commented on code in PR #11577: URL: https://github.com/apache/iceberg/pull/11577#discussion_r1996371312 ## azure/src/main/java/org/apache/iceberg/azure/AzureProperties.java: ## @@ -25,24 +25,42 @@ import java.util.Collections; import java.util.Map; import java.u

Re: [PR] Azure: Support vended credentials refresh in ADLSFileIO. [iceberg]

2025-03-14 Thread via GitHub
danielcweeks commented on code in PR #11577: URL: https://github.com/apache/iceberg/pull/11577#discussion_r1996371312 ## azure/src/main/java/org/apache/iceberg/azure/AzureProperties.java: ## @@ -25,24 +25,42 @@ import java.util.Collections; import java.util.Map; import java.u

Re: [PR] Flink 1.20: Support Avro and Parquet timestamp(9), unknown, and defaults [iceberg]

2025-03-14 Thread via GitHub
rdblue commented on PR #12470: URL: https://github.com/apache/iceberg/pull/12470#issuecomment-2725892975 Thanks for reviewing, @pvary and @danielcweeks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] feat: (catalog/glue) Add support for CreateTable [iceberg-go]

2025-03-14 Thread via GitHub
dttung2905 commented on code in PR #326: URL: https://github.com/apache/iceberg-go/pull/326#discussion_r1996346856 ## catalog/glue/schema.go: ## @@ -0,0 +1,107 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the

Re: [PR] feat: (catalog/glue) Add support for CreateTable [iceberg-go]

2025-03-14 Thread via GitHub
zeroshade merged PR #326: URL: https://github.com/apache/iceberg-go/pull/326 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] Core: PartitionsTable#partitions returns incomplete list in case of partition evolution and null partition values [iceberg]

2025-03-14 Thread via GitHub
deniskuzZ commented on code in PR #12528: URL: https://github.com/apache/iceberg/pull/12528#discussion_r1996343029 ## api/src/main/java/org/apache/iceberg/types/Comparators.java: ## @@ -108,6 +109,15 @@ public int compare(StructLike o1, StructLike o2) { return 0;

Re: [PR] Core: PartitionsTable#partitions returns incomplete list in case of partition evolution and null partition values [iceberg]

2025-03-14 Thread via GitHub
RussellSpitzer commented on code in PR #12528: URL: https://github.com/apache/iceberg/pull/12528#discussion_r1996345959 ## api/src/main/java/org/apache/iceberg/types/Comparators.java: ## @@ -108,6 +109,15 @@ public int compare(StructLike o1, StructLike o2) { return 0;

Re: [PR] API, Core: Add geometry and geography types support [iceberg]

2025-03-14 Thread via GitHub
rdblue commented on code in PR #12346: URL: https://github.com/apache/iceberg/pull/12346#discussion_r1996308827 ## core/src/test/java/org/apache/iceberg/TestTableMetadata.java: ## @@ -1825,6 +1825,43 @@ public void testConstructV3Metadata() { 3); } + @Test + publ

Re: [PR] API, Core: Add geometry and geography types support [iceberg]

2025-03-14 Thread via GitHub
rdblue commented on code in PR #12346: URL: https://github.com/apache/iceberg/pull/12346#discussion_r1996297055 ## api/src/main/java/org/apache/iceberg/types/Types.java: ## @@ -543,6 +568,124 @@ public int hashCode() { } } + public static class GeometryType extends Pr

Re: [PR] Flink 1.20: Support Avro and Parquet timestamp(9), unknown, and defaults [iceberg]

2025-03-14 Thread via GitHub
rdblue commented on code in PR #12470: URL: https://github.com/apache/iceberg/pull/12470#discussion_r1996272927 ## flink/v1.20/flink/src/test/java/org/apache/iceberg/flink/data/AbstractTestFlinkAvroReaderWriter.java: ## @@ -1,182 +0,0 @@ -/* - * Licensed to the Apache Software F

Re: [PR] Flink 1.20: Support Avro and Parquet timestamp(9), unknown, and defaults [iceberg]

2025-03-14 Thread via GitHub
rdblue commented on code in PR #12470: URL: https://github.com/apache/iceberg/pull/12470#discussion_r1996281732 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/data/FlinkParquetWriters.java: ## @@ -194,8 +177,104 @@ public ParquetValueWriter primitive(LogicalType fTy

Re: [PR] Flink 1.20: Support Avro and Parquet timestamp(9), unknown, and defaults [iceberg]

2025-03-14 Thread via GitHub
rdblue commented on code in PR #12470: URL: https://github.com/apache/iceberg/pull/12470#discussion_r1996274844 ## flink/v1.20/flink/src/test/java/org/apache/iceberg/flink/TestHelpers.java: ## @@ -187,11 +188,38 @@ public static void assertRowData( types.add(field.type())

Re: [PR] feat: support S3 Table Buckets with S3TablesCatalog [iceberg-python]

2025-03-14 Thread via GitHub
0x26res commented on code in PR #1429: URL: https://github.com/apache/iceberg-python/pull/1429#discussion_r1995965268 ## mkdocs/docs/configuration.md: ## @@ -550,6 +550,104 @@ catalog: +### S3Tables Catalog + +The S3Tables Catalog leverages the catalog functionalities of t

Re: [PR] WIP: Interface based DataFile reader and writer API [iceberg]

2025-03-14 Thread via GitHub
rdblue commented on code in PR #12298: URL: https://github.com/apache/iceberg/pull/12298#discussion_r1996077906 ## data/src/main/java/org/apache/iceberg/data/RegistryBasedFileWriterFactory.java: ## @@ -0,0 +1,191 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] Core: PartitionsTable#partitions returns incomplete list in case of partition evolution and null partition values [iceberg]

2025-03-14 Thread via GitHub
deniskuzZ commented on code in PR #12528: URL: https://github.com/apache/iceberg/pull/12528#discussion_r1996252807 ## api/src/main/java/org/apache/iceberg/types/Comparators.java: ## @@ -108,6 +109,15 @@ public int compare(StructLike o1, StructLike o2) { return 0;

Re: [PR] Core: PartitionsTable#partitions returns incomplete list in case of partition evolution and null partition values [iceberg]

2025-03-14 Thread via GitHub
deniskuzZ commented on code in PR #12528: URL: https://github.com/apache/iceberg/pull/12528#discussion_r1996252807 ## api/src/main/java/org/apache/iceberg/types/Comparators.java: ## @@ -108,6 +109,15 @@ public int compare(StructLike o1, StructLike o2) { return 0;

Re: [PR] Auth Manager API part 6: API enablement [iceberg]

2025-03-14 Thread via GitHub
adutra commented on code in PR #12197: URL: https://github.com/apache/iceberg/pull/12197#discussion_r1996148530 ## aws/src/test/java/org/apache/iceberg/aws/s3/signer/TestS3RestSigner.java: ## @@ -96,45 +98,10 @@ public static void beforeClass() throws Exception { if (null =

Re: [PR] WIP: Interface based DataFile reader and writer API [iceberg]

2025-03-14 Thread via GitHub
rdblue commented on code in PR #12298: URL: https://github.com/apache/iceberg/pull/12298#discussion_r1996105772 ## core/src/main/java/org/apache/iceberg/io/datafile/WriteBuilder.java: ## @@ -0,0 +1,340 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or

Re: [PR] WIP: Interface based DataFile reader and writer API [iceberg]

2025-03-14 Thread via GitHub
rdblue commented on code in PR #12298: URL: https://github.com/apache/iceberg/pull/12298#discussion_r1996106267 ## core/src/main/java/org/apache/iceberg/io/datafile/WriteBuilder.java: ## @@ -0,0 +1,340 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or

Re: [PR] feat(table): Basic Transaction and AddFiles [iceberg-go]

2025-03-14 Thread via GitHub
zeroshade commented on PR #330: URL: https://github.com/apache/iceberg-go/pull/330#issuecomment-2725492072 @Fokko I've added the test cases and handling of decimals. Do you still need me to split this PR into smaller chunks or anything else? Or are we good here and ready for the next one? :

Re: [PR] feat(table): Basic Transaction and AddFiles [iceberg-go]

2025-03-14 Thread via GitHub
zeroshade commented on code in PR #330: URL: https://github.com/apache/iceberg-go/pull/330#discussion_r1996104193 ## table/arrow_utils_internal_test.go: ## @@ -408,3 +478,158 @@ func TestStatsTypes(t *testing.T) { iceberg.PrimitiveTypes.Int32, }, actual)

Re: [PR] WIP: Interface based DataFile reader and writer API [iceberg]

2025-03-14 Thread via GitHub
danielcweeks commented on code in PR #12298: URL: https://github.com/apache/iceberg/pull/12298#discussion_r1996093918 ## core/src/main/java/org/apache/iceberg/io/datafile/DataFileServiceRegistry.java: ## @@ -0,0 +1,231 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [I] REST Catalog `createTransaction` throws with overridden table defaults [iceberg]

2025-03-14 Thread via GitHub
nastra closed issue #12525: REST Catalog `createTransaction` throws with overridden table defaults URL: https://github.com/apache/iceberg/issues/12525 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] OpenAPI: Handle NamespaceNotEmptyException when dropping a namespace [iceberg]

2025-03-14 Thread via GitHub
nastra commented on code in PR #12518: URL: https://github.com/apache/iceberg/pull/12518#discussion_r1996077379 ## open-api/rest-catalog-open-api.yaml: ## @@ -410,7 +410,17 @@ paths: 204: description: Success, no content 400: - $ref: '#/comp

Re: [PR] WIP: Interface based DataFile reader and writer API [iceberg]

2025-03-14 Thread via GitHub
rdblue commented on code in PR #12298: URL: https://github.com/apache/iceberg/pull/12298#discussion_r1996064834 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/DataFileServices.java: ## @@ -0,0 +1,132 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] WIP: Interface based DataFile reader and writer API [iceberg]

2025-03-14 Thread via GitHub
rdblue commented on code in PR #12298: URL: https://github.com/apache/iceberg/pull/12298#discussion_r1996048683 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/BaseBatchReader.java: ## @@ -65,76 +59,32 @@ protected CloseableIterable newBatchIterable( Exp

Re: [PR] WIP: Interface based DataFile reader and writer API [iceberg]

2025-03-14 Thread via GitHub
rdblue commented on code in PR #12298: URL: https://github.com/apache/iceberg/pull/12298#discussion_r1996047261 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/BaseBatchReader.java: ## @@ -65,76 +59,32 @@ protected CloseableIterable newBatchIterable( Exp

Re: [PR] feat: support S3 Table Buckets with S3TablesCatalog [iceberg-python]

2025-03-14 Thread via GitHub
felixscherz commented on code in PR #1429: URL: https://github.com/apache/iceberg-python/pull/1429#discussion_r1996046870 ## mkdocs/docs/configuration.md: ## @@ -550,6 +550,104 @@ catalog: +### S3Tables Catalog + +The S3Tables Catalog leverages the catalog functionalities

[PR] Core: PartitionsTable#partitions returns incomplete list in case of partition evolution and null partition values [iceberg]

2025-03-14 Thread via GitHub
deniskuzZ opened a new pull request, #12528: URL: https://github.com/apache/iceberg/pull/12528 BugFix: Below 3 StructProjections are considered equal leading to key overwrite in PartitionMap; "ice_orc(company_id=null)" "ice_orc(company_id=null/dept_id=null)" "ice_orc(

Re: [PR] WIP: Interface based DataFile reader and writer API [iceberg]

2025-03-14 Thread via GitHub
rdblue commented on code in PR #12298: URL: https://github.com/apache/iceberg/pull/12298#discussion_r1996031442 ## core/src/main/java/org/apache/iceberg/io/datafile/DataFileServiceRegistry.java: ## @@ -0,0 +1,231 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] Auth Manager API part 6: API enablement [iceberg]

2025-03-14 Thread via GitHub
nastra commented on code in PR #12197: URL: https://github.com/apache/iceberg/pull/12197#discussion_r1995608942 ## core/src/main/java/org/apache/iceberg/rest/RESTSessionCatalog.java: ## @@ -1206,12 +1006,15 @@ public void commitTransaction(SessionContext context, List commits)

Re: [PR] WIP: Interface based DataFile reader and writer API [iceberg]

2025-03-14 Thread via GitHub
liurenjie1024 commented on code in PR #12298: URL: https://github.com/apache/iceberg/pull/12298#discussion_r1995602495 ## core/src/main/java/org/apache/iceberg/io/datafile/AppenderBuilder.java: ## @@ -0,0 +1,121 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under o

Re: [PR] WIP: Interface based DataFile reader and writer API [iceberg]

2025-03-14 Thread via GitHub
rdblue commented on code in PR #12298: URL: https://github.com/apache/iceberg/pull/12298#discussion_r1996040462 ## core/src/main/java/org/apache/iceberg/io/datafile/ReadBuilder.java: ## @@ -0,0 +1,139 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or m

Re: [PR] Core: JDBCCatalog's dropView() should purge metadata files if GC is enabled [iceberg]

2025-03-14 Thread via GitHub
hsiang-c commented on code in PR #12511: URL: https://github.com/apache/iceberg/pull/12511#discussion_r1996039962 ## core/src/test/java/org/apache/iceberg/jdbc/TestJdbcViewCatalog.java: ## @@ -138,4 +142,50 @@ public void testCommitExceptionWithMessage() { .hasMessage

Re: [PR] WIP: Interface based DataFile reader and writer API [iceberg]

2025-03-14 Thread via GitHub
rdblue commented on code in PR #12298: URL: https://github.com/apache/iceberg/pull/12298#discussion_r1996038345 ## core/src/main/java/org/apache/iceberg/io/datafile/ReadBuilder.java: ## @@ -0,0 +1,139 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or m

Re: [PR] feat: support S3 Table Buckets with S3TablesCatalog [iceberg-python]

2025-03-14 Thread via GitHub
kevinjqliu commented on PR #1429: URL: https://github.com/apache/iceberg-python/pull/1429#issuecomment-2725402615 hey @felixscherz thanks for following up on this PR. I haven't forgotten about this! :) I'll do another review soon -- This is an automated message from the Apache Git Service

Re: [PR] WIP: Interface based DataFile reader and writer API [iceberg]

2025-03-14 Thread via GitHub
rdblue commented on code in PR #12298: URL: https://github.com/apache/iceberg/pull/12298#discussion_r1996036485 ## core/src/main/java/org/apache/iceberg/io/datafile/DeleteFilter.java: ## @@ -0,0 +1,52 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or m

Re: [PR] WIP: Interface based DataFile reader and writer API [iceberg]

2025-03-14 Thread via GitHub
rdblue commented on code in PR #12298: URL: https://github.com/apache/iceberg/pull/12298#discussion_r1996033869 ## core/src/main/java/org/apache/iceberg/io/datafile/DataFileServiceRegistry.java: ## @@ -0,0 +1,231 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] WIP: Interface based DataFile reader and writer API [iceberg]

2025-03-14 Thread via GitHub
rdblue commented on code in PR #12298: URL: https://github.com/apache/iceberg/pull/12298#discussion_r1996030623 ## core/src/main/java/org/apache/iceberg/io/datafile/DataFileServiceRegistry.java: ## @@ -0,0 +1,231 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] WIP: Interface based DataFile reader and writer API [iceberg]

2025-03-14 Thread via GitHub
rdblue commented on code in PR #12298: URL: https://github.com/apache/iceberg/pull/12298#discussion_r1996028761 ## core/src/main/java/org/apache/iceberg/io/datafile/DataFileServiceRegistry.java: ## @@ -0,0 +1,231 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] WIP: Interface based DataFile reader and writer API [iceberg]

2025-03-14 Thread via GitHub
rdblue commented on code in PR #12298: URL: https://github.com/apache/iceberg/pull/12298#discussion_r1996029195 ## core/src/main/java/org/apache/iceberg/io/datafile/DataFileServiceRegistry.java: ## @@ -0,0 +1,231 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] feat: support S3 Table Buckets with S3TablesCatalog [iceberg-python]

2025-03-14 Thread via GitHub
0x26res commented on code in PR #1429: URL: https://github.com/apache/iceberg-python/pull/1429#discussion_r1996023936 ## mkdocs/docs/configuration.md: ## @@ -550,6 +550,104 @@ catalog: +### S3Tables Catalog + +The S3Tables Catalog leverages the catalog functionalities of t

Re: [PR] feat: support S3 Table Buckets with S3TablesCatalog [iceberg-python]

2025-03-14 Thread via GitHub
felixscherz commented on code in PR #1429: URL: https://github.com/apache/iceberg-python/pull/1429#discussion_r1996003679 ## mkdocs/docs/configuration.md: ## @@ -550,6 +550,104 @@ catalog: +### S3Tables Catalog + +The S3Tables Catalog leverages the catalog functionalities

Re: [PR] OpenAPI: Handle NamespaceNotEmptyException when dropping a namespace [iceberg]

2025-03-14 Thread via GitHub
rdblue commented on code in PR #12518: URL: https://github.com/apache/iceberg/pull/12518#discussion_r1995983897 ## open-api/rest-catalog-open-api.yaml: ## @@ -410,7 +410,17 @@ paths: 204: description: Success, no content 400: - $ref: '#/comp

Re: [PR] Kafka Connect: Handle no coordinator and data loss in ICR mode [iceberg]

2025-03-14 Thread via GitHub
kumarpritam863 commented on PR #12372: URL: https://github.com/apache/iceberg/pull/12372#issuecomment-2725283871 Thanks @bryanck for all the support, insights and reviews. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and us

Re: [PR] Flink 1.20: Support Avro and Parquet timestamp(9), unknown, and defaults [iceberg]

2025-03-14 Thread via GitHub
danielcweeks commented on code in PR #12470: URL: https://github.com/apache/iceberg/pull/12470#discussion_r1995967404 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/data/FlinkParquetWriters.java: ## @@ -194,8 +177,104 @@ public ParquetValueWriter primitive(LogicalTy

Re: [PR] Kafka Connect: Handling no coordinator and data loss in ICR mode [iceberg]

2025-03-14 Thread via GitHub
bryanck merged PR #12372: URL: https://github.com/apache/iceberg/pull/12372 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.a

Re: [PR] API, Core: Add geometry and geography types support [iceberg]

2025-03-14 Thread via GitHub
rdblue commented on code in PR #12346: URL: https://github.com/apache/iceberg/pull/12346#discussion_r1995890334 ## api/src/main/java/org/apache/iceberg/types/Types.java: ## @@ -543,6 +563,122 @@ public int hashCode() { } } + public static class GeometryType extends Pr

Re: [PR] API, Core: Add geometry and geography types support [iceberg]

2025-03-14 Thread via GitHub
rdblue commented on code in PR #12346: URL: https://github.com/apache/iceberg/pull/12346#discussion_r1995896505 ## core/src/main/java/org/apache/iceberg/SchemaParser.java: ## @@ -141,11 +145,39 @@ static void toJson(Types.MapType map, JsonGenerator generator) throws IOExceptio

Re: [PR] API, Core: Add geometry and geography types support [iceberg]

2025-03-14 Thread via GitHub
rdblue commented on code in PR #12346: URL: https://github.com/apache/iceberg/pull/12346#discussion_r1995885931 ## api/src/main/java/org/apache/iceberg/types/Types.java: ## @@ -543,6 +561,120 @@ public int hashCode() { } } + public static class GeometryType extends Pr

Re: [PR] Kafka Connect: Handling no coordinator and data loss in ICR mode [iceberg]

2025-03-14 Thread via GitHub
bryanck commented on code in PR #12372: URL: https://github.com/apache/iceberg/pull/12372#discussion_r1995860043 ## kafka-connect/kafka-connect/src/main/java/org/apache/iceberg/connect/channel/CommitterImpl.java: ## @@ -125,4 +161,35 @@ private void processControlEvents() {

  1   2   >