Re: [PR] feat(table): Implement converting Iceberg schema and types to Arrow [iceberg-go]

2024-10-18 Thread via GitHub
nastra commented on PR #168: URL: https://github.com/apache/iceberg-go/pull/168#issuecomment-2421911918 @zeroshade can you please rebase due to the merge conflict? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

Re: [PR] [KafkaConnect] Fix RecordConverter [iceberg]

2024-10-18 Thread via GitHub
singhpk234 commented on code in PR #11346: URL: https://github.com/apache/iceberg/pull/11346#discussion_r1806695970 ## orc/src/main/java/org/apache/iceberg/data/orc/GenericOrcWriters.java: ## @@ -101,8 +101,8 @@ public static OrcValueWriter byteBuffers() { return ByteBuffer

[PR] OpenAPI: Update Snapshots to be a common denominator of V1 and V2 specs [iceberg]

2024-10-18 Thread via GitHub
sungwy opened a new pull request, #11353: URL: https://github.com/apache/iceberg/pull/11353 In a recent [mail list discussion](https://lists.apache.org/thread/h9qmrmlgxh91ol0y2v8olt90b9q6p9xr), it has come to the community's attention that the Open API Spec does not conform to the current

Re: [I] REST Catalog pagination can throw IndexOutOfBoundsException [iceberg]

2024-10-18 Thread via GitHub
munendrasn commented on issue #11142: URL: https://github.com/apache/iceberg/issues/11142#issuecomment-2422468094 @rcjverhoef Is this good to be closed as the PR is merged? cc @amogh-jahagirdar -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] Spec: add variant type [iceberg]

2024-10-18 Thread via GitHub
aihuaxu commented on PR #10831: URL: https://github.com/apache/iceberg/pull/10831#issuecomment-2423052011 > This needs some notes in `Partition Transforms` , I think explicitly we should disallow identity > > For Appendix B - We should define something or state explicitly we don't de

Re: [PR] AWS: Introduce opt-in S3LocationProvider which is optimized for S3 performance [iceberg]

2024-10-18 Thread via GitHub
ookumuso commented on code in PR #2: URL: https://github.com/apache/iceberg/pull/2#discussion_r1806927534 ## core/src/main/java/org/apache/iceberg/LocationProviders.java: ## @@ -172,10 +193,45 @@ private static String pathContext(String tableLocation) { } pri

Re: [PR] Core: code clean up for deleteRemovedMetadataFiles [iceberg]

2024-10-18 Thread via GitHub
leesf commented on code in PR #11352: URL: https://github.com/apache/iceberg/pull/11352#discussion_r1806407924 ## core/src/main/java/org/apache/iceberg/TableMetadata.java: ## @@ -77,6 +83,51 @@ public static TableMetadata newTableMetadata( return newTableMetadata(schema, sp

Re: [I] REST Catalog pagination can throw IndexOutOfBoundsException [iceberg]

2024-10-18 Thread via GitHub
rcjverhoef closed issue #11142: REST Catalog pagination can throw IndexOutOfBoundsException URL: https://github.com/apache/iceberg/issues/11142 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

Re: [I] REST Catalog pagination can throw IndexOutOfBoundsException [iceberg]

2024-10-18 Thread via GitHub
rcjverhoef commented on issue #11142: URL: https://github.com/apache/iceberg/issues/11142#issuecomment-2422470740 yes it! let me do so. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

Re: [PR] feat(table): Implement converting Iceberg schema and types to Arrow [iceberg-go]

2024-10-18 Thread via GitHub
nastra merged PR #168: URL: https://github.com/apache/iceberg-go/pull/168 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] Spark 3.5: Fix NotSerializableException when migrating partitioned Spark tables [iceberg]

2024-10-18 Thread via GitHub
manuzhang commented on code in PR #11157: URL: https://github.com/apache/iceberg/pull/11157#discussion_r1806768662 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/SparkTableUtil.java: ## @@ -711,7 +719,7 @@ public static void importSparkPartitions( spec,

Re: [PR] REST: AuthManager API [iceberg]

2024-10-18 Thread via GitHub
danielcweeks commented on code in PR #10753: URL: https://github.com/apache/iceberg/pull/10753#discussion_r1807055242 ## core/src/main/java/org/apache/iceberg/rest/RESTSessionCatalog.java: ## @@ -273,35 +227,11 @@ public void initialize(String name, Map unresolved) { thi

Re: [PR] Arrow: Fix indexing in Parquet dictionary encoded values readers [iceberg]

2024-10-18 Thread via GitHub
wypoon commented on code in PR #11247: URL: https://github.com/apache/iceberg/pull/11247#discussion_r1807089230 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/data/parquet/vectorized/TestParquetDictionaryEncodedVectorizedReads.java: ## @@ -93,4 +125,64 @@ public void

Re: [I] Create empty snapshot for metadata operations [iceberg]

2024-10-18 Thread via GitHub
github-actions[bot] closed issue #7075: Create empty snapshot for metadata operations URL: https://github.com/apache/iceberg/issues/7075 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [I] Support Parquet v2 Spark vectorized read [iceberg]

2024-10-18 Thread via GitHub
github-actions[bot] closed issue #7162: Support Parquet v2 Spark vectorized read URL: https://github.com/apache/iceberg/issues/7162 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

Re: [I] Support Page Skipping in Iceberg Parquet Reader [iceberg]

2024-10-18 Thread via GitHub
github-actions[bot] closed issue #193: Support Page Skipping in Iceberg Parquet Reader URL: https://github.com/apache/iceberg/issues/193 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [I] Does JDBC connector uses any retry mechanism? [iceberg]

2024-10-18 Thread via GitHub
github-actions[bot] commented on issue #7173: URL: https://github.com/apache/iceberg/issues/7173#issuecomment-2423396446 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Does JDBC connector uses any retry mechanism? [iceberg]

2024-10-18 Thread via GitHub
github-actions[bot] closed issue #7173: Does JDBC connector uses any retry mechanism? URL: https://github.com/apache/iceberg/issues/7173 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [I] org.apache.iceberg.hive.RuntimeMetaException: Failed to connect to Hive Metastore at [iceberg]

2024-10-18 Thread via GitHub
github-actions[bot] closed issue #9030: org.apache.iceberg.hive.RuntimeMetaException: Failed to connect to Hive Metastore at URL: https://github.com/apache/iceberg/issues/9030 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [I] org.apache.iceberg.hive.RuntimeMetaException: Failed to connect to Hive Metastore at [iceberg]

2024-10-18 Thread via GitHub
github-actions[bot] commented on issue #9030: URL: https://github.com/apache/iceberg/issues/9030#issuecomment-2423396460 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Create empty snapshot for metadata operations [iceberg]

2024-10-18 Thread via GitHub
github-actions[bot] commented on issue #7075: URL: https://github.com/apache/iceberg/issues/7075#issuecomment-2423396430 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Support Page Skipping in Iceberg Parquet Reader [iceberg]

2024-10-18 Thread via GitHub
github-actions[bot] commented on issue #193: URL: https://github.com/apache/iceberg/issues/193#issuecomment-2423396417 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Git

Re: [I] Support Parquet v2 Spark vectorized read [iceberg]

2024-10-18 Thread via GitHub
github-actions[bot] commented on issue #7162: URL: https://github.com/apache/iceberg/issues/7162#issuecomment-2423396436 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Integration tests performance degradation [iceberg-python]

2024-10-18 Thread via GitHub
github-actions[bot] commented on issue #604: URL: https://github.com/apache/iceberg-python/issues/604#issuecomment-2423397846 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity oc

Re: [I] Inconsistency in catalog.list_tables Behavior Across Python and Java: Returns Non-Iceberg Tables in Python Only [iceberg-python]

2024-10-18 Thread via GitHub
sungwy closed issue #314: Inconsistency in catalog.list_tables Behavior Across Python and Java: Returns Non-Iceberg Tables in Python Only URL: https://github.com/apache/iceberg-python/issues/314 -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [PR] core: use testcontainers-minio [iceberg]

2024-10-18 Thread via GitHub
sullis commented on PR #11349: URL: https://github.com/apache/iceberg/pull/11349#issuecomment-2423404051 GitHub Actions CI passed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

Re: [PR] Glue and Hive catalog return only Iceberg tables [iceberg-python]

2024-10-18 Thread via GitHub
sungwy merged PR #1145: URL: https://github.com/apache/iceberg-python/pull/1145 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@icebe

Re: [PR] Build: Bump parquet from 1.13.1 to 1.14.3 [iceberg]

2024-10-18 Thread via GitHub
findepi commented on PR #11264: URL: https://github.com/apache/iceberg/pull/11264#issuecomment-2423244567 thank you @nastra ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] core: use testcontainers-minio [iceberg]

2024-10-18 Thread via GitHub
sullis commented on code in PR #11349: URL: https://github.com/apache/iceberg/pull/11349#discussion_r1807054261 ## aws/src/test/java/org/apache/iceberg/aws/s3/signer/TestS3RestSigner.java: ## @@ -71,11 +73,10 @@ public class TestS3RestSigner { private static final Region RE

Re: [PR] AWS: Introduce opt-in S3LocationProvider which is optimized for S3 performance [iceberg]

2024-10-18 Thread via GitHub
danielcweeks commented on code in PR #2: URL: https://github.com/apache/iceberg/pull/2#discussion_r1806946923 ## core/src/main/java/org/apache/iceberg/LocationProviders.java: ## @@ -172,10 +193,45 @@ private static String pathContext(String tableLocation) { }

Re: [PR] Flink: make FLIP-27 default in SQL and mark the old FlinkSource as deprecated [iceberg]

2024-10-18 Thread via GitHub
stevenzwu commented on code in PR #11345: URL: https://github.com/apache/iceberg/pull/11345#discussion_r1806716122 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/source/IcebergSource.java: ## @@ -86,7 +85,6 @@ import org.slf4j.Logger; import org.slf4j.LoggerFactor

Re: [I] REST: Standardize vended credentials used in loadTable / loadView responses [iceberg]

2024-10-18 Thread via GitHub
nastra closed issue #8: REST: Standardize vended credentials used in loadTable / loadView responses URL: https://github.com/apache/iceberg/issues/8 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] OpenAPI: Standardize credentials in loadTable/loadView responses [iceberg]

2024-10-18 Thread via GitHub
nastra merged PR #10722: URL: https://github.com/apache/iceberg/pull/10722 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Core: Add credentials to loadTable / loadView responses [iceberg]

2024-10-18 Thread via GitHub
nastra merged PR #11173: URL: https://github.com/apache/iceberg/pull/11173 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [I] [Feature Request] Speed up InspectTable.files() [iceberg-python]

2024-10-18 Thread via GitHub
corleyma commented on issue #1229: URL: https://github.com/apache/iceberg-python/issues/1229#issuecomment-2422931184 > Most of the time is spent processing the manifests record-by-record and converting each record to a dict I haven't looked at this closely, but if memory serves, pyic

Re: [PR] Bump mkdocstrings from 0.26.1 to 0.26.2 [iceberg-python]

2024-10-18 Thread via GitHub
sungwy merged PR #1235: URL: https://github.com/apache/iceberg-python/pull/1235 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@icebe

Re: [PR] Bump mypy-boto3-glue from 1.35.23 to 1.35.25 [iceberg-python]

2024-10-18 Thread via GitHub
sungwy merged PR #1236: URL: https://github.com/apache/iceberg-python/pull/1236 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@icebe

Re: [PR] AWS: Update ObjectStorageLocationProvider hash to optimize for S3 performance [iceberg]

2024-10-18 Thread via GitHub
jackye1995 merged PR #2: URL: https://github.com/apache/iceberg/pull/2 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [PR] AWS: Update ObjectStorageLocationProvider hash to optimize for S3 performance [iceberg]

2024-10-18 Thread via GitHub
jackye1995 commented on PR #2: URL: https://github.com/apache/iceberg/pull/2#issuecomment-2423380108 looks like CI has passed, merging. Thanks for the work and patience @ookumuso , and thanks for the review @danielcweeks ! -- This is an automated message from the Apache Git Servic

Re: [PR] Spec: add variant type [iceberg]

2024-10-18 Thread via GitHub
flyrain commented on code in PR #10831: URL: https://github.com/apache/iceberg/pull/10831#discussion_r1807111582 ## format/spec.md: ## @@ -444,6 +449,9 @@ Sorting floating-point numbers should produce the following behavior: `-NaN` < ` A data or delete file is associated wit

Re: [I] [Feature Request] Speed up InspectTable.files() [iceberg-python]

2024-10-18 Thread via GitHub
kevinjqliu commented on issue #1229: URL: https://github.com/apache/iceberg-python/issues/1229#issuecomment-2423385861 > pyiceberg implements its own avro reader/writer using Cython yep, optimistically build avro decoder, fall back to pure python. See https://github.com/apache/ice

Re: [PR] Spark: support rewrite on specified target branch [iceberg]

2024-10-18 Thread via GitHub
amitgilad3 commented on PR #8797: URL: https://github.com/apache/iceberg/pull/8797#issuecomment-2422284309 Tnx for looking into this @jackye1995 , Will fix all the comment -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[PR] Custom fileio implementation is now in the docs [iceberg-python]

2024-10-18 Thread via GitHub
sikehish opened a new pull request, #1237: URL: https://github.com/apache/iceberg-python/pull/1237 This PR addresses #1233. Do let me know if any changes are to be made. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

Re: [PR] [KafkaConnect] Fix RecordConverter for UUID and Fixed Types [iceberg]

2024-10-18 Thread via GitHub
singhpk234 commented on code in PR #11346: URL: https://github.com/apache/iceberg/pull/11346#discussion_r1806749659 ## orc/src/main/java/org/apache/iceberg/data/orc/GenericOrcWriters.java: ## @@ -101,8 +101,8 @@ public static OrcValueWriter byteBuffers() { return ByteBuffer

Re: [PR] [KafkaConnect] Fix RecordConverter for UUID and Fixed Types [iceberg]

2024-10-18 Thread via GitHub
singhpk234 commented on code in PR #11346: URL: https://github.com/apache/iceberg/pull/11346#discussion_r1806695581 ## kafka-connect/kafka-connect/src/test/java/org/apache/iceberg/connect/data/RecordConverterTest.java: ## @@ -921,11 +929,24 @@ private void assertRecordValues(Rec

Re: [PR] docs/configuration.md: Documented table properties (#1231) [iceberg-python]

2024-10-18 Thread via GitHub
mths1 commented on PR #1232: URL: https://github.com/apache/iceberg-python/pull/1232#issuecomment-2422796878 Hi all I was trying 'target-file-size-bytes' lately, and to my understanding in the pyiceberg version we were using, it somehow violates the principle of least surprise. As f

[PR] Custom fileio docs [iceberg-python]

2024-10-18 Thread via GitHub
sikehish opened a new pull request, #1238: URL: https://github.com/apache/iceberg-python/pull/1238 This PR addresses https://github.com/apache/iceberg-python/issues/1233. Do let me know if any changes are to be made. -- This is an automated message from the Apache Git Service. To respon

Re: [I] Document Custom FileIO [iceberg-python]

2024-10-18 Thread via GitHub
sikehish commented on issue #1233: URL: https://github.com/apache/iceberg-python/issues/1233#issuecomment-2422849621 > Assigned to you. I think we can add it under [the `FileIO` section](https://py.iceberg.apache.org/configuration/#fileio) as something like "Custom FileIO Implementations"

Re: [PR] Flink: disable the flaky range distribution bucketing tests for now [iceberg]

2024-10-18 Thread via GitHub
stevenzwu merged PR #11347: URL: https://github.com/apache/iceberg/pull/11347 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [PR] Spark: Add RewriteTablePath action interface [iceberg]

2024-10-18 Thread via GitHub
szehon-ho merged PR #10920: URL: https://github.com/apache/iceberg/pull/10920 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [PR] Spark: Add RewriteTablePath action interface [iceberg]

2024-10-18 Thread via GitHub
szehon-ho commented on PR #10920: URL: https://github.com/apache/iceberg/pull/10920#issuecomment-2422978173 Merged , thanks @laithalzyoud -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

Re: [PR] AWS: Introduce opt-in S3LocationProvider which is optimized for S3 performance [iceberg]

2024-10-18 Thread via GitHub
danielcweeks commented on code in PR #2: URL: https://github.com/apache/iceberg/pull/2#discussion_r1806878880 ## core/src/main/java/org/apache/iceberg/LocationProviders.java: ## @@ -108,10 +108,15 @@ public String newDataLocation(String filename) { static class Object

Re: [PR] AWS: Introduce opt-in S3LocationProvider which is optimized for S3 performance [iceberg]

2024-10-18 Thread via GitHub
danielcweeks commented on code in PR #2: URL: https://github.com/apache/iceberg/pull/2#discussion_r1806873426 ## core/src/main/java/org/apache/iceberg/LocationProviders.java: ## @@ -172,10 +193,45 @@ private static String pathContext(String tableLocation) { }

Re: [PR] AWS: Introduce opt-in S3LocationProvider which is optimized for S3 performance [iceberg]

2024-10-18 Thread via GitHub
danielcweeks commented on code in PR #2: URL: https://github.com/apache/iceberg/pull/2#discussion_r1806873426 ## core/src/main/java/org/apache/iceberg/LocationProviders.java: ## @@ -172,10 +193,45 @@ private static String pathContext(String tableLocation) { }

Re: [PR] Spark: Add RewriteTablePath action interface [iceberg]

2024-10-18 Thread via GitHub
flyrain commented on PR #10920: URL: https://github.com/apache/iceberg/pull/10920#issuecomment-2423035986 Great! Thanks a lot for working on it, guys! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] Custom fileio implementation is now in the docs [iceberg-python]

2024-10-18 Thread via GitHub
sikehish closed pull request #1237: Custom fileio implementation is now in the docs URL: https://github.com/apache/iceberg-python/pull/1237 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

Re: [PR] Spark: Add RewriteTablePath action interface [iceberg]

2024-10-18 Thread via GitHub
laithalzyoud commented on PR #10920: URL: https://github.com/apache/iceberg/pull/10920#issuecomment-2422990247 Thanks @szehon-ho! Will follow up with the implementation PR soon 👍 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

Re: [PR] Spark 3.5: Fix NotSerializableException when migrating partitioned Spark tables [iceberg]

2024-10-18 Thread via GitHub
manuzhang commented on code in PR #11157: URL: https://github.com/apache/iceberg/pull/11157#discussion_r1806768662 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/SparkTableUtil.java: ## @@ -711,7 +719,7 @@ public static void importSparkPartitions( spec,

Re: [PR] Glue and Hive catalog return only Iceberg tables [iceberg-python]

2024-10-18 Thread via GitHub
mark-major commented on code in PR #1145: URL: https://github.com/apache/iceberg-python/pull/1145#discussion_r1806419626 ## pyiceberg/catalog/dynamodb.py: ## @@ -393,7 +393,7 @@ def drop_namespace(self, namespace: Union[str, Identifier]) -> None: raise NoSuchNamesp

Re: [PR] Spec: Support geo type [iceberg]

2024-10-18 Thread via GitHub
desruisseaux commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1806416230 ## format/spec.md: ## @@ -198,6 +199,9 @@ Notes: - Timestamp values _with time zone_ represent a point in time: values are stored as UTC and do not retain

Re: [PR] Flink: make FLIP-27 default in SQL and mark the old FlinkSource as deprecated [iceberg]

2024-10-18 Thread via GitHub
stevenzwu commented on PR #11345: URL: https://github.com/apache/iceberg/pull/11345#issuecomment-2422803067 thanks @pvary for the review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

Re: [PR] Flink: make FLIP-27 default in SQL and mark the old FlinkSource as deprecated [iceberg]

2024-10-18 Thread via GitHub
stevenzwu merged PR #11345: URL: https://github.com/apache/iceberg/pull/11345 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [PR] feat(table): Implement converting Iceberg schema and types to Arrow [iceberg-go]

2024-10-18 Thread via GitHub
zeroshade commented on PR #168: URL: https://github.com/apache/iceberg-go/pull/168#issuecomment-2422814408 @nastra rebased! Thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

Re: [PR] AWS: Introduce opt-in S3LocationProvider which is optimized for S3 performance [iceberg]

2024-10-18 Thread via GitHub
danielcweeks commented on code in PR #2: URL: https://github.com/apache/iceberg/pull/2#discussion_r1806946923 ## core/src/main/java/org/apache/iceberg/LocationProviders.java: ## @@ -172,10 +193,45 @@ private static String pathContext(String tableLocation) { }

Re: [PR] AWS: Introduce opt-in S3LocationProvider which is optimized for S3 performance [iceberg]

2024-10-18 Thread via GitHub
ookumuso commented on code in PR #2: URL: https://github.com/apache/iceberg/pull/2#discussion_r1806988984 ## core/src/main/java/org/apache/iceberg/LocationProviders.java: ## @@ -172,10 +193,45 @@ private static String pathContext(String tableLocation) { } pri

Re: [PR] AWS: Introduce opt-in S3LocationProvider which is optimized for S3 performance [iceberg]

2024-10-18 Thread via GitHub
ookumuso commented on code in PR #2: URL: https://github.com/apache/iceberg/pull/2#discussion_r1806988739 ## core/src/main/java/org/apache/iceberg/LocationProviders.java: ## @@ -108,10 +108,15 @@ public String newDataLocation(String filename) { static class ObjectStor

Re: [PR] Flink: Add RowConverter for Iceberg Source [iceberg]

2024-10-18 Thread via GitHub
abharath9 commented on code in PR #11301: URL: https://github.com/apache/iceberg/pull/11301#discussion_r1807179938 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/source/reader/RowConverter.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache Software Foundation

[PR] Build: Bump parquet from 1.13.1 to 1.14.3 [iceberg]

2024-10-18 Thread via GitHub
dependabot[bot] opened a new pull request, #11264: URL: https://github.com/apache/iceberg/pull/11264 Bumps `parquet` from 1.13.1 to 1.14.3. Updates `org.apache.parquet:parquet-avro` from 1.13.1 to 1.14.3 Release notes Sourced from https://github.com/apache/parquet-mr/releases";>o

Re: [PR] Spec: Support geo type [iceberg]

2024-10-18 Thread via GitHub
dmeaux commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1805976050 ## format/spec.md: ## @@ -483,6 +485,8 @@ Notes: 2. For `float` and `double`, the value `-0.0` must precede `+0.0`, as in the IEEE 754 `totalOrder` predicate. NaNs a

Re: [PR] Build: Bump parquet from 1.13.1 to 1.14.3 [iceberg]

2024-10-18 Thread via GitHub
nastra closed pull request #11264: Build: Bump parquet from 1.13.1 to 1.14.3 URL: https://github.com/apache/iceberg/pull/11264 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

Re: [PR] Core: delete temp metadata file when version already exists [iceberg]

2024-10-18 Thread via GitHub
nastra commented on code in PR #11350: URL: https://github.com/apache/iceberg/pull/11350#discussion_r1806163830 ## core/src/main/java/org/apache/iceberg/hadoop/HadoopTableOperations.java: ## @@ -368,7 +368,13 @@ private void renameToFinal(FileSystem fs, Path src, Path dst, int

Re: [PR] Build: Bump parquet from 1.13.1 to 1.14.3 [iceberg]

2024-10-18 Thread via GitHub
nastra commented on code in PR #11264: URL: https://github.com/apache/iceberg/pull/11264#discussion_r1806145764 ## flink/v1.18/flink/src/test/java/org/apache/iceberg/flink/source/TestMetadataTableReadableMetrics.java: ## @@ -217,27 +217,27 @@ public void testPrimitiveColumns() t

Re: [PR] Revert "Support wasb[s] paths in ADLSFileIO" [iceberg]

2024-10-18 Thread via GitHub
nastra merged PR #11344: URL: https://github.com/apache/iceberg/pull/11344 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [I] Add SparkSessionCatalog support for views [iceberg]

2024-10-18 Thread via GitHub
nqvuong1998 commented on issue #9845: URL: https://github.com/apache/iceberg/issues/9845#issuecomment-2421995463 Hi @nastra , any updates for this issue? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [I] Lack of integration tests for SQL catalog implementation [iceberg-rust]

2024-10-18 Thread via GitHub
liurenjie1024 commented on issue #673: URL: https://github.com/apache/iceberg-rust/issues/673#issuecomment-2422012602 Thanks @callum-ryan ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

[PR] Core: code clean up for deleteRemovedMetadataFiles [iceberg]

2024-10-18 Thread via GitHub
leesf opened a new pull request, #11352: URL: https://github.com/apache/iceberg/pull/11352 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-ma

Re: [PR] Core: code clean up for deleteRemovedMetadataFiles [iceberg]

2024-10-18 Thread via GitHub
nastra commented on code in PR #11352: URL: https://github.com/apache/iceberg/pull/11352#discussion_r1806159950 ## core/src/main/java/org/apache/iceberg/TableMetadata.java: ## @@ -77,6 +83,51 @@ public static TableMetadata newTableMetadata( return newTableMetadata(schema, s

Re: [PR] core: use testcontainers-minio [iceberg]

2024-10-18 Thread via GitHub
nastra commented on code in PR #11349: URL: https://github.com/apache/iceberg/pull/11349#discussion_r1806186638 ## aws/src/test/java/org/apache/iceberg/aws/s3/signer/TestS3RestSigner.java: ## @@ -71,11 +73,10 @@ public class TestS3RestSigner { private static final Region RE

Re: [PR] core: use testcontainers-minio [iceberg]

2024-10-18 Thread via GitHub
nastra commented on code in PR #11349: URL: https://github.com/apache/iceberg/pull/11349#discussion_r1806186638 ## aws/src/test/java/org/apache/iceberg/aws/s3/signer/TestS3RestSigner.java: ## @@ -71,11 +73,10 @@ public class TestS3RestSigner { private static final Region RE

Re: [PR] Spec: Support geo type [iceberg]

2024-10-18 Thread via GitHub
desruisseaux commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1806236061 ## format/spec.md: ## @@ -198,6 +199,9 @@ Notes: - Timestamp values _with time zone_ represent a point in time: values are stored as UTC and do not retain

Re: [PR] Core: code clean up for deleteRemovedMetadataFiles [iceberg]

2024-10-18 Thread via GitHub
leesf commented on code in PR #11352: URL: https://github.com/apache/iceberg/pull/11352#discussion_r1806236088 ## core/src/main/java/org/apache/iceberg/TableMetadata.java: ## @@ -77,6 +83,51 @@ public static TableMetadata newTableMetadata( return newTableMetadata(schema, sp

Re: [PR] Spec: Support geo type [iceberg]

2024-10-18 Thread via GitHub
desruisseaux commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1806246237 ## format/spec.md: ## @@ -200,12 +200,16 @@ Supported primitive types are defined in the table below. Primitive types added | | **`uuid`**

Re: [PR] Core: delete temp metadata file when version already exists [iceberg]

2024-10-18 Thread via GitHub
leesf commented on code in PR #11350: URL: https://github.com/apache/iceberg/pull/11350#discussion_r1806229187 ## core/src/main/java/org/apache/iceberg/hadoop/HadoopTableOperations.java: ## @@ -368,7 +368,13 @@ private void renameToFinal(FileSystem fs, Path src, Path dst, int n

Re: [PR] Spec: Support geo type [iceberg]

2024-10-18 Thread via GitHub
desruisseaux commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1806278747 ## format/spec.md: ## @@ -483,6 +485,8 @@ Notes: 2. For `float` and `double`, the value `-0.0` must precede `+0.0`, as in the IEEE 754 `totalOrder` predicate.

Re: [PR] Spec: Support geo type [iceberg]

2024-10-18 Thread via GitHub
desruisseaux commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1806309872 ## format/spec.md: ## @@ -483,6 +485,8 @@ Notes: 2. For `float` and `double`, the value `-0.0` must precede `+0.0`, as in the IEEE 754 `totalOrder` predicate.

Re: [PR] Core: code clean up for deleteRemovedMetadataFiles [iceberg]

2024-10-18 Thread via GitHub
nastra commented on code in PR #11352: URL: https://github.com/apache/iceberg/pull/11352#discussion_r1806309295 ## core/src/main/java/org/apache/iceberg/TableMetadata.java: ## @@ -77,6 +83,51 @@ public static TableMetadata newTableMetadata( return newTableMetadata(schema, s

Re: [PR] Spec: Support geo type [iceberg]

2024-10-18 Thread via GitHub
desruisseaux commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1806309872 ## format/spec.md: ## @@ -483,6 +485,8 @@ Notes: 2. For `float` and `double`, the value `-0.0` must precede `+0.0`, as in the IEEE 754 `totalOrder` predicate.