[I] Discussion: Rethink `PrimitiveLiteral`. [iceberg-rust]

2024-01-08 Thread via GitHub
liurenjie1024 opened a new issue, #159: URL: https://github.com/apache/iceberg-rust/issues/159 In our discussion in #77, we decoupled type info from struct values. I'm also facing some challenge when implementing #153. In short, the `PrimitiveLiteral` is kind of weird: 1. Sometimes i

Re: [PR] Flink: Watermark read options [iceberg]

2024-01-08 Thread via GitHub
pvary commented on code in PR #9346: URL: https://github.com/apache/iceberg/pull/9346#discussion_r1445734747 ## flink/v1.18/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceSql.java: ## @@ -18,25 +18,136 @@ */ package org.apache.iceberg.flink.source; +im

[PR] Spark 3.5: Support specifying filter in RewriteManifestsProcedure [iceberg]

2024-01-08 Thread via GitHub
bknbkn opened a new pull request, #9447: URL: https://github.com/apache/iceberg/pull/9447 This patch add `where` args in `RewritingManifestsProcedure`, It allows us to only rewrite the manifest file of the specified conditions. When we optimize the manifests of a large table, rewriting the

Re: [PR] init writer framework [iceberg-rust]

2024-01-08 Thread via GitHub
liurenjie1024 commented on PR #135: URL: https://github.com/apache/iceberg-rust/pull/135#issuecomment-1882539731 > Sorry for being late. I drew a UML diagram, hope it can make the design clearer. Please let me know if there is something confused. ![image](https://private-user-images.githubu

Re: [PR] Spark: Fix flaky TestSparkReaderDeletes tests due to metric not found [iceberg]

2024-01-08 Thread via GitHub
manuzhang commented on PR #9445: URL: https://github.com/apache/iceberg/pull/9445#issuecomment-1882502379 @ajantha-bhat please help review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

Re: [PR] Add `iceberg-bom` artifact [iceberg]

2024-01-08 Thread via GitHub
ajantha-bhat commented on PR #8065: URL: https://github.com/apache/iceberg/pull/8065#issuecomment-1882502224 We do need to rebase/rework this since spark 3.2 is removed. But overall I feel we should have bom published. So, I will go ahead and add it to 1.5.0 milestone. -- This is

Re: [PR] Spec: add multi-arg transform support [iceberg]

2024-01-08 Thread via GitHub
szehon-ho commented on code in PR #8579: URL: https://github.com/apache/iceberg/pull/8579#discussion_r1445557050 ## format/spec.md: ## @@ -322,14 +322,22 @@ The `void` transform may be used to replace the transform in an existing partiti Bucket Transform Details -Buck

Re: [PR] Spec: add multi-arg transform support [iceberg]

2024-01-08 Thread via GitHub
szehon-ho commented on PR #8579: URL: https://github.com/apache/iceberg/pull/8579#issuecomment-1882496190 Hi, @advancedxy , thanks for the work. Sorry for the delay, I am just returning from paternity leave. Will love to see this get in to get work on zorder and geo-transforms. I left so

Re: [PR] Core: Add alias for glue, nessie, snowflake, jdbc catalog [iceberg]

2024-01-08 Thread via GitHub
ajantha-bhat commented on code in PR #8641: URL: https://github.com/apache/iceberg/pull/8641#discussion_r1445684878 ## core/src/main/java/org/apache/iceberg/CatalogUtil.java: ## @@ -67,10 +67,18 @@ public class CatalogUtil { public static final String ICEBERG_CATALOG_TYPE_HAD

Re: [PR] Core: Add alias for glue, nessie, snowflake, jdbc catalog [iceberg]

2024-01-08 Thread via GitHub
ajantha-bhat commented on code in PR #8641: URL: https://github.com/apache/iceberg/pull/8641#discussion_r1445684406 ## core/src/main/java/org/apache/iceberg/CatalogUtil.java: ## @@ -67,10 +67,18 @@ public class CatalogUtil { public static final String ICEBERG_CATALOG_TYPE_HAD

Re: [PR] Spec: add multi-arg transform support [iceberg]

2024-01-08 Thread via GitHub
szehon-ho commented on code in PR #8579: URL: https://github.com/apache/iceberg/pull/8579#discussion_r1445563921 ## format/spec.md: ## @@ -322,14 +322,22 @@ The `void` transform may be used to replace the transform in an existing partiti Bucket Transform Details -Buck

Re: [PR] Hive: Unwrap RuntimeException for Hive InvalidOperationException with rename table [iceberg]

2024-01-08 Thread via GitHub
nk1506 commented on code in PR #9432: URL: https://github.com/apache/iceberg/pull/9432#discussion_r1445621596 ## hive-metastore/src/main/java/org/apache/iceberg/hive/MetastoreUtil.java: ## @@ -72,6 +73,23 @@ public static void alterTable( env.putAll(extraEnv); env.put(

Re: [PR] Homepage added to Cargo.toml [iceberg-rust]

2024-01-08 Thread via GitHub
liurenjie1024 commented on PR #157: URL: https://github.com/apache/iceberg-rust/pull/157#issuecomment-1882403476 > Should i follow the exact same approach for all of them , in essence referring them from workspace Cargo.toml? Yes, thanks! -- This is an automated message from the Ap

Re: [PR] Homepage added to Cargo.toml [iceberg-rust]

2024-01-08 Thread via GitHub
liurenjie1024 commented on PR #157: URL: https://github.com/apache/iceberg-rust/pull/157#issuecomment-1882403624 > I will start doing those other packages asap , but i think i should do that in a different pr as too much modified code in one pr might not be ideal . Please tell me how you th

Re: [PR] Homepage added to Cargo.toml [iceberg-rust]

2024-01-08 Thread via GitHub
hiirrxnn commented on PR #157: URL: https://github.com/apache/iceberg-rust/pull/157#issuecomment-1882388296 Should i follow the exact same approach for all of them , in essence referring them from workspace Cargo.toml? -- This is an automated message from the Apache Git Service. To re

Re: [PR] Homepage added to Cargo.toml [iceberg-rust]

2024-01-08 Thread via GitHub
hiirrxnn commented on PR #157: URL: https://github.com/apache/iceberg-rust/pull/157#issuecomment-1882387296 I will start doing those other packages asap , but i think i should do that in a different pr as too much modified code in one pr might not be ideal . Please tell me how you think abo

Re: [PR] Spark: Fix flaky TestSparkReaderDeletes tests due to metric not found [iceberg]

2024-01-08 Thread via GitHub
manuzhang commented on code in PR #9445: URL: https://github.com/apache/iceberg/pull/9445#discussion_r1445607629 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestSparkReaderDeletes.java: ## @@ -126,6 +126,7 @@ public static void startMetastoreAndSpark() {

Re: [I] Snowflake Iceberg Partitioned data read issue [iceberg]

2024-01-08 Thread via GitHub
rakesh-das08 commented on issue #9404: URL: https://github.com/apache/iceberg/issues/9404#issuecomment-1882331165 We also kind of faced this issue where when we try to create a new Iceberg table using snowflake as the catalog, using the provided syntax here : https://docs.snowflake.com/en/s

Re: [PR] Homepage added to Cargo.toml [iceberg-rust]

2024-01-08 Thread via GitHub
liurenjie1024 commented on PR #157: URL: https://github.com/apache/iceberg-rust/pull/157#issuecomment-1882312785 > I have done that ! Is that what you meant ? Let me know if there's something wrong ! Yeah, exactly, thanks! There are also several fields which are shared among al

[I] Caused by: java.net.SocketException: Connection reset [iceberg]

2024-01-08 Thread via GitHub
javrasya opened a new issue, #9444: URL: https://github.com/apache/iceberg/issues/9444 ### Apache Iceberg version 1.4.2 (latest release) ### Query engine Flink ### Please describe the bug 🐞 I am using Flink version 1.15 and trying to consume from an Iceberg

Re: [PR] Homepage added to Cargo.toml [iceberg-rust]

2024-01-08 Thread via GitHub
hiirrxnn commented on PR #157: URL: https://github.com/apache/iceberg-rust/pull/157#issuecomment-1882264839 I have done that ! Is that what you meant ? Let me know if there's something wrong ! -- This is an automated message from the Apache Git Service. To respond to the message, please l

Re: [I] How to move Iceberg table from one location to another [iceberg]

2024-01-08 Thread via GitHub
meicanhong commented on issue #3142: URL: https://github.com/apache/iceberg/issues/3142#issuecomment-1882206973 @atifiu I didn't solve this problem. For a discussion on this issue, check out this issue https://github.com/apache/iceberg/issues/1617 -- This is an automated message from the

[I] refactor: Remove support of manifest list format as a list of file paths. [iceberg-rust]

2024-01-08 Thread via GitHub
liurenjie1024 opened a new issue, #158: URL: https://github.com/apache/iceberg-rust/issues/158 Do you think it's a good idea to check this first so we don't need to write in enum pattern matching? _Originally posted by @Xuanwo in https://github.com/apache/iceberg-rust/p

Re: [PR] feat: Introduce basic file scan planning. [iceberg-rust]

2024-01-08 Thread via GitHub
liurenjie1024 commented on code in PR #129: URL: https://github.com/apache/iceberg-rust/pull/129#discussion_r1445553493 ## crates/iceberg/src/spec/snapshot.rs: ## @@ -124,6 +150,70 @@ impl Snapshot { Utc.timestamp_millis_opt(self.timestamp_ms).unwrap() } +///

Re: [PR] feat: Introduce basic file scan planning. [iceberg-rust]

2024-01-08 Thread via GitHub
liurenjie1024 commented on code in PR #129: URL: https://github.com/apache/iceberg-rust/pull/129#discussion_r1445554235 ## crates/iceberg/src/spec/snapshot.rs: ## @@ -124,6 +150,70 @@ impl Snapshot { Utc.timestamp_millis_opt(self.timestamp_ms).unwrap() } +///

Re: [PR] feat: Introduce basic file scan planning. [iceberg-rust]

2024-01-08 Thread via GitHub
liurenjie1024 commented on code in PR #129: URL: https://github.com/apache/iceberg-rust/pull/129#discussion_r1445553809 ## crates/iceberg/src/scan.rs: ## @@ -0,0 +1,448 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements.

Re: [PR] feat: Introduce basic file scan planning. [iceberg-rust]

2024-01-08 Thread via GitHub
liurenjie1024 commented on code in PR #129: URL: https://github.com/apache/iceberg-rust/pull/129#discussion_r1445553493 ## crates/iceberg/src/spec/snapshot.rs: ## @@ -124,6 +150,70 @@ impl Snapshot { Utc.timestamp_millis_opt(self.timestamp_ms).unwrap() } +///

Re: [PR] Write support [iceberg-python]

2024-01-08 Thread via GitHub
syun64 commented on code in PR #41: URL: https://github.com/apache/iceberg-python/pull/41#discussion_r1445440825 ## pyiceberg/table/snapshots.py: ## @@ -277,23 +278,30 @@ def _truncate_table_summary(summary: Summary, previous_summary: Mapping[str, str }: summary[p

Re: [PR] feat: Introduce basic file scan planning. [iceberg-rust]

2024-01-08 Thread via GitHub
Xuanwo commented on code in PR #129: URL: https://github.com/apache/iceberg-rust/pull/129#discussion_r1445548092 ## crates/iceberg/src/spec/snapshot.rs: ## @@ -124,6 +150,70 @@ impl Snapshot { Utc.timestamp_millis_opt(self.timestamp_ms).unwrap() } +/// Get th

Re: [PR] Core: Remove deprecated method from BaseMetadataTable [iceberg]

2024-01-08 Thread via GitHub
ajantha-bhat commented on PR #9298: URL: https://github.com/apache/iceberg/pull/9298#issuecomment-1882157466 PR is ready. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

Re: [PR] feat: Introduce basic file scan planning. [iceberg-rust]

2024-01-08 Thread via GitHub
ZENOTME commented on code in PR #129: URL: https://github.com/apache/iceberg-rust/pull/129#discussion_r1445545384 ## crates/iceberg/src/scan.rs: ## @@ -0,0 +1,448 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See t

Re: [I] Remove deprecated import [iceberg-python]

2024-01-08 Thread via GitHub
Fokko closed issue #221: Remove deprecated import URL: https://github.com/apache/iceberg-python/issues/221 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mai

Re: [I] Remove deprecated import [iceberg-python]

2024-01-08 Thread via GitHub
Fokko commented on issue #221: URL: https://github.com/apache/iceberg-python/issues/221#issuecomment-1882136902 This is actually in Spark, and you're right @Rohit-Pujari, I think we can close it. Thanks for checking -- This is an automated message from the Apache Git Service. To respond t

Re: [PR] feat: Introduce basic file scan planning. [iceberg-rust]

2024-01-08 Thread via GitHub
liurenjie1024 commented on code in PR #129: URL: https://github.com/apache/iceberg-rust/pull/129#discussion_r1445538967 ## crates/iceberg/src/scan.rs: ## @@ -0,0 +1,448 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements.

Re: [PR] Homepage added to Cargo.toml [iceberg-rust]

2024-01-08 Thread via GitHub
liurenjie1024 commented on PR #157: URL: https://github.com/apache/iceberg-rust/pull/157#issuecomment-1882123647 Hi, @hiirrxnn There is another approach for this, please follow these steps: 1. You can add the home page info in `Cargo.toml` of workspace. 2. Then in `Cargo.toml` of ea

Re: [I] Add the homepage to the `Cargo.toml` [iceberg-rust]

2024-01-08 Thread via GitHub
liurenjie1024 commented on issue #154: URL: https://github.com/apache/iceberg-rust/issues/154#issuecomment-1882109553 > Just wanted to confirm , is : https://iceberg.apache.org the homepage i'm required to add ? Correct me if I'm wrong . No, it's https://rust.iceberg.apache.org/ --

Re: [PR] Build: Bump com.fasterxml.jackson.core:jackson-annotations from 2.16.0 to 2.16.1 [iceberg]

2024-01-08 Thread via GitHub
amogh-jahagirdar commented on PR #9375: URL: https://github.com/apache/iceberg/pull/9375#issuecomment-1882103658 @dependabot rebase -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

Re: [I] Capture successful snapshot expiration deletion volume [iceberg]

2024-01-08 Thread via GitHub
puchengy closed issue #9443: Capture successful snapshot expiration deletion volume URL: https://github.com/apache/iceberg/issues/9443 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

Re: [PR] Build: Bump com.fasterxml.jackson.dataformat:jackson-dataformat-xml from 2.16.0 to 2.16.1 [iceberg]

2024-01-08 Thread via GitHub
amogh-jahagirdar merged PR #9395: URL: https://github.com/apache/iceberg/pull/9395 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [PR] Build: Bump software.amazon.awssdk:bom from 2.21.42 to 2.22.12 [iceberg]

2024-01-08 Thread via GitHub
amogh-jahagirdar merged PR #9426: URL: https://github.com/apache/iceberg/pull/9426 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [I] Review Table/View refresh Operation for InMemoryCatalog [iceberg]

2024-01-08 Thread via GitHub
nastra commented on issue #9434: URL: https://github.com/apache/iceberg/issues/9434#issuecomment-1882064663 @nk1506 I'm not sure I fully understand what the scope of this issue is. I think what the `InMemoryCatalog` currently does is correct. Please re-open if you think this needs to be add

Re: [I] Review Table/View refresh Operation for InMemoryCatalog [iceberg]

2024-01-08 Thread via GitHub
nastra closed issue #9434: Review Table/View refresh Operation for InMemoryCatalog URL: https://github.com/apache/iceberg/issues/9434 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

Re: [PR] Build: Bump com.diffplug.spotless:spotless-plugin-gradle from 6.13.0 to 6.23.3 [iceberg]

2024-01-08 Thread via GitHub
nastra closed pull request #9262: Build: Bump com.diffplug.spotless:spotless-plugin-gradle from 6.13.0 to 6.23.3 URL: https://github.com/apache/iceberg/pull/9262 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

Re: [PR] Build: Bump com.diffplug.spotless:spotless-plugin-gradle from 6.13.0 to 6.23.3 [iceberg]

2024-01-08 Thread via GitHub
nastra commented on PR #9262: URL: https://github.com/apache/iceberg/pull/9262#issuecomment-1882055910 Closing as this requires JDK11 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] Build: Bump com.diffplug.spotless:spotless-plugin-gradle from 6.13.0 to 6.23.3 [iceberg]

2024-01-08 Thread via GitHub
dependabot[bot] commented on PR #9262: URL: https://github.com/apache/iceberg/pull/9262#issuecomment-1882055983 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version, let

Re: [PR] Build: Bump io.netty:netty-buffer from 4.1.68.Final to 4.1.104.Final [iceberg]

2024-01-08 Thread via GitHub
dependabot[bot] commented on PR #9315: URL: https://github.com/apache/iceberg/pull/9315#issuecomment-1882053872 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version, let

Re: [PR] Build: Bump io.netty:netty-buffer from 4.1.68.Final to 4.1.104.Final [iceberg]

2024-01-08 Thread via GitHub
nastra closed pull request #9315: Build: Bump io.netty:netty-buffer from 4.1.68.Final to 4.1.104.Final URL: https://github.com/apache/iceberg/pull/9315 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

Re: [PR] Build: Bump org.springframework:spring-web from 5.3.30 to 6.1.2 [iceberg]

2024-01-08 Thread via GitHub
nastra commented on PR #9316: URL: https://github.com/apache/iceberg/pull/9316#issuecomment-1882052644 Closing as this requires JDK17 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] Build: Bump org.springframework:spring-web from 5.3.30 to 6.1.2 [iceberg]

2024-01-08 Thread via GitHub
nastra closed pull request #9316: Build: Bump org.springframework:spring-web from 5.3.30 to 6.1.2 URL: https://github.com/apache/iceberg/pull/9316 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] Build: Bump org.springframework:spring-web from 5.3.30 to 6.1.2 [iceberg]

2024-01-08 Thread via GitHub
dependabot[bot] commented on PR #9316: URL: https://github.com/apache/iceberg/pull/9316#issuecomment-1882052710 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version, let

Re: [PR] Adding Junit 5 conversion and AssertJ style for TestFlinkCatalogTable… [iceberg]

2024-01-08 Thread via GitHub
nastra commented on PR #9381: URL: https://github.com/apache/iceberg/pull/9381#issuecomment-188203 > I have done changes in TestStreamScanSql locally but running testcases on my local machine is taking lot of time. It is running indefinitely for this class. @nastra Do you have any idea

Re: [I] [Rest Catalog Open API] Usage of "oneof" in the definition [iceberg]

2024-01-08 Thread via GitHub
github-actions[bot] commented on issue #6798: URL: https://github.com/apache/iceberg/issues/6798#issuecomment-1882020405 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] Exceptions when FlinkSQL writes to partitioned tables [iceberg]

2024-01-08 Thread via GitHub
github-actions[bot] commented on issue #7618: URL: https://github.com/apache/iceberg/issues/7618#issuecomment-1882020269 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] Consolidate file path creation to LocationProvider [iceberg]

2024-01-08 Thread via GitHub
github-actions[bot] closed issue #6809: Consolidate file path creation to LocationProvider URL: https://github.com/apache/iceberg/issues/6809 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

Re: [I] Consolidate file path creation to LocationProvider [iceberg]

2024-01-08 Thread via GitHub
github-actions[bot] commented on issue #6809: URL: https://github.com/apache/iceberg/issues/6809#issuecomment-1882020379 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [PR] Build: Bump com.palantir.baseline:gradle-baseline-java from 4.42.0 to 5.32.0 [iceberg]

2024-01-08 Thread via GitHub
dependabot[bot] commented on PR #9428: URL: https://github.com/apache/iceberg/pull/9428#issuecomment-1882015202 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version, let

Re: [PR] Build: Bump com.palantir.baseline:gradle-baseline-java from 4.42.0 to 5.32.0 [iceberg]

2024-01-08 Thread via GitHub
nastra commented on PR #9428: URL: https://github.com/apache/iceberg/pull/9428#issuecomment-1882015175 Closing as this requires a newer JDK version -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Build: Bump com.palantir.baseline:gradle-baseline-java from 4.42.0 to 5.32.0 [iceberg]

2024-01-08 Thread via GitHub
nastra closed pull request #9428: Build: Bump com.palantir.baseline:gradle-baseline-java from 4.42.0 to 5.32.0 URL: https://github.com/apache/iceberg/pull/9428 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] Homepage added to Cargo.toml [iceberg-rust]

2024-01-08 Thread via GitHub
hiirrxnn commented on PR #157: URL: https://github.com/apache/iceberg-rust/pull/157#issuecomment-1882014699 Is it alright now ? I've done it in the way mentioned by the doc referenced here . Let me know if something has to be changed here . -- This is an automated message from the Apache

Re: [PR] Build: Bump com.adobe.testing:s3mock-junit5 from 2.11.0 to 3.3.0 [iceberg]

2024-01-08 Thread via GitHub
nastra commented on PR #9394: URL: https://github.com/apache/iceberg/pull/9394#issuecomment-1882013401 closing as this requires JDK17 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] Build: Bump com.adobe.testing:s3mock-junit5 from 2.11.0 to 3.3.0 [iceberg]

2024-01-08 Thread via GitHub
nastra closed pull request #9394: Build: Bump com.adobe.testing:s3mock-junit5 from 2.11.0 to 3.3.0 URL: https://github.com/apache/iceberg/pull/9394 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

Re: [PR] Build: Bump com.adobe.testing:s3mock-junit5 from 2.11.0 to 3.3.0 [iceberg]

2024-01-08 Thread via GitHub
dependabot[bot] commented on PR #9394: URL: https://github.com/apache/iceberg/pull/9394#issuecomment-1882013438 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version, let

Re: [I] java.lang.IllegalStateException: Connection pool shut down when refreshing table metadata on s3 [iceberg]

2024-01-08 Thread via GitHub
javrasya commented on issue #8601: URL: https://github.com/apache/iceberg/issues/8601#issuecomment-1881985061 That is great. But why not it won't be also supported for 1.15? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] Flink: Fix IcebergSource tableloader lifecycle management in batch mode [iceberg]

2024-01-08 Thread via GitHub
javrasya commented on PR #9173: URL: https://github.com/apache/iceberg/pull/9173#issuecomment-1881978604 Why not supporting this for 1.15? Unfortunately that is still the latest version one can use on managed AWS Flink service. -- This is an automated message from the Apache Git Service.

[PR] Build: Bump moto from 4.2.12 to 4.2.13 [iceberg-python]

2024-01-08 Thread via GitHub
dependabot[bot] opened a new pull request, #257: URL: https://github.com/apache/iceberg-python/pull/257 Bumps [moto](https://github.com/getmoto/moto) from 4.2.12 to 4.2.13. Changelog Sourced from https://github.com/getmoto/moto/blob/master/CHANGELOG.md";>moto's changelog. 4.

[PR] Build: Bump mkdocstrings-python from 1.7.5 to 1.8.0 [iceberg-python]

2024-01-08 Thread via GitHub
dependabot[bot] opened a new pull request, #256: URL: https://github.com/apache/iceberg-python/pull/256 Bumps [mkdocstrings-python](https://github.com/mkdocstrings/python) from 1.7.5 to 1.8.0. Release notes Sourced from https://github.com/mkdocstrings/python/releases";>mkdocstrings

Re: [PR] Write support [iceberg-python]

2024-01-08 Thread via GitHub
syun64 commented on code in PR #41: URL: https://github.com/apache/iceberg-python/pull/41#discussion_r1445440825 ## pyiceberg/table/snapshots.py: ## @@ -277,23 +278,30 @@ def _truncate_table_summary(summary: Summary, previous_summary: Mapping[str, str }: summary[p

Re: [PR] Spark 3.5: Migrate tests that depend on SparkDistributedDataScanTestBase to JUnit5 [iceberg]

2024-01-08 Thread via GitHub
nastra commented on code in PR #9416: URL: https://github.com/apache/iceberg/pull/9416#discussion_r1445397133 ## core/src/test/java/org/apache/iceberg/ScanTestBase.java: ## @@ -19,114 +19,104 @@ package org.apache.iceberg; import static org.apache.iceberg.types.Types.NestedF

Re: [PR] Flink: Watermark read options [iceberg]

2024-01-08 Thread via GitHub
stevenzwu commented on code in PR #9346: URL: https://github.com/apache/iceberg/pull/9346#discussion_r1445375116 ## flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/source/ScanContext.java: ## @@ -272,6 +279,14 @@ public int maxAllowedPlanningFailures() { return max

Re: [PR] Flink: Watermark read options [iceberg]

2024-01-08 Thread via GitHub
stevenzwu commented on code in PR #9346: URL: https://github.com/apache/iceberg/pull/9346#discussion_r1445324248 ## flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/FlinkReadOptions.java: ## @@ -109,4 +110,14 @@ private FlinkReadOptions() {} public static final String

Re: [I] Expose PyIceberg table as PyArrow Dataset [iceberg-python]

2024-01-08 Thread via GitHub
stefnba commented on issue #30: URL: https://github.com/apache/iceberg-python/issues/30#issuecomment-1881808334 Hi, is there any update on this topic? Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

Re: [PR] Core: Iceberg streaming streaming-skip-overwrite-snapshots SparkMicroBatchStream only skips over one file per trigger [iceberg]

2024-01-08 Thread via GitHub
singhpk234 commented on PR #8980: URL: https://github.com/apache/iceberg/pull/8980#issuecomment-1881801609 > I'm also very skeptical of counting the added files, and I think we might want to remove that piece of logic (in a separate PR). +1 on this @Fokko other than this no further co

Re: [PR] Flink: Watermark read options [iceberg]

2024-01-08 Thread via GitHub
rodmeneses commented on code in PR #9346: URL: https://github.com/apache/iceberg/pull/9346#discussion_r1445280024 ## flink/v1.18/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceSql.java: ## @@ -39,4 +62,76 @@ public void before() throws IOException {

Re: [PR] Spark 3.5: Migrate remaining tests in source directory to JUnit5 [iceberg]

2024-01-08 Thread via GitHub
nastra commented on code in PR #9380: URL: https://github.com/apache/iceberg/pull/9380#discussion_r1445308708 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestSparkReaderDeletes.java: ## @@ -81,32 +86,29 @@ import org.apache.spark.sql.internal.SQLConf; imp

Re: [PR] Spark 3.5: Migrate remaining tests in source directory to JUnit5 [iceberg]

2024-01-08 Thread via GitHub
nastra commented on code in PR #9380: URL: https://github.com/apache/iceberg/pull/9380#discussion_r1445307042 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestSparkReadProjection.java: ## @@ -110,10 +110,10 @@ public static void stopSpark() { @Override

Re: [PR] Spark 3.5: Migrate remaining tests in source directory to JUnit5 [iceberg]

2024-01-08 Thread via GitHub
nastra commented on code in PR #9380: URL: https://github.com/apache/iceberg/pull/9380#discussion_r1445306485 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestSparkReadProjection.java: ## @@ -51,40 +55,36 @@ import org.apache.spark.sql.Dataset; import org.

Re: [PR] Spark 3.5: Migrate remaining tests in source directory to JUnit5 [iceberg]

2024-01-08 Thread via GitHub
nastra commented on code in PR #9380: URL: https://github.com/apache/iceberg/pull/9380#discussion_r1445302815 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestReadProjection.java: ## @@ -36,23 +38,19 @@ import org.apache.iceberg.relocated.com.google.common.

Re: [PR] Spark 3.5: Migrate remaining tests in source directory to JUnit5 [iceberg]

2024-01-08 Thread via GitHub
nastra commented on code in PR #9380: URL: https://github.com/apache/iceberg/pull/9380#discussion_r1445301963 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestPositionDeletesTable.java: ## @@ -1278,10 +1321,10 @@ public void testNormalWritesNotAllowed() thro

Re: [PR] Spark 3.5: Migrate remaining tests in source directory to JUnit5 [iceberg]

2024-01-08 Thread via GitHub
nastra commented on code in PR #9380: URL: https://github.com/apache/iceberg/pull/9380#discussion_r1445301351 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestPositionDeletesTable.java: ## @@ -1278,10 +1321,10 @@ public void testNormalWritesNotAllowed() thro

Re: [PR] Spark 3.5: Migrate remaining tests in source directory to JUnit5 [iceberg]

2024-01-08 Thread via GitHub
nastra commented on code in PR #9380: URL: https://github.com/apache/iceberg/pull/9380#discussion_r1445300841 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestPositionDeletesTable.java: ## @@ -1051,24 +1086,24 @@ public void testWritePartitionEvolutionDisall

Re: [PR] Spark 3.5: Migrate remaining tests in source directory to JUnit5 [iceberg]

2024-01-08 Thread via GitHub
nastra commented on code in PR #9380: URL: https://github.com/apache/iceberg/pull/9380#discussion_r1445299723 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestPositionDeletesTable.java: ## @@ -244,11 +244,14 @@ public void testSelect() throws IOException {

Re: [PR] Spark 3.5: Migrate remaining tests in source directory to JUnit5 [iceberg]

2024-01-08 Thread via GitHub
nastra commented on code in PR #9380: URL: https://github.com/apache/iceberg/pull/9380#discussion_r1445298490 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestPartitionValues.java: ## @@ -102,35 +105,33 @@ public static Object[][] parameters() { private

Re: [PR] Spark 3.5: Migrate remaining tests in source directory to JUnit5 [iceberg]

2024-01-08 Thread via GitHub
nastra commented on code in PR #9380: URL: https://github.com/apache/iceberg/pull/9380#discussion_r1445297500 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestPartitionValues.java: ## @@ -102,35 +105,33 @@ public static Object[][] parameters() { private

Re: [PR] Spark 3.5: Migrate remaining tests in source directory to JUnit5 [iceberg]

2024-01-08 Thread via GitHub
nastra commented on code in PR #9380: URL: https://github.com/apache/iceberg/pull/9380#discussion_r1445292960 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestFilteredScan.java: ## @@ -314,7 +317,7 @@ public void testDayPartitionedTimestampFilters() {

Re: [PR] Spark 3.5: Migrate remaining tests in source directory to JUnit5 [iceberg]

2024-01-08 Thread via GitHub
nastra commented on code in PR #9380: URL: https://github.com/apache/iceberg/pull/9380#discussion_r1445292587 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestFilteredScan.java: ## @@ -286,7 +287,8 @@ public void testBucketPartitionedIDFilters() { Inp

Re: [PR] Spark 3.5: Migrate remaining tests in source directory to JUnit5 [iceberg]

2024-01-08 Thread via GitHub
nastra commented on code in PR #9380: URL: https://github.com/apache/iceberg/pull/9380#discussion_r1445291604 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestCompressionSettings.java: ## @@ -69,66 +75,90 @@ import org.apache.spark.sql.Dataset; import org.

Re: [PR] Spark 3.5: Migrate tests in SQL directory to JUnit5 [iceberg]

2024-01-08 Thread via GitHub
nastra commented on code in PR #9401: URL: https://github.com/apache/iceberg/pull/9401#discussion_r1445284087 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/sql/TestCreateTableAsSelect.java: ## @@ -21,40 +21,64 @@ import static org.apache.spark.sql.functions.col; i

Re: [PR] Spark 3.5: Migrate tests in SQL directory to JUnit5 [iceberg]

2024-01-08 Thread via GitHub
nastra commented on code in PR #9401: URL: https://github.com/apache/iceberg/pull/9401#discussion_r1445283708 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/sql/TestSelect.java: ## @@ -18,70 +18,95 @@ */ package org.apache.iceberg.spark.sql; +import static org.a

Re: [PR] Flink: Watermark read options [iceberg]

2024-01-08 Thread via GitHub
rodmeneses commented on code in PR #9346: URL: https://github.com/apache/iceberg/pull/9346#discussion_r1445280024 ## flink/v1.18/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceSql.java: ## @@ -39,4 +62,76 @@ public void before() throws IOException {

Re: [I] Create JUnit5-version of TableTestBase [iceberg]

2024-01-08 Thread via GitHub
nastra closed issue #9073: Create JUnit5-version of TableTestBase URL: https://github.com/apache/iceberg/issues/9073 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscr

Re: [PR] Core: Add JUnit5 version of TableTestBase [iceberg]

2024-01-08 Thread via GitHub
nastra merged PR #9424: URL: https://github.com/apache/iceberg/pull/9424 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

Re: [PR] Spark 3.5: Migrate tests in SQL directory to JUnit5 [iceberg]

2024-01-08 Thread via GitHub
nastra commented on code in PR #9401: URL: https://github.com/apache/iceberg/pull/9401#discussion_r1445274742 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/sql/TestNamespaceSQL.java: ## @@ -110,134 +151,156 @@ public void testDropNonEmptyNamespace() { sql("DROP

Re: [PR] Spark 3.5: Migrate tests in SQL directory to JUnit5 [iceberg]

2024-01-08 Thread via GitHub
nastra commented on code in PR #9401: URL: https://github.com/apache/iceberg/pull/9401#discussion_r1445274451 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/sql/TestNamespaceSQL.java: ## @@ -110,134 +151,156 @@ public void testDropNonEmptyNamespace() { sql("DROP

Re: [PR] Spark 3.5: Migrate tests in SQL directory to JUnit5 [iceberg]

2024-01-08 Thread via GitHub
nastra commented on code in PR #9401: URL: https://github.com/apache/iceberg/pull/9401#discussion_r1445274223 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/sql/TestNamespaceSQL.java: ## @@ -110,134 +151,156 @@ public void testDropNonEmptyNamespace() { sql("DROP

Re: [PR] Spark 3.5: Migrate tests in SQL directory to JUnit5 [iceberg]

2024-01-08 Thread via GitHub
nastra commented on code in PR #9401: URL: https://github.com/apache/iceberg/pull/9401#discussion_r1445273981 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/sql/TestNamespaceSQL.java: ## @@ -110,134 +151,156 @@ public void testDropNonEmptyNamespace() { sql("DROP

Re: [PR] Spark 3.5: Migrate tests in SQL directory to JUnit5 [iceberg]

2024-01-08 Thread via GitHub
nastra commented on code in PR #9401: URL: https://github.com/apache/iceberg/pull/9401#discussion_r1445272797 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/sql/TestNamespaceSQL.java: ## @@ -18,90 +18,131 @@ */ package org.apache.iceberg.spark.sql; +import stati

Re: [PR] Spark 3.5: Migrate tests in SQL directory to JUnit5 [iceberg]

2024-01-08 Thread via GitHub
nastra commented on code in PR #9401: URL: https://github.com/apache/iceberg/pull/9401#discussion_r1445263754 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/sql/TestCreateTable.java: ## @@ -289,22 +312,25 @@ public void testCreateTableProperties() { tableNam

Re: [PR] Parquet: Move to ValueReader generation to a visitor [iceberg]

2024-01-08 Thread via GitHub
Fokko commented on PR #9063: URL: https://github.com/apache/iceberg/pull/9063#issuecomment-1881712303 Thanks @jacobmarble, @nk1506 & @amogh-jahagirdar for the review 🙌 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

Re: [PR] Parquet: Move to ValueReader generation to a visitor [iceberg]

2024-01-08 Thread via GitHub
Fokko merged PR #9063: URL: https://github.com/apache/iceberg/pull/9063 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apach

Re: [PR] Homepage added to Cargo.toml [iceberg-rust]

2024-01-08 Thread via GitHub
Fokko commented on PR #157: URL: https://github.com/apache/iceberg-rust/pull/157#issuecomment-1881694033 @hiirrxnn I think we want to add the website to each of the `Cargo.toml` in each of the crates, for example here: https://github.com/apache/iceberg-rust/blob/b7031464dfd57dbfb9e872121377

  1   2   >