Re: [PR] Spark 3.5: Fix the setting of equalAuthorities in RemoveOrphanFilesProcedure [iceberg]

2024-05-15 Thread via GitHub
nastra commented on code in PR #10334: URL: https://github.com/apache/iceberg/pull/10334#discussion_r1602708411 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestRemoveOrphanFilesProcedure.java: ## @@ -666,4 +666,87 @@ public void testRemoveOrp

Re: [PR] Iceberg/Comet integration POC [iceberg]

2024-05-15 Thread via GitHub
huaxingao commented on code in PR #9841: URL: https://github.com/apache/iceberg/pull/9841#discussion_r1602698512 ## spark/v3.4/spark/src/main/java/org/apache/iceberg/spark/BatchReadConf.java: ## @@ -0,0 +1,47 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

Re: [PR] AWS: add retry logic to S3InputStream [iceberg]

2024-05-15 Thread via GitHub
puchengy commented on PR #4912: URL: https://github.com/apache/iceberg/pull/4912#issuecomment-2114092322 @jackye1995 Hi, I also encountered the same issue as described in https://github.com/apache/iceberg/issues/10340 can we continue the PR to address the issue here? ALSo I wonder have you

[I] javax.net.ssl.SSLException: Connection reset on S3 w/ S3FileIO and Apache HTTP client [iceberg]

2024-05-15 Thread via GitHub
puchengy opened a new issue, #10340: URL: https://github.com/apache/iceberg/issues/10340 ### Apache Iceberg version 1.3.1 ### Query engine Spark ### Please describe the bug 🐞 ``` 24/05/15 15:10:31 ERROR [Executor task launch worker for task 34.0 in stage

Re: [PR] Flink: refactor sink shuffling statistics collection [iceberg]

2024-05-15 Thread via GitHub
stevenzwu commented on code in PR #10331: URL: https://github.com/apache/iceberg/pull/10331#discussion_r1602632214 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/SketchUtil.java: ## @@ -0,0 +1,182 @@ +/* + * Licensed to the Apache Software Foundation (A

[PR] Core: Remove unused ManifestGroup#filterManifests(Predicate) [iceberg]

2024-05-15 Thread via GitHub
amogh-jahagirdar opened a new pull request, #10339: URL: https://github.com/apache/iceberg/pull/10339 ManifestGroup#filterManifests(Predicate) is an unused package private API that we can remove since it's always true. -- This is an automated message from the Apache Git Service. To respo

Re: [PR] Implement BoundPredicateVisitor trait for ManifestFilterVisitor [iceberg-rust]

2024-05-15 Thread via GitHub
marvinlanhenke commented on code in PR #367: URL: https://github.com/apache/iceberg-rust/pull/367#discussion_r1602587866 ## crates/iceberg/src/expr/visitors/manifest_evaluator.rs: ## @@ -103,98 +106,245 @@ impl BoundPredicateVisitor for ManifestFilterVisitor<'_> { refe

Re: [I] Allow to configure thread-pool while using Iceberg to read the data (plan files/tasks) [iceberg]

2024-05-15 Thread via GitHub
osscm commented on issue #10335: URL: https://github.com/apache/iceberg/issues/10335#issuecomment-2113659715 Thanks Amogh, For us, it the first problem, (didn’t debug if worker pool is not used for certain tasks, but for scanning it is using) I also have similar idea of passi

[PR] Build: Bump mkdocs-material from 9.5.22 to 9.5.23 [iceberg-python]

2024-05-15 Thread via GitHub
dependabot[bot] opened a new pull request, #747: URL: https://github.com/apache/iceberg-python/pull/747 Bumps [mkdocs-material](https://github.com/squidfunk/mkdocs-material) from 9.5.22 to 9.5.23. Release notes Sourced from https://github.com/squidfunk/mkdocs-material/releases";>mk

[PR] Build: Bump mkdocstrings-python from 1.10.1 to 1.10.2 [iceberg-python]

2024-05-15 Thread via GitHub
dependabot[bot] opened a new pull request, #746: URL: https://github.com/apache/iceberg-python/pull/746 Bumps [mkdocstrings-python](https://github.com/mkdocstrings/python) from 1.10.1 to 1.10.2. Release notes Sourced from https://github.com/mkdocstrings/python/releases";>mkdocstrin

Re: [PR] Flink: refactor sink shuffling statistics collection [iceberg]

2024-05-15 Thread via GitHub
stevenzwu commented on code in PR #10331: URL: https://github.com/apache/iceberg/pull/10331#discussion_r1602342274 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/StatisticsType.java: ## @@ -0,0 +1,55 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] Implement BoundPredicateVisitor trait for ManifestFilterVisitor [iceberg-rust]

2024-05-15 Thread via GitHub
s-akhtar-baig commented on code in PR #367: URL: https://github.com/apache/iceberg-rust/pull/367#discussion_r1602276980 ## crates/iceberg/src/expr/visitors/manifest_evaluator.rs: ## @@ -103,98 +106,245 @@ impl BoundPredicateVisitor for ManifestFilterVisitor<'_> { refer

[PR] Add azure configuration variables [iceberg-python]

2024-05-15 Thread via GitHub
kevinzwang opened a new pull request, #745: URL: https://github.com/apache/iceberg-python/pull/745 Add the Iceberg table Azure (ADLFS) config options as Python variables to match S3 and GCS -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [PR] Avoid adding a closed client to the pool [iceberg]

2024-05-15 Thread via GitHub
flyrain commented on PR #10337: URL: https://github.com/apache/iceberg/pull/10337#issuecomment-2113464108 Thanks @amogh-jahagirdar for the review! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Avoid adding a closed client to the pool [iceberg]

2024-05-15 Thread via GitHub
flyrain merged PR #10337: URL: https://github.com/apache/iceberg/pull/10337 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.a

[I] 0x1F as nested namespace separator for REST URL paths conflicts with Servlet spec [iceberg]

2024-05-15 Thread via GitHub
westse opened a new issue, #10338: URL: https://github.com/apache/iceberg/issues/10338 ### Feature Request / Improvement The [current Iceberg REST spec](https://github.com/apache/iceberg/blob/4c9f47d208b16921f825a66e24d0693f2b76b03b/open-api/rest-catalog-open-api.yaml#L1440) uses `0x

Re: [PR] REST: honor OAuth config sent by the server [iceberg]

2024-05-15 Thread via GitHub
rdblue commented on PR #10256: URL: https://github.com/apache/iceberg/pull/10256#issuecomment-2113392645 @adutra, thanks for noticing and pointing out the inconsistencies. I do think it's important to address those. In particular, I did not realize that the table session that was introduced

Re: [PR] Avoid adding a closed client to the pool [iceberg]

2024-05-15 Thread via GitHub
amogh-jahagirdar commented on code in PR #10337: URL: https://github.com/apache/iceberg/pull/10337#discussion_r1601983226 ## core/src/main/java/org/apache/iceberg/ClientPoolImpl.java: ## @@ -70,37 +71,29 @@ public R run(Action action, boolean retry) throws E, InterruptedExc

Re: [PR] REST: honor OAuth config sent by the server [iceberg]

2024-05-15 Thread via GitHub
danielcweeks commented on PR #10256: URL: https://github.com/apache/iceberg/pull/10256#issuecomment-2113348511 > Why should a (malicious) Iceberg REST endpoint do the more complex redirect-dance, if it can get the nearly clear-text credentials due to the `/v1/oauth/tokens` route introduced

Re: [PR] Url encode field names for partition paths [iceberg]

2024-05-15 Thread via GitHub
danielcweeks commented on code in PR #10329: URL: https://github.com/apache/iceberg/pull/10329#discussion_r1602150488 ## core/src/test/java/org/apache/iceberg/TestLocationProvider.java: ## @@ -285,4 +286,22 @@ public void testObjectStorageWithinTableLocation() { assertThat(

Re: [PR] Kafka-connect: Handle namespace creation for auto table creation [iceberg]

2024-05-15 Thread via GitHub
bryanck merged PR #10186: URL: https://github.com/apache/iceberg/pull/10186 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.a

Re: [PR] Kafka-connect: Handle namespace creation for auto table creation [iceberg]

2024-05-15 Thread via GitHub
bryanck commented on PR #10186: URL: https://github.com/apache/iceberg/pull/10186#issuecomment-2113284956 Thanks for the contribution @ajantha-bhat , it looks good! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

Re: [I] Support writing to a branch [iceberg-python]

2024-05-15 Thread via GitHub
kevinjqliu commented on issue #306: URL: https://github.com/apache/iceberg-python/issues/306#issuecomment-2113260320 @Gowthami03B definitely, I've assigned the issue to you! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [I] Support writing to a branch [iceberg-python]

2024-05-15 Thread via GitHub
Gowthami03B commented on issue #306: URL: https://github.com/apache/iceberg-python/issues/306#issuecomment-2113233736 Hello, can I hop back on this train if no one else is actively working on this (again building off of kevin's work)? @kevinjqliu @Fokko -- This is an automated message fr

Re: [PR] Make current-snapshot-id optional while maintaining backwards compatibility [iceberg-rust]

2024-05-15 Thread via GitHub
s-akhtar-baig commented on PR #374: URL: https://github.com/apache/iceberg-rust/pull/374#issuecomment-2113206505 Also, do we have a preference on where we want to document the "backwards compatibility" section? -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] Url encode field names for partition paths [iceberg]

2024-05-15 Thread via GitHub
dimas-b commented on code in PR #10329: URL: https://github.com/apache/iceberg/pull/10329#discussion_r1602092875 ## api/src/main/java/org/apache/iceberg/PartitionSpec.java: ## @@ -189,7 +189,7 @@ public String partitionToPath(StructLike data) { if (i > 0) { sb.ap

Re: [PR] Make current-snapshot-id optional while maintaining backwards compatibility [iceberg-rust]

2024-05-15 Thread via GitHub
s-akhtar-baig commented on PR #374: URL: https://github.com/apache/iceberg-rust/pull/374#issuecomment-2113200473 @Fokko, these changes make current-snapshot-id optional and uses a flag to maintain backwards compatibility. Let me know what you think. Please note that, with the flag on

Re: [PR] Url encode field names for partition paths [iceberg]

2024-05-15 Thread via GitHub
dimas-b commented on code in PR #10329: URL: https://github.com/apache/iceberg/pull/10329#discussion_r1602074394 ## api/src/main/java/org/apache/iceberg/PartitionSpec.java: ## @@ -189,7 +189,7 @@ public String partitionToPath(StructLike data) { if (i > 0) { sb.ap

[I] Enhancement: Add support for a config file in iceberg-rust [iceberg-rust]

2024-05-15 Thread via GitHub
s-akhtar-baig opened a new issue, #375: URL: https://github.com/apache/iceberg-rust/issues/375 As a follow-up to https://github.com/apache/iceberg-rust/pull/374, - Implement a config file for iceberg-rust - Provide users with an additional option to set `legacy-current-snapshot-id`

[PR] Make current-snapshot-id optional while maintaining backwards compatibility [iceberg-rust]

2024-05-15 Thread via GitHub
s-akhtar-baig opened a new pull request, #374: URL: https://github.com/apache/iceberg-rust/pull/374 GitHub issue: https://github.com/apache/iceberg-rust/issues/352 **Problem:** Previous versions of Java (`<1.4.0`) implementations incorrectly assume the optional attribute `current-snap

Re: [I] Support Snapshot Management Operations [iceberg-python]

2024-05-15 Thread via GitHub
chinmay-bhat commented on issue #737: URL: https://github.com/apache/iceberg-python/issues/737#issuecomment-2113132702 > Yeah I think it would be helpful to introduce a snapshot_by_timestamp utility function to get the snapshot - just like you mentioned, that would help recover feature pari

Re: [I] Support Snapshot Management Operations [iceberg-python]

2024-05-15 Thread via GitHub
syun64 commented on issue #737: URL: https://github.com/apache/iceberg-python/issues/737#issuecomment-2113105812 > * How does rollback_to_timestamp use `set_ref_snapshot()`? In the [rollback_to_timestamp documentation](https://iceberg.apache.org/docs/1.5.1/spark-procedures/#rollback_to_time

Re: [PR] Support creating tags by adding `set_ref_snapshot` API [iceberg-python]

2024-05-15 Thread via GitHub
syun64 commented on PR #728: URL: https://github.com/apache/iceberg-python/pull/728#issuecomment-2113090368 @HonahX @Fokko -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [I] Support Snapshot Management Operations: Cherry Pick Operation [iceberg-python]

2024-05-15 Thread via GitHub
chinmay-bhat commented on issue #737: URL: https://github.com/apache/iceberg-python/issues/737#issuecomment-2113081895 @syun64 I have a few questions about the operations and I couldn't find more info in the docs. Apologies if these have been answered elsewhere. - How does rollback_to

Re: [PR] Introduce hierarchical namespaces into SqlCatalog [iceberg-python]

2024-05-15 Thread via GitHub
cccs-eric commented on PR #591: URL: https://github.com/apache/iceberg-python/pull/591#issuecomment-2113066343 > @cccs-eric Thanks for working on this, could you resolve the conflicts and fix the tests? Sorry for the late reply, I was traveling last week and also the Iceberg summit took som

Re: [PR] REST: honor OAuth config sent by the server [iceberg]

2024-05-15 Thread via GitHub
snazy commented on PR #10256: URL: https://github.com/apache/iceberg/pull/10256#issuecomment-2113057306 > This change allows redirecting the auth server which should expose sensitive information to the wrong party. Why should a (malicious) Iceberg REST endpoint do the more complex re

Re: [PR] REST: assume issued token type is access token [iceberg]

2024-05-15 Thread via GitHub
adutra commented on code in PR #10314: URL: https://github.com/apache/iceberg/pull/10314#discussion_r1601962494 ## core/src/main/java/org/apache/iceberg/rest/auth/OAuth2Util.java: ## @@ -763,11 +763,19 @@ private static AuthSession fromTokenResponse( long startTimeMilli

Re: [PR] Support creating tags by adding `set_ref_snapshot` API [iceberg-python]

2024-05-15 Thread via GitHub
chinmay-bhat commented on PR #728: URL: https://github.com/apache/iceberg-python/pull/728#issuecomment-2113015146 Can anyone with write access review? I'd like to make sure the PR passes all tests! -- This is an automated message from the Apache Git Service. To respond to the message, ple

Re: [I] REST Catalog to support custom-catalog name like HMS/Glue [iceberg]

2024-05-15 Thread via GitHub
ajantha-bhat commented on issue #10205: URL: https://github.com/apache/iceberg/issues/10205#issuecomment-2112968638 > We can explore supporting alternatives, but I don't feel like making a change is necessary to address the issue proposed. Agree. -- This is an automated message f

Re: [I] REST Catalog to support custom-catalog name like HMS/Glue [iceberg]

2024-05-15 Thread via GitHub
jbonofre commented on issue #10205: URL: https://github.com/apache/iceberg/issues/10205#issuecomment-2112965457 I agree with @danielcweeks here with the current REST API. I think it makes sense to include in the new REST Spec, but no need to be addressed with the current one. -- This is

Re: [I] REST Catalog to support custom-catalog name like HMS/Glue [iceberg]

2024-05-15 Thread via GitHub
danielcweeks commented on issue #10205: URL: https://github.com/apache/iceberg/issues/10205#issuecomment-2112927398 > Warehouse has a different meaning for catalogs . . . There is no agreed upon standard for the naming of parts of an identifier, so while some people will find confusin

Re: [PR] Avoid adding a closed client to the pool [iceberg]

2024-05-15 Thread via GitHub
flyrain commented on code in PR #10337: URL: https://github.com/apache/iceberg/pull/10337#discussion_r1601860137 ## core/src/main/java/org/apache/iceberg/ClientPoolImpl.java: ## @@ -70,37 +71,29 @@ public R run(Action action, boolean retry) throws E, InterruptedExc try {

Re: [PR] Avoid adding a closed client to the pool [iceberg]

2024-05-15 Thread via GitHub
flyrain commented on code in PR #10337: URL: https://github.com/apache/iceberg/pull/10337#discussion_r1601856725 ## core/src/main/java/org/apache/iceberg/ClientPoolImpl.java: ## @@ -70,37 +71,29 @@ public R run(Action action, boolean retry) throws E, InterruptedExc try {

Re: [PR] Avoid adding a closed client to the pool [iceberg]

2024-05-15 Thread via GitHub
flyrain commented on code in PR #10337: URL: https://github.com/apache/iceberg/pull/10337#discussion_r1601856725 ## core/src/main/java/org/apache/iceberg/ClientPoolImpl.java: ## @@ -70,37 +71,29 @@ public R run(Action action, boolean retry) throws E, InterruptedExc try {

Re: [PR] REST: honor OAuth config sent by the server [iceberg]

2024-05-15 Thread via GitHub
danielcweeks commented on PR #10256: URL: https://github.com/apache/iceberg/pull/10256#issuecomment-2112854713 > My intention was to bring the spec in line with the current implementation by honoring credential and oauth2-server-uri, among others, from the config endpoint. I don

Re: [PR] Spark Action to Analyze table [iceberg]

2024-05-15 Thread via GitHub
huaxingao commented on PR #10288: URL: https://github.com/apache/iceberg/pull/10288#issuecomment-2112792277 > there was an old PR on the same: https://github.com/apache/iceberg/pull/6582 I don't have time to work on this, so karuppayya will take over. Thanks a lot @karuppayya for co

[PR] Spark: Add SparkSQLProperty to control split-size [iceberg]

2024-05-15 Thread via GitHub
sumedhsakdeo opened a new pull request, #10336: URL: https://github.com/apache/iceberg/pull/10336 We have a scheduled job that deletes rows in an Iceberg table. The job is authored in SQL. Given we use CoW technique for data deletion the job would rewrite the files without the deleted rows.

Re: [PR] Introduce hierarchical namespaces into SqlCatalog [iceberg-python]

2024-05-15 Thread via GitHub
Fokko commented on PR #591: URL: https://github.com/apache/iceberg-python/pull/591#issuecomment-2112665131 @cccs-eric Thanks for working on this, could you resolve the conflicts and fix the tests? Thanks! -- This is an automated message from the Apache Git Service. To respond to the messa

Re: [PR] Flink: Maintenance - MonitorSource [iceberg]

2024-05-15 Thread via GitHub
pvary commented on code in PR #10308: URL: https://github.com/apache/iceberg/pull/10308#discussion_r1601589261 ## flink/v1.19/flink/src/test/java/org/apache/iceberg/flink/maintenance/operator/ManualSource.java: ## @@ -0,0 +1,160 @@ +/* + * Licensed to the Apache Software Foundat

Re: [PR] Flink: Maintenance - MonitorSource [iceberg]

2024-05-15 Thread via GitHub
pvary commented on code in PR #10308: URL: https://github.com/apache/iceberg/pull/10308#discussion_r1601585403 ## flink/v1.19/flink/src/test/java/org/apache/iceberg/flink/maintenance/operator/TestMonitorSource.java: ## @@ -0,0 +1,338 @@ +/* + * Licensed to the Apache Software Fo

Re: [PR] REST: assume issued token type is access token [iceberg]

2024-05-15 Thread via GitHub
adutra commented on code in PR #10314: URL: https://github.com/apache/iceberg/pull/10314#discussion_r1601403672 ## core/src/test/java/org/apache/iceberg/rest/RESTCatalogAdapter.java: ## @@ -254,7 +254,6 @@ private static OAuthTokenResponse handleOAuthRequest(Object body) {

Re: [PR] REST: assume issued token type is access token [iceberg]

2024-05-15 Thread via GitHub
adutra commented on code in PR #10314: URL: https://github.com/apache/iceberg/pull/10314#discussion_r1601402075 ## core/src/main/java/org/apache/iceberg/rest/auth/OAuth2Util.java: ## @@ -763,11 +763,19 @@ private static AuthSession fromTokenResponse( long startTimeMilli

Re: [PR] Spark Action to Analyze table [iceberg]

2024-05-15 Thread via GitHub
ajantha-bhat commented on PR #10288: URL: https://github.com/apache/iceberg/pull/10288#issuecomment-2112166283 there was an old PR on the same: https://github.com/apache/iceberg/pull/6582 -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [PR] REST: assume issued token type is access token [iceberg]

2024-05-15 Thread via GitHub
adutra commented on code in PR #10314: URL: https://github.com/apache/iceberg/pull/10314#discussion_r1601396353 ## core/src/main/java/org/apache/iceberg/rest/auth/OAuth2Util.java: ## @@ -763,11 +763,19 @@ private static AuthSession fromTokenResponse( long startTimeMilli

Re: [PR] Add a Spark procedure to collect NDV [iceberg]

2024-05-15 Thread via GitHub
ajantha-bhat commented on PR #6582: URL: https://github.com/apache/iceberg/pull/6582#issuecomment-2112165738 I saw a new PR on the same : https://github.com/apache/iceberg/pull/10288 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Gi

Re: [PR] REST: honor OAuth config sent by the server [iceberg]

2024-05-15 Thread via GitHub
adutra commented on code in PR #10256: URL: https://github.com/apache/iceberg/pull/10256#discussion_r1601352324 ## core/src/main/java/org/apache/iceberg/rest/RESTSessionCatalog.java: ## @@ -215,6 +215,12 @@ public void initialize(String name, Map unresolved) { this.paths =

Re: [PR] Flink: Maintenance - MonitorSource [iceberg]

2024-05-15 Thread via GitHub
pvary commented on code in PR #10308: URL: https://github.com/apache/iceberg/pull/10308#discussion_r1601318958 ## flink/v1.19/flink/src/test/java/org/apache/iceberg/flink/maintenance/operator/TestMonitorSource.java: ## @@ -0,0 +1,338 @@ +/* + * Licensed to the Apache Software Fo

Re: [PR] Flink: Maintenance - MonitorSource [iceberg]

2024-05-15 Thread via GitHub
pvary commented on code in PR #10308: URL: https://github.com/apache/iceberg/pull/10308#discussion_r1601318215 ## flink/v1.19/flink/src/test/java/org/apache/iceberg/flink/maintenance/operator/ManualSource.java: ## @@ -0,0 +1,160 @@ +/* + * Licensed to the Apache Software Foundat

Re: [PR] Flink: Maintenance - MonitorSource [iceberg]

2024-05-15 Thread via GitHub
pvary commented on code in PR #10308: URL: https://github.com/apache/iceberg/pull/10308#discussion_r1601317394 ## flink/v1.19/flink/src/test/java/org/apache/iceberg/flink/maintenance/operator/ManualSource.java: ## @@ -0,0 +1,160 @@ +/* + * Licensed to the Apache Software Foundat

Re: [PR] Flink: Maintenance - MonitorSource [iceberg]

2024-05-15 Thread via GitHub
pvary commented on code in PR #10308: URL: https://github.com/apache/iceberg/pull/10308#discussion_r1601316408 ## flink/v1.19/flink/src/test/java/org/apache/iceberg/flink/maintenance/operator/FlinkStreamingTestUtils.java: ## @@ -0,0 +1,73 @@ +/* + * Licensed to the Apache Softwa

Re: [PR] Flink: Maintenance - MonitorSource [iceberg]

2024-05-15 Thread via GitHub
pvary commented on code in PR #10308: URL: https://github.com/apache/iceberg/pull/10308#discussion_r1601312872 ## flink/v1.19/flink/src/test/java/org/apache/iceberg/flink/maintenance/operator/FlinkStreamingTestUtils.java: ## @@ -0,0 +1,73 @@ +/* + * Licensed to the Apache Softwa

Re: [PR] Flink: Maintenance - MonitorSource [iceberg]

2024-05-15 Thread via GitHub
pvary commented on code in PR #10308: URL: https://github.com/apache/iceberg/pull/10308#discussion_r1601312090 ## flink/v1.19/flink/src/test/java/org/apache/iceberg/flink/maintenance/operator/FlinkSqlExtension.java: ## @@ -0,0 +1,145 @@ +/* + * Licensed to the Apache Software Fo

Re: [PR] Flink: Maintenance - MonitorSource [iceberg]

2024-05-15 Thread via GitHub
pvary commented on code in PR #10308: URL: https://github.com/apache/iceberg/pull/10308#discussion_r1601301144 ## flink/v1.19/flink/src/test/java/org/apache/iceberg/flink/maintenance/operator/FlinkSqlExtension.java: ## @@ -0,0 +1,145 @@ +/* + * Licensed to the Apache Software Fo

Re: [PR] Flink: Maintenance - MonitorSource [iceberg]

2024-05-15 Thread via GitHub
pvary commented on code in PR #10308: URL: https://github.com/apache/iceberg/pull/10308#discussion_r1601296963 ## flink/v1.19/flink/src/test/java/org/apache/iceberg/flink/maintenance/operator/CollectingSink.java: ## @@ -0,0 +1,115 @@ +/* + * Licensed to the Apache Software Found

Re: [PR] REST: honor OAuth config sent by the server [iceberg]

2024-05-15 Thread via GitHub
adutra commented on PR #10256: URL: https://github.com/apache/iceberg/pull/10256#issuecomment-2112020920 @rdblue I have no other intention than the one of bringing the spec in line with the current implementation. There are two areas where spec and implementation differ with respect t

Re: [PR] Flink: Maintenance - MonitorSource [iceberg]

2024-05-15 Thread via GitHub
pvary commented on code in PR #10308: URL: https://github.com/apache/iceberg/pull/10308#discussion_r1601295680 ## flink/v1.19/flink/src/test/java/org/apache/iceberg/flink/maintenance/operator/CollectingSink.java: ## @@ -0,0 +1,115 @@ +/* + * Licensed to the Apache Software Found

Re: [PR] Flink: Maintenance - MonitorSource [iceberg]

2024-05-15 Thread via GitHub
pvary commented on code in PR #10308: URL: https://github.com/apache/iceberg/pull/10308#discussion_r1601295173 ## flink/v1.19/flink/src/test/java/org/apache/iceberg/flink/maintenance/operator/CollectingSink.java: ## @@ -0,0 +1,115 @@ +/* + * Licensed to the Apache Software Found

Re: [PR] Flink: Maintenance - MonitorSource [iceberg]

2024-05-15 Thread via GitHub
pvary commented on code in PR #10308: URL: https://github.com/apache/iceberg/pull/10308#discussion_r1601292857 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/TableChange.java: ## @@ -0,0 +1,127 @@ +/* + * Licensed to the Apache Software Foundati

Re: [PR] Flink: Maintenance - MonitorSource [iceberg]

2024-05-15 Thread via GitHub
pvary commented on code in PR #10308: URL: https://github.com/apache/iceberg/pull/10308#discussion_r1601292448 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/TableChange.java: ## @@ -0,0 +1,127 @@ +/* + * Licensed to the Apache Software Foundati

Re: [PR] Flink: Maintenance - MonitorSource [iceberg]

2024-05-15 Thread via GitHub
pvary commented on code in PR #10308: URL: https://github.com/apache/iceberg/pull/10308#discussion_r1601291444 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/SingleThreadedIteratorSource.java: ## @@ -0,0 +1,196 @@ +/* + * Licensed to the Apache

Re: [PR] Flink: Maintenance - MonitorSource [iceberg]

2024-05-15 Thread via GitHub
pvary commented on code in PR #10308: URL: https://github.com/apache/iceberg/pull/10308#discussion_r1601291185 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/SingleThreadedIteratorSource.java: ## @@ -0,0 +1,196 @@ +/* + * Licensed to the Apache

Re: [PR] Flink: Maintenance - MonitorSource [iceberg]

2024-05-15 Thread via GitHub
pvary commented on code in PR #10308: URL: https://github.com/apache/iceberg/pull/10308#discussion_r1601289077 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/SingleThreadedIteratorSource.java: ## @@ -0,0 +1,196 @@ +/* + * Licensed to the Apache

Re: [PR] Flink: Maintenance - MonitorSource [iceberg]

2024-05-15 Thread via GitHub
pvary commented on code in PR #10308: URL: https://github.com/apache/iceberg/pull/10308#discussion_r1601287576 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/SingleThreadedIteratorSource.java: ## @@ -0,0 +1,196 @@ +/* + * Licensed to the Apache

Re: [PR] Flink: Maintenance - MonitorSource [iceberg]

2024-05-15 Thread via GitHub
pvary commented on code in PR #10308: URL: https://github.com/apache/iceberg/pull/10308#discussion_r1601279619 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/SingleThreadedIteratorSource.java: ## @@ -0,0 +1,196 @@ +/* + * Licensed to the Apache

Re: [PR] Flink: Maintenance - MonitorSource [iceberg]

2024-05-15 Thread via GitHub
pvary commented on code in PR #10308: URL: https://github.com/apache/iceberg/pull/10308#discussion_r1601259054 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/MonitorSource.java: ## @@ -0,0 +1,187 @@ +/* + * Licensed to the Apache Software Founda

Re: [PR] Flink: Maintenance - MonitorSource [iceberg]

2024-05-15 Thread via GitHub
pvary commented on code in PR #10308: URL: https://github.com/apache/iceberg/pull/10308#discussion_r1601256199 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/MonitorSource.java: ## @@ -0,0 +1,187 @@ +/* + * Licensed to the Apache Software Founda

Re: [PR] Flink: Maintenance - MonitorSource [iceberg]

2024-05-15 Thread via GitHub
pvary commented on code in PR #10308: URL: https://github.com/apache/iceberg/pull/10308#discussion_r1601256199 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/MonitorSource.java: ## @@ -0,0 +1,187 @@ +/* + * Licensed to the Apache Software Founda

Re: [PR] Flink: Maintenance - MonitorSource [iceberg]

2024-05-15 Thread via GitHub
pvary commented on code in PR #10308: URL: https://github.com/apache/iceberg/pull/10308#discussion_r1601255807 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/MonitorSource.java: ## @@ -0,0 +1,187 @@ +/* + * Licensed to the Apache Software Founda

Re: [PR] Flink: Maintenance - MonitorSource [iceberg]

2024-05-15 Thread via GitHub
pvary commented on code in PR #10308: URL: https://github.com/apache/iceberg/pull/10308#discussion_r1601251764 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/MonitorSource.java: ## @@ -0,0 +1,187 @@ +/* + * Licensed to the Apache Software Founda

Re: [PR] REST: assume issued token type is access token [iceberg]

2024-05-15 Thread via GitHub
nastra commented on code in PR #10314: URL: https://github.com/apache/iceberg/pull/10314#discussion_r1601214325 ## core/src/main/java/org/apache/iceberg/rest/auth/OAuth2Util.java: ## @@ -763,11 +763,19 @@ private static AuthSession fromTokenResponse( long startTimeMilli

Re: [PR] REST: assume issued token type is access token [iceberg]

2024-05-15 Thread via GitHub
nastra commented on code in PR #10314: URL: https://github.com/apache/iceberg/pull/10314#discussion_r1601220154 ## core/src/test/java/org/apache/iceberg/rest/RESTCatalogAdapter.java: ## @@ -254,7 +254,6 @@ private static OAuthTokenResponse handleOAuthRequest(Object body) {

Re: [PR] REST: assume issued token type is access token [iceberg]

2024-05-15 Thread via GitHub
nastra commented on code in PR #10314: URL: https://github.com/apache/iceberg/pull/10314#discussion_r1601214325 ## core/src/main/java/org/apache/iceberg/rest/auth/OAuth2Util.java: ## @@ -763,11 +763,19 @@ private static AuthSession fromTokenResponse( long startTimeMilli

Re: [PR] REST: assume issued token type is access token [iceberg]

2024-05-15 Thread via GitHub
nastra commented on code in PR #10314: URL: https://github.com/apache/iceberg/pull/10314#discussion_r1601201951 ## core/src/main/java/org/apache/iceberg/rest/auth/OAuth2Util.java: ## @@ -763,11 +763,19 @@ private static AuthSession fromTokenResponse( long startTimeMilli

Re: [PR] REST: honor OAuth config sent by the server [iceberg]

2024-05-15 Thread via GitHub
nastra commented on code in PR #10256: URL: https://github.com/apache/iceberg/pull/10256#discussion_r1601172497 ## core/src/main/java/org/apache/iceberg/rest/RESTSessionCatalog.java: ## @@ -215,6 +215,12 @@ public void initialize(String name, Map unresolved) { this.paths =

Re: [PR] Build: Bump mkdocstrings-python from 1.10.0 to 1.10.1 [iceberg-python]

2024-05-15 Thread via GitHub
HonahX merged PR #744: URL: https://github.com/apache/iceberg-python/pull/744 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [PR] Build: Bump pyarrow from 16.0.0 to 16.1.0 [iceberg-python]

2024-05-15 Thread via GitHub
HonahX merged PR #743: URL: https://github.com/apache/iceberg-python/pull/743 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [PR] Build: Bump ray from 2.9.2 to 2.21.0 [iceberg-python]

2024-05-15 Thread via GitHub
dependabot[bot] commented on PR #720: URL: https://github.com/apache/iceberg-python/pull/720#issuecomment-2111765665 OK, I won't notify you about version 2.x.x again, unless you re-open this PR. -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] Build: Bump ray from 2.9.2 to 2.21.0 [iceberg-python]

2024-05-15 Thread via GitHub
dependabot[bot] closed pull request #720: Build: Bump ray from 2.9.2 to 2.21.0 URL: https://github.com/apache/iceberg-python/pull/720 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

Re: [PR] Build: Bump ray from 2.9.2 to 2.21.0 [iceberg-python]

2024-05-15 Thread via GitHub
HonahX commented on PR #720: URL: https://github.com/apache/iceberg-python/pull/720#issuecomment-2111765584 @dependabot ignore this major version -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

Re: [I] `catalog.create_namespace(...)` throws both 404 and 409 which is unintuitive [iceberg-python]

2024-05-15 Thread via GitHub
HonahX closed issue #430: `catalog.create_namespace(...)` throws both 404 and 409 which is unintuitive URL: https://github.com/apache/iceberg-python/issues/430 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] Remove NoSuchNamespaceError on namespace creation [iceberg-python]

2024-05-15 Thread via GitHub
HonahX merged PR #726: URL: https://github.com/apache/iceberg-python/pull/726 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [PR] Add create_namespace_if_not_exists method [iceberg-python]

2024-05-15 Thread via GitHub
HonahX merged PR #725: URL: https://github.com/apache/iceberg-python/pull/725 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [I] create namespace if not exist [iceberg-python]

2024-05-15 Thread via GitHub
HonahX closed issue #709: create namespace if not exist URL: https://github.com/apache/iceberg-python/issues/709 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

Re: [PR] Hive catalog: Add retry logic for hive locking [iceberg-python]

2024-05-15 Thread via GitHub
HonahX merged PR #701: URL: https://github.com/apache/iceberg-python/pull/701 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [PR] Hive catalog: Add retry logic for hive locking [iceberg-python]

2024-05-15 Thread via GitHub
HonahX commented on PR #701: URL: https://github.com/apache/iceberg-python/pull/701#issuecomment-2111753317 Looks like all the review comments are addressed. I'll merge this. Thanks everyone! -- This is an automated message from the Apache Git Service. To respond to the message, please lo