[PR] Build: Fix minor compilation warnings [iceberg]

2023-10-10 Thread via GitHub
nk1506 opened a new pull request, #8758: URL: https://github.com/apache/iceberg/pull/8758 There were few warnings with `./gradlew clean build -x test -x integrationTest` . this change is to make build **green**. -- This is an automated message from the Apache Git Service. To resp

Re: [I] flink1.14.4+iceberg0.13.1+hive-metastore3.1.2+minio(S3) error! [iceberg]

2023-10-10 Thread via GitHub
pvary commented on issue #4743: URL: https://github.com/apache/iceberg/issues/4743#issuecomment-1754585935 Maybe opening another issue would have been > @pvary I know the error is diff than the issue. Maybe opening another issue would have been better in this case > Do w

Re: [I] flink1.14.4+iceberg0.13.1+hive-metastore3.1.2+minio(S3) error! [iceberg]

2023-10-10 Thread via GitHub
ramdas-jagtap commented on issue #4743: URL: https://github.com/apache/iceberg/issues/4743#issuecomment-1754588737 Thanks @pvary for sharing the docs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[PR] Disable merge-commit and enforce linear history [iceberg-python]

2023-10-10 Thread via GitHub
Fokko opened a new pull request, #57: URL: https://github.com/apache/iceberg-python/pull/57 This keeps the git history clear -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [I] Migrate Files using TestRule in dell package to Junit5 [iceberg]

2023-10-10 Thread via GitHub
nastra closed issue #7888: Migrate Files using TestRule in dell package to Junit5 URL: https://github.com/apache/iceberg/issues/7888 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

Re: [PR] Dell : Migrate Files using TestRule to Junit5. [iceberg]

2023-10-10 Thread via GitHub
nastra merged PR #8707: URL: https://github.com/apache/iceberg/pull/8707 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

Re: [I] Failed to find data source: iceberg. Please find packages at [iceberg]

2023-10-10 Thread via GitHub
NhatDuy11 commented on issue #7268: URL: https://github.com/apache/iceberg/issues/7268#issuecomment-1754691644 Can someone tell me if I am using Spark 2.4.5 and Scala 2.11.12 which version of Apache Iceberg should I use?, thank you very much everyone! -- This is an automated message from

Re: [I] java.lang.IllegalStateException: Connection pool shut down when refreshing table metadata on s3 [iceberg]

2023-10-10 Thread via GitHub
AkshayWise commented on issue #8601: URL: https://github.com/apache/iceberg/issues/8601#issuecomment-1754706576 @Kontinuation @stevenzwu I believe this fix was released over 1.4.0 over last week, but I am still getting this error over Flink (1.15) Iceberg jobs: ``` java.lang.IllegalSta

[PR] Core: Use more permissive check when registering existing table [iceberg]

2023-10-10 Thread via GitHub
nastra opened a new pull request, #8759: URL: https://github.com/apache/iceberg/pull/8759 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mai

Re: [I] java.lang.IllegalStateException: Connection pool shut down when refreshing table metadata on s3 [iceberg]

2023-10-10 Thread via GitHub
nastra commented on issue #8601: URL: https://github.com/apache/iceberg/issues/8601#issuecomment-1754735024 @AkshayWise this fix didn't make it into 1.4.0 unfortunately -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

Re: [PR] Construct a writer tree [iceberg-python]

2023-10-10 Thread via GitHub
Fokko commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1351918311 ## pyiceberg/avro/resolver.py: ## @@ -233,7 +255,107 @@ def skip(self, decoder: BinaryDecoder) -> None: pass -class SchemaResolver(PrimitiveWithPartnerVis

Re: [PR] Add logic for table format-version updates [iceberg-python]

2023-10-10 Thread via GitHub
Fokko commented on PR #55: URL: https://github.com/apache/iceberg-python/pull/55#issuecomment-1754783886 @rdblue I agree with you there. I think we can still update the method name since it was just raising an exception. -- This is an automated message from the Apache Git Service. To resp

Re: [PR] Core: Allow missing object in ErrorResponse [iceberg]

2023-10-10 Thread via GitHub
amogh-jahagirdar commented on code in PR #8760: URL: https://github.com/apache/iceberg/pull/8760#discussion_r1352021537 ## core/src/main/java/org/apache/iceberg/rest/responses/ErrorResponseParser.java: ## @@ -76,17 +76,20 @@ public static ErrorResponse fromJson(JsonNode jsonNode

Re: [PR] Core: Allow missing object in ErrorResponse [iceberg]

2023-10-10 Thread via GitHub
amogh-jahagirdar commented on code in PR #8760: URL: https://github.com/apache/iceberg/pull/8760#discussion_r135149 ## core/src/main/java/org/apache/iceberg/rest/responses/ErrorResponse.java: ## @@ -22,18 +22,17 @@ import java.io.StringWriter; import java.util.Arrays; imp

[I] Flaky test/env TestFlinkParquetReader, TestFlinkParquetWriter, TestIcebergSourceBoundedSql [iceberg]

2023-10-10 Thread via GitHub
nk1506 opened a new issue, #8761: URL: https://github.com/apache/iceberg/issues/8761 ### Apache Iceberg version 1.4.0 (latest release) ### Query engine Flink ### Please describe the bug 🐞 Flink 1.16 ` at org.apache.flink.streaming.api.envi

Re: [I] Some questions about Iceberg's capabilities in Flink [iceberg]

2023-10-10 Thread via GitHub
jonathf commented on issue #8754: URL: https://github.com/apache/iceberg/issues/8754#issuecomment-1754998172 Okay, that explains it. Last two questions: * Will #8553 support some sort of ordering guaranty? * Is the streaminig feature associated with icebergs tagging features? Thi

Re: [PR] Add logic for table format-version updates [iceberg-python]

2023-10-10 Thread via GitHub
Fokko merged PR #55: URL: https://github.com/apache/iceberg-python/pull/55 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Core: Allow missing object in ErrorResponse [iceberg]

2023-10-10 Thread via GitHub
Fokko commented on code in PR #8760: URL: https://github.com/apache/iceberg/pull/8760#discussion_r1352239032 ## core/src/main/java/org/apache/iceberg/rest/responses/ErrorResponseParser.java: ## @@ -76,17 +76,20 @@ public static ErrorResponse fromJson(JsonNode jsonNode) {

Re: [PR] Core: Allow missing object in ErrorResponse [iceberg]

2023-10-10 Thread via GitHub
Fokko commented on code in PR #8760: URL: https://github.com/apache/iceberg/pull/8760#discussion_r1352243492 ## core/src/main/java/org/apache/iceberg/rest/responses/ErrorResponse.java: ## @@ -22,18 +22,17 @@ import java.io.StringWriter; import java.util.Arrays; import java.ut

[I] is there anyway to rewrite onto a specific branch? [iceberg]

2023-10-10 Thread via GitHub
zinking opened a new issue, #8762: URL: https://github.com/apache/iceberg/issues/8762 ### Query engine Spark ### Question I thought this might do ``` val table = s"iceberg_catalog.${tableIdentifier}.branch_${branch}" val t = Spark3Util.loadIce

Re: [PR] Construct a writer tree [iceberg-python]

2023-10-10 Thread via GitHub
Fokko commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1352295604 ## pyiceberg/avro/resolver.py: ## @@ -233,7 +255,107 @@ def skip(self, decoder: BinaryDecoder) -> None: pass -class SchemaResolver(PrimitiveWithPartnerVis

[I] Pass in the correct type for the VisitorWithParent [iceberg-python]

2023-10-10 Thread via GitHub
Fokko opened a new issue, #58: URL: https://github.com/apache/iceberg-python/issues/58 ### Feature Request / Improvement So we can avoid the checks, see: https://github.com/apache/iceberg-python/pull/40#discussion_r1349776857 -- This is an automated message from the Apache Git Serv

Re: [PR] Construct a writer tree [iceberg-python]

2023-10-10 Thread via GitHub
Fokko commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1352299345 ## pyiceberg/avro/resolver.py: ## @@ -192,7 +195,26 @@ def visit_binary(self, binary_type: BinaryType) -> Writer: return BinaryWriter() -def resolve( +CON

Re: [PR] Disable merge-commit and enforce linear history [iceberg-python]

2023-10-10 Thread via GitHub
liurenjie1024 commented on code in PR #57: URL: https://github.com/apache/iceberg-python/pull/57#discussion_r1352301058 ## .asf.yaml: ## @@ -28,6 +28,16 @@ github: - apache - hacktoberfest - pyiceberg + enabled_merge_buttons: +merge: false +squash: true +

Re: [PR] Construct a writer tree [iceberg-python]

2023-10-10 Thread via GitHub
Fokko commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1352300965 ## pyiceberg/avro/resolver.py: ## @@ -192,7 +194,28 @@ def visit_binary(self, binary_type: BinaryType) -> Writer: return BinaryWriter() -def resolve( +CON

Re: [PR] Construct a writer tree [iceberg-python]

2023-10-10 Thread via GitHub
Fokko commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1352299345 ## pyiceberg/avro/resolver.py: ## @@ -192,7 +195,26 @@ def visit_binary(self, binary_type: BinaryType) -> Writer: return BinaryWriter() -def resolve( +CON

Re: [PR] Construct a writer tree [iceberg-python]

2023-10-10 Thread via GitHub
Fokko commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1352318212 ## pyiceberg/avro/resolver.py: ## @@ -233,7 +256,93 @@ def skip(self, decoder: BinaryDecoder) -> None: pass -class SchemaResolver(PrimitiveWithPartnerVisi

Re: [PR] Construct a writer tree [iceberg-python]

2023-10-10 Thread via GitHub
Fokko commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1352317115 ## pyiceberg/avro/resolver.py: ## @@ -233,7 +256,93 @@ def skip(self, decoder: BinaryDecoder) -> None: pass -class SchemaResolver(PrimitiveWithPartnerVisi

Re: [PR] Construct a writer tree [iceberg-python]

2023-10-10 Thread via GitHub
Fokko commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1352317115 ## pyiceberg/avro/resolver.py: ## @@ -233,7 +256,93 @@ def skip(self, decoder: BinaryDecoder) -> None: pass -class SchemaResolver(PrimitiveWithPartnerVisi

[PR] Build: Fix compiler warnings [iceberg]

2023-10-10 Thread via GitHub
nk1506 opened a new pull request, #8763: URL: https://github.com/apache/iceberg/pull/8763 Fixing few xlint related warnings. Before: https://github.com/apache/iceberg/assets/4146188/c3666c0d-f879-4dbc-8fe4-89ab91b93079";> After: https://github.com/apache/iceberg/a

Re: [PR] Construct a writer tree [iceberg-python]

2023-10-10 Thread via GitHub
Fokko commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1352346498 ## pyiceberg/avro/resolver.py: ## @@ -233,7 +256,93 @@ def skip(self, decoder: BinaryDecoder) -> None: pass -class SchemaResolver(PrimitiveWithPartnerVisi

Re: [PR] Disable merge-commit and enforce linear history [iceberg-python]

2023-10-10 Thread via GitHub
Fokko commented on code in PR #57: URL: https://github.com/apache/iceberg-python/pull/57#discussion_r1352350208 ## .asf.yaml: ## @@ -28,6 +28,16 @@ github: - apache - hacktoberfest - pyiceberg + enabled_merge_buttons: +merge: false +squash: true +reba

Re: [PR] Build: Fix minor compilation warnings [iceberg]

2023-10-10 Thread via GitHub
nastra merged PR #8758: URL: https://github.com/apache/iceberg/pull/8758 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

Re: [I] Optimize metadata tables? [iceberg]

2023-10-10 Thread via GitHub
ajantha-bhat commented on issue #8714: URL: https://github.com/apache/iceberg/issues/8714#issuecomment-1755285785 @RussellSpitzer and @aokolnychyi: WDYT? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] Spec: Add partition stats spec [iceberg]

2023-10-10 Thread via GitHub
ajantha-bhat commented on code in PR #7105: URL: https://github.com/apache/iceberg/pull/7105#discussion_r1352477202 ## format/spec.md: ## @@ -702,6 +703,58 @@ Blob metadata is a struct with the following fields: | _optional_ | _optional_ | **`properties`** | `map` | Additional

Re: [PR] Spec: Add partition stats spec [iceberg]

2023-10-10 Thread via GitHub
ajantha-bhat commented on code in PR #7105: URL: https://github.com/apache/iceberg/pull/7105#discussion_r1352482149 ## format/spec.md: ## @@ -702,6 +703,58 @@ Blob metadata is a struct with the following fields: | _optional_ | _optional_ | **`properties`** | `map` | Additional

Re: [PR] Spec: Add partition stats spec [iceberg]

2023-10-10 Thread via GitHub
ajantha-bhat commented on code in PR #7105: URL: https://github.com/apache/iceberg/pull/7105#discussion_r1352483805 ## format/spec.md: ## @@ -702,6 +703,58 @@ Blob metadata is a struct with the following fields: | _optional_ | _optional_ | **`properties`** | `map` | Additional

Re: [PR] Spec: Add partition stats spec [iceberg]

2023-10-10 Thread via GitHub
ajantha-bhat commented on code in PR #7105: URL: https://github.com/apache/iceberg/pull/7105#discussion_r1352485101 ## format/spec.md: ## @@ -702,6 +703,58 @@ Blob metadata is a struct with the following fields: | _optional_ | _optional_ | **`properties`** | `map` | Additional

Re: [PR] Spec: Add partition stats spec [iceberg]

2023-10-10 Thread via GitHub
ajantha-bhat commented on code in PR #7105: URL: https://github.com/apache/iceberg/pull/7105#discussion_r1352490767 ## format/spec.md: ## @@ -702,6 +703,58 @@ Blob metadata is a struct with the following fields: | _optional_ | _optional_ | **`properties`** | `map` | Additional

Re: [PR] Spec: Add partition stats spec [iceberg]

2023-10-10 Thread via GitHub
ajantha-bhat commented on code in PR #7105: URL: https://github.com/apache/iceberg/pull/7105#discussion_r1352492103 ## format/spec.md: ## @@ -702,6 +703,58 @@ Blob metadata is a struct with the following fields: | _optional_ | _optional_ | **`properties`** | `map` | Additional

Re: [PR] Spec: Add partition stats spec [iceberg]

2023-10-10 Thread via GitHub
ajantha-bhat commented on PR #7105: URL: https://github.com/apache/iceberg/pull/7105#issuecomment-1755391052 @aokolnychyi: Thanks for the detailed review and also going through the POC PRs. I have addressed all the comments. Please have a look again. -- This is an automated message

Re: [PR] Spec: Add partition stats spec [iceberg]

2023-10-10 Thread via GitHub
ajantha-bhat commented on code in PR #7105: URL: https://github.com/apache/iceberg/pull/7105#discussion_r1352500873 ## format/spec.md: ## @@ -702,6 +703,58 @@ Blob metadata is a struct with the following fields: | _optional_ | _optional_ | **`properties`** | `map` | Additional

Re: [I] Optimize metadata tables? [iceberg]

2023-10-10 Thread via GitHub
RussellSpitzer commented on issue #8714: URL: https://github.com/apache/iceberg/issues/8714#issuecomment-1755413260 I don't see any particular reason but I also don't see any reason to change an existing public api here. -- This is an automated message from the Apache Git Service. To res

Re: [I] Optimize metadata tables? [iceberg]

2023-10-10 Thread via GitHub
ajantha-bhat commented on issue #8714: URL: https://github.com/apache/iceberg/issues/8714#issuecomment-1755421403 > I don't see any particular reason but I also don't see any reason to change an existing public api here. 🙃🙃🙃 -- This is an automated message from the Apache Git Servi

Re: [I] Optimize metadata tables? [iceberg]

2023-10-10 Thread via GitHub
ajantha-bhat commented on issue #8714: URL: https://github.com/apache/iceberg/issues/8714#issuecomment-1755423504 > I don't see any particular reason but I also don't see any reason to change an existing public api here. My case for removing is to keep things simple and less metadata

Re: [PR] Spec: Add partition stats spec [iceberg]

2023-10-10 Thread via GitHub
ajantha-bhat commented on code in PR #7105: URL: https://github.com/apache/iceberg/pull/7105#discussion_r1352528356 ## format/spec.md: ## @@ -702,6 +703,58 @@ Blob metadata is a struct with the following fields: | _optional_ | _optional_ | **`properties`** | `map` | Additional

Re: [PR] Add ASF DOAP rdf file [iceberg]

2023-10-10 Thread via GitHub
jbonofre commented on code in PR #8586: URL: https://github.com/apache/iceberg/pull/8586#discussion_r1352566918 ## doap.rdf: ## @@ -0,0 +1,55 @@ + + +http://usefulinc.com/ns/doap#"; + xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"; + xmlns:asfext="http

Re: [PR] Add ASF DOAP rdf file [iceberg]

2023-10-10 Thread via GitHub
jbonofre commented on code in PR #8586: URL: https://github.com/apache/iceberg/pull/8586#discussion_r1352567808 ## doap.rdf: ## @@ -0,0 +1,55 @@ + + +http://usefulinc.com/ns/doap#"; + xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"; + xmlns:asfext="http

Re: [PR] Add ASF DOAP rdf file [iceberg]

2023-10-10 Thread via GitHub
jbonofre commented on code in PR #8586: URL: https://github.com/apache/iceberg/pull/8586#discussion_r1352568422 ## doap.rdf: ## @@ -0,0 +1,55 @@ + + +http://usefulinc.com/ns/doap#"; + xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"; + xmlns:asfext="http

Re: [PR] Build: Bump slf4j from 1.7.36 to 2.0.9 [iceberg]

2023-10-10 Thread via GitHub
jbonofre commented on PR #8737: URL: https://github.com/apache/iceberg/pull/8737#issuecomment-1755484959 @dependabot rebase -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] Build: Bump slf4j from 1.7.36 to 2.0.9 [iceberg]

2023-10-10 Thread via GitHub
jbonofre commented on PR #8737: URL: https://github.com/apache/iceberg/pull/8737#issuecomment-1755484513 It should work with most of the engines. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

Re: [PR] Build: Bump slf4j from 1.7.36 to 2.0.9 [iceberg]

2023-10-10 Thread via GitHub
dependabot[bot] commented on PR #8737: URL: https://github.com/apache/iceberg/pull/8737#issuecomment-1755485022 Sorry, only users with push access can use that command. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

Re: [I] Upgrade to Apache Arrow 13.0.0 [iceberg]

2023-10-10 Thread via GitHub
Fokko commented on issue #8764: URL: https://github.com/apache/iceberg/issues/8764#issuecomment-1755497790 Curious why this hasn't been picked up by dependabot. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

Re: [I] Upgrade to Apache Arrow 13.0.0 [iceberg]

2023-10-10 Thread via GitHub
jbonofre commented on issue #8764: URL: https://github.com/apache/iceberg/issues/8764#issuecomment-1755501456 @Fokko I found a few dependencies not detected by dependabot. I'm doing the updates and checking why dependabot didn't find it :) -- This is an automated message from the Apache

Re: [I] Investigate why dependabot didn't detect upgrades [iceberg]

2023-10-10 Thread via GitHub
ajantha-bhat commented on issue #8764: URL: https://github.com/apache/iceberg/issues/8764#issuecomment-1755583628 @snazy: Hi, Do you already have analyzed or have an info on this dependabot + version catalog problems? Do you recommend [renovateBot](https://github.com/renovatebot) to add

Re: [I] Investigate why dependabot didn't detect upgrades [iceberg]

2023-10-10 Thread via GitHub
Fokko commented on issue #8764: URL: https://github.com/apache/iceberg/issues/8764#issuecomment-1755598241 I think this is because we limit to 5 PRs: https://github.com/apache/iceberg/blob/master/.github/dependabot.yml#L32 It looks like all five are open: https://github.com/apache/ic

Re: [PR] Core: Allow missing object in ErrorResponse [iceberg]

2023-10-10 Thread via GitHub
Fokko commented on PR #8760: URL: https://github.com/apache/iceberg/pull/8760#issuecomment-1755602836 Let's do this the other way around -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

Re: [PR] Core: Allow missing object in ErrorResponse [iceberg]

2023-10-10 Thread via GitHub
Fokko closed pull request #8760: Core: Allow missing object in ErrorResponse URL: https://github.com/apache/iceberg/pull/8760 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[PR] Open-API: Make error required [iceberg]

2023-10-10 Thread via GitHub
Fokko opened a new pull request, #8765: URL: https://github.com/apache/iceberg/pull/8765 I think we want to make `error` required, otherwise it would just be an empty document `{}`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Gi

Re: [I] Investigate why dependabot didn't detect upgrades [iceberg]

2023-10-10 Thread via GitHub
jbonofre commented on issue #8764: URL: https://github.com/apache/iceberg/issues/8764#issuecomment-1755627197 @Fokko yes, I think we use the 5 PRs pool. +1 to upgrade. I'm doing it in a PR attached to this issue. -- This is an automated message from the Apache Git Service. To respond to t

Re: [PR] Docs: Fix missing semicolons in SQL snippets. [iceberg]

2023-10-10 Thread via GitHub
Fokko commented on code in PR #8748: URL: https://github.com/apache/iceberg/pull/8748#discussion_r1352755607 ## docs/spark-getting-started.md: ## @@ -69,7 +69,7 @@ To create your first Iceberg table in Spark, use the `spark-sql` shell or `spark ```sql -- local is the path-b

Re: [I] Unable to write to iceberg table using spark [iceberg]

2023-10-10 Thread via GitHub
RussellSpitzer commented on issue #8419: URL: https://github.com/apache/iceberg/issues/8419#issuecomment-1755633933 Pyspark I think has some issues with setting "packages" in the Spark conf since the py4j execution means that the Spark Context has to be started a bit weirdly. I would try us

Re: [PR] Docs: Fix missing semicolons in SQL snippets. [iceberg]

2023-10-10 Thread via GitHub
Fokko commented on PR #8748: URL: https://github.com/apache/iceberg/pull/8748#issuecomment-1755637911 Great work @Priyansh121096 If we find more we can create a new PR. (I also noticed that some blocks start with: ``` ```SQL ``` For consistency it would be nice to have everything

Re: [PR] Docs: Fix missing semicolons in SQL snippets. [iceberg]

2023-10-10 Thread via GitHub
Fokko merged PR #8748: URL: https://github.com/apache/iceberg/pull/8748 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apach

Re: [PR] Core: Use more permissive check when registering existing table [iceberg]

2023-10-10 Thread via GitHub
Fokko merged PR #8759: URL: https://github.com/apache/iceberg/pull/8759 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apach

Re: [PR] Build: Bump slf4j from 1.7.36 to 2.0.9 [iceberg]

2023-10-10 Thread via GitHub
nastra commented on PR #8737: URL: https://github.com/apache/iceberg/pull/8737#issuecomment-1755681588 @jbonofre there's an issue with Spark that needs some investigation -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[I] Parquet.write to S3 with GlueCatalog requires commit [iceberg]

2023-10-10 Thread via GitHub
djchapm opened a new issue, #8767: URL: https://github.com/apache/iceberg/issues/8767 ### Feature Request / Improvement Hi, writing this in an effort to improve documentation - I spent a crazy amount of time writing to glue catalog and parquet-avro files in S3 with Iceberg, but could

Re: [PR] Build: Fix compiler warnings [iceberg]

2023-10-10 Thread via GitHub
ajantha-bhat commented on code in PR #8763: URL: https://github.com/apache/iceberg/pull/8763#discussion_r1352796153 ## nessie/src/main/java/org/apache/iceberg/nessie/NessieIcebergClient.java: ## @@ -477,7 +477,7 @@ public void commitTable( Branch branch = getApi()

Re: [PR] Build: Fix compiler warnings [iceberg]

2023-10-10 Thread via GitHub
ajantha-bhat commented on code in PR #8763: URL: https://github.com/apache/iceberg/pull/8763#discussion_r1352798678 ## nessie/src/main/java/org/apache/iceberg/nessie/NessieIcebergClient.java: ## @@ -477,7 +477,7 @@ public void commitTable( Branch branch = getApi()

Re: [PR] Build: Bump slf4j from 1.7.36 to 2.0.9 [iceberg]

2023-10-10 Thread via GitHub
jbonofre commented on PR #8737: URL: https://github.com/apache/iceberg/pull/8737#issuecomment-1755700578 @nastra on a specific Spark version or any ? I can take a look if you want :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Gi

Re: [PR] Build: Add note about running tests/itests on MacOS [iceberg]

2023-10-10 Thread via GitHub
ajantha-bhat commented on PR #8766: URL: https://github.com/apache/iceberg/pull/8766#issuecomment-1755702793 > LGTM, but would be great if somebody with OSX could confirm this I can confirm. I use MAC. Not just for Iceberg project, any project that uses `TestContainers` on MAC will th

Re: [PR] Build: Add note about running tests/itests on MacOS [iceberg]

2023-10-10 Thread via GitHub
jbonofre commented on PR #8766: URL: https://github.com/apache/iceberg/pull/8766#issuecomment-1755703566 @nastra actually it's the workaround I have to do on my Mac :) I'm using MacOS (tested both on 13 & 14 with Docker Desktop) on M1. -- This is an automated message from the Apache Git S

Re: [PR] Build: increase open-pull-requests-limit to 50 [iceberg]

2023-10-10 Thread via GitHub
Fokko commented on code in PR #8768: URL: https://github.com/apache/iceberg/pull/8768#discussion_r1352807655 ## .github/dependabot.yml: ## @@ -28,6 +28,6 @@ updates: directory: "/" schedule: interval: "weekly" - day: "sunday" -open-pull-requests-limit:

Re: [PR] Build: increase open-pull-requests-limit to 50 [iceberg]

2023-10-10 Thread via GitHub
jbonofre commented on code in PR #8768: URL: https://github.com/apache/iceberg/pull/8768#discussion_r1352811489 ## .github/dependabot.yml: ## @@ -28,6 +28,6 @@ updates: directory: "/" schedule: interval: "weekly" - day: "sunday" -open-pull-requests-limi

Re: [PR] Build: Bump slf4j from 1.7.36 to 2.0.9 [iceberg]

2023-10-10 Thread via GitHub
nastra commented on PR #8737: URL: https://github.com/apache/iceberg/pull/8737#issuecomment-1755713705 https://github.com/apache/iceberg/actions/runs/6455472887/job/17523030796?pr=8737 contains a CI run with failures -- This is an automated message from the Apache Git Service. To respond

Re: [PR] Build: Upgrade to gradle 8.4 [iceberg]

2023-10-10 Thread via GitHub
jbonofre commented on PR #8486: URL: https://github.com/apache/iceberg/pull/8486#issuecomment-1755722580 Unfortunately `gradle-revapi-plugin` doesn't seem super active (https://github.com/palantir/gradle-revapi). I think it's important to be up to date in regards of Gradle. I will pro

Re: [PR] Build: Fix compiler warnings [iceberg]

2023-10-10 Thread via GitHub
dimas-b commented on code in PR #8763: URL: https://github.com/apache/iceberg/pull/8763#discussion_r1352823020 ## nessie/src/main/java/org/apache/iceberg/nessie/NessieIcebergClient.java: ## @@ -477,7 +477,7 @@ public void commitTable( Branch branch = getApi()

Re: [PR] Build: Fix compiler warnings [iceberg]

2023-10-10 Thread via GitHub
dimas-b commented on code in PR #8763: URL: https://github.com/apache/iceberg/pull/8763#discussion_r1352824568 ## nessie/src/main/java/org/apache/iceberg/nessie/NessieIcebergClient.java: ## @@ -477,7 +477,7 @@ public void commitTable( Branch branch = getApi()

Re: [PR] Build: Fix compiler warnings [iceberg]

2023-10-10 Thread via GitHub
ajantha-bhat commented on code in PR #8763: URL: https://github.com/apache/iceberg/pull/8763#discussion_r1352827194 ## nessie/src/main/java/org/apache/iceberg/nessie/NessieIcebergClient.java: ## @@ -477,7 +477,7 @@ public void commitTable( Branch branch = getApi()

Re: [PR] Build: Fix compiler warnings [iceberg]

2023-10-10 Thread via GitHub
ajantha-bhat commented on code in PR #8763: URL: https://github.com/apache/iceberg/pull/8763#discussion_r1352827194 ## nessie/src/main/java/org/apache/iceberg/nessie/NessieIcebergClient.java: ## @@ -477,7 +477,7 @@ public void commitTable( Branch branch = getApi()

Re: [PR] Build: Fix compiler warnings [iceberg]

2023-10-10 Thread via GitHub
dimas-b commented on code in PR #8763: URL: https://github.com/apache/iceberg/pull/8763#discussion_r1352831841 ## nessie/src/main/java/org/apache/iceberg/nessie/NessieIcebergClient.java: ## @@ -477,7 +477,7 @@ public void commitTable( Branch branch = getApi()

Re: [PR] Build: Fix compiler warnings [iceberg]

2023-10-10 Thread via GitHub
dimas-b commented on code in PR #8763: URL: https://github.com/apache/iceberg/pull/8763#discussion_r1352831841 ## nessie/src/main/java/org/apache/iceberg/nessie/NessieIcebergClient.java: ## @@ -477,7 +477,7 @@ public void commitTable( Branch branch = getApi()

Re: [PR] Build: Upgrade to gradle 8.4 [iceberg]

2023-10-10 Thread via GitHub
ajantha-bhat commented on PR #8486: URL: https://github.com/apache/iceberg/pull/8486#issuecomment-1755743335 > Unfortunately gradle-revapi-plugin doesn't seem super active (https://github.com/palantir/gradle-revapi). I think it's important to be up to date in regards of Gradle. I will pro

Re: [PR] Build: Fix compiler warnings [iceberg]

2023-10-10 Thread via GitHub
dimas-b commented on code in PR #8763: URL: https://github.com/apache/iceberg/pull/8763#discussion_r1352835064 ## nessie/src/main/java/org/apache/iceberg/nessie/NessieIcebergClient.java: ## @@ -477,7 +477,7 @@ public void commitTable( Branch branch = getApi()

Re: [I] Upsert support for keyless Apache Flink tables [iceberg]

2023-10-10 Thread via GitHub
Ge commented on issue #8719: URL: https://github.com/apache/iceberg/issues/8719#issuecomment-1755752149 `SELECT word, COUNT(*) FROM word_table GROUP BY word;` is the retract stream: ``` Flink SQL> SELECT word, COUNT(*) FROM word_table GROUP BY word; ++

Re: [PR] Build: increase open-pull-requests-limit to 50 [iceberg]

2023-10-10 Thread via GitHub
jbonofre commented on PR #8768: URL: https://github.com/apache/iceberg/pull/8768#issuecomment-1755758543 @ajantha-bhat maybe we have 5 pending PRs not closed/merged, so blocking any new PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [PR] Build: increase open-pull-requests-limit to 50 [iceberg]

2023-10-10 Thread via GitHub
ajantha-bhat commented on PR #8768: URL: https://github.com/apache/iceberg/pull/8768#issuecomment-1755762122 > @ajantha-bhat maybe we have 5 pending PRs not closed/merged, so blocking any new PR. Yeah, anyways this change will definitely give a clarity if that was the problem. So

Re: [PR] Docs: Document all metadata tables. [iceberg]

2023-10-10 Thread via GitHub
nastra merged PR #8709: URL: https://github.com/apache/iceberg/pull/8709 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

Re: [I] Document all metadata tables [iceberg]

2023-10-10 Thread via GitHub
nastra closed issue #757: Document all metadata tables URL: https://github.com/apache/iceberg/issues/757 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Construct a writer tree [iceberg-python]

2023-10-10 Thread via GitHub
Fokko commented on PR #40: URL: https://github.com/apache/iceberg-python/pull/40#issuecomment-1755767706 Forgot to push, just pushed the latest changes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

Re: [PR] Disable merge-commit and enforce linear history [iceberg-python]

2023-10-10 Thread via GitHub
rdblue merged PR #57: URL: https://github.com/apache/iceberg-python/pull/57 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.a

Re: [I] Unable to write to iceberg table using spark [iceberg]

2023-10-10 Thread via GitHub
di2mot commented on issue #8419: URL: https://github.com/apache/iceberg/issues/8419#issuecomment-1755796639 This works for me in general it works: ``` ("spark.jars.packages", "org.apache.iceberg:iceberg-spark3:0.11.0"), ("spark.sql.extensions", "org.apache.iceberg.spark.extensions.I

Re: [PR] push down min/max/count to iceberg [iceberg]

2023-10-10 Thread via GitHub
atifiu commented on PR #6252: URL: https://github.com/apache/iceberg/pull/6252#issuecomment-1755857764 @huaxingao I was executing max/count query on iceberg table version 1.3.0 and Spark3.3.1 but unable to see aggregate pushdown i.e. LocalTableScan Cc: @RussellSpitzer `spark.s

Re: [PR] Spec: Add partition stats spec [iceberg]

2023-10-10 Thread via GitHub
aokolnychyi commented on code in PR #7105: URL: https://github.com/apache/iceberg/pull/7105#discussion_r1353203145 ## format/spec.md: ## @@ -702,6 +703,58 @@ Blob metadata is a struct with the following fields: | _optional_ | _optional_ | **`properties`** | `map` | Additional

Re: [PR] Spec: Add partition stats spec [iceberg]

2023-10-10 Thread via GitHub
aokolnychyi commented on PR #7105: URL: https://github.com/apache/iceberg/pull/7105#issuecomment-1756079392 I added this PR to our community sync. I am not sure I will be there this week but I'll sync with Russell and Yufei afterwards. -- This is an automated message from the Apache Git S

[PR] Fix column rename doc example to reflect correct API [iceberg-python]

2023-10-10 Thread via GitHub
cabhishek opened a new pull request, #59: URL: https://github.com/apache/iceberg-python/pull/59 * Rename column example in [this](https://py.iceberg.apache.org/api/#rename-column) doc is incorrect. * This PR updates the example to use `update.rename_column(...)` instead of `update.renam

Re: [PR] Avro: Add Avro-assisted name mapping [iceberg]

2023-10-10 Thread via GitHub
wmoustafa commented on code in PR #7392: URL: https://github.com/apache/iceberg/pull/7392#discussion_r1353271608 ## core/src/main/java/org/apache/iceberg/avro/AvroWithPartnerByStructureVisitor.java: ## @@ -93,14 +94,23 @@ private static T visitRecord( private static T visit

Re: [PR] Avro: Add Avro-assisted name mapping [iceberg]

2023-10-10 Thread via GitHub
wmoustafa commented on PR #7392: URL: https://github.com/apache/iceberg/pull/7392#issuecomment-1756153921 > I think this is ready. Just a few minor updates needed; mostly https://github.com/apache/iceberg/pull/7392/files#r1224853756. Addressed. -- This is an automated message from

Re: [PR] Data: Support reading default values from generic Avro readers [iceberg]

2023-10-10 Thread via GitHub
wmoustafa commented on code in PR #6004: URL: https://github.com/apache/iceberg/pull/6004#discussion_r1353272532 ## .palantir/revapi.yml: ## @@ -451,6 +451,15 @@ acceptedBreaks: - code: "java.field.removedWithConstant" old: "field org.apache.iceberg.TableProperties.H

Re: [PR] Data: Support reading default values from generic Avro readers [iceberg]

2023-10-10 Thread via GitHub
wmoustafa commented on PR #6004: URL: https://github.com/apache/iceberg/pull/6004#issuecomment-1756154580 > @wmoustafa, looks like there are test failures. Can you take a look? Fixed. -- This is an automated message from the Apache Git Service. To respond to the message, please log

  1   2   >