Re: [I] [DeleteManifest] Making file validation optional [iceberg]

dramaticlly commented on issue #10535: URL: https://github.com/apache/iceberg/issues/10535#issuecomment-2177054658 thank you Szehon, I think `DeleteFiles` API only work with data files, delete files but not for manifest files. I tried to leverage delete by file path in [DeleteFiles](https:

Re: [I] [DeleteManifest] Making file validation optional [iceberg]

szehon-ho commented on issue #10535: URL: https://github.com/apache/iceberg/issues/10535#issuecomment-2176968134 I guess they initially made rewriteManifest as a safe API. I wonder, does DeleteFiles API achieve this? The next commit will remove that manifest if all its contents are r

Re: [I] [DeleteManifest] Making file validation optional [iceberg]

dramaticlly commented on issue #10535: URL: https://github.com/apache/iceberg/issues/10535#issuecomment-2176943839 @aokolnychyi @szehon-ho I am wondering what do you think. Basically the problem we are trying to resolve is remove existing manifests from table and I hope to achieve th

Re: [PR] Spark Action to Analyze table [iceberg]

amogh-jahagirdar commented on code in PR #10288: URL: https://github.com/apache/iceberg/pull/10288#discussion_r1645022945 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/AnalyzeTableSparkAction.java: ## @@ -0,0 +1,150 @@ +/* + * Licensed to the Apache Software

Re: [I] [DeleteManifest] Making file validation optional [iceberg]

haizhou-zhao commented on issue #10535: URL: https://github.com/apache/iceberg/issues/10535#issuecomment-2176916293 cc: @dramaticlly -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[I] [DeleteManifest] Making file validation optional [iceberg]

haizhou-zhao opened a new issue, #10535: URL: https://github.com/apache/iceberg/issues/10535 ### Feature Request / Improvement ## Background This issue comes up when we are trying to fix a corrupted table where one manifest file (with active reference to data files) was (uninte

Re: [PR] Core: Throw CommitStateUnknownException if RuntimeException that is not marked as cleanable is thrown [iceberg]

amogh-jahagirdar commented on code in PR #10373: URL: https://github.com/apache/iceberg/pull/10373#discussion_r1645018496 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/RewriteManifestsSparkAction.java: ## @@ -355,8 +356,11 @@ private void replaceManifests(

Re: [PR] Spark Action to Analyze table [iceberg]

singhpk234 commented on code in PR #10288: URL: https://github.com/apache/iceberg/pull/10288#discussion_r1645018212 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/AnalyzeTableSparkAction.java: ## @@ -0,0 +1,150 @@ +/* + * Licensed to the Apache Software Found

Re: [PR] Core: Throw CommitStateUnknownException if RuntimeException that is not marked as cleanable is thrown [iceberg]

amogh-jahagirdar commented on code in PR #10373: URL: https://github.com/apache/iceberg/pull/10373#discussion_r1645017182 ## core/src/main/java/org/apache/iceberg/BaseTransaction.java: ## @@ -333,6 +333,8 @@ private void commitCreateTransaction() { // the commit failed an

Re: [PR] Core: Throw CommitStateUnknownException if RuntimeException that is not marked as cleanable is thrown [iceberg]

amogh-jahagirdar commented on PR #10373: URL: https://github.com/apache/iceberg/pull/10373#issuecomment-2176908633 @stevenzwu I updated so that instead of normalizing non-cleanable failures to `CommitStateUnknownExcpetion` now engines, in this case Spark will explicitly handle `CleanableFai

[I] iceberg-aws-bundle jar includes org.slf4j.LoggerFactory [iceberg]

devinrsmith opened a new issue, #10534: URL: https://github.com/apache/iceberg/issues/10534 ### Apache Iceberg version 1.5.2 (latest release) ### Query engine None ### Please describe the bug 🐞 I noticed some strange logging issues after depending on `icebe

Re: [PR] API: add resultSchema() method to StructTransform [iceberg]

stevenzwu commented on PR #10496: URL: https://github.com/apache/iceberg/pull/10496#issuecomment-2176866976 > @stevenzwu do we have a corresponding issue for this PR ? Your PR looks like a good idea to me. @jbonofre thanks for taking a look. I don't think Github issue is required for

Re: [PR] Spark Action to Analyze table [iceberg]

karuppayya commented on code in PR #10288: URL: https://github.com/apache/iceberg/pull/10288#discussion_r164931 ## api/src/main/java/org/apache/iceberg/actions/AnalyzeTable.java: ## @@ -0,0 +1,63 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or mo

Re: [I] Filter predicate is duplicated in `TableScanBuilder` and `TableScan` 🤦🏼‍♂️ [iceberg-rust]

sdd commented on issue #407: URL: https://github.com/apache/iceberg-rust/issues/407#issuecomment-2176740079 @liurenjie1024 can you add the `good first issue` label please? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and us

[I] Filter predicate is duplicated in `TableScanBuilder` and `TableScan` 🤦🏼‍♂️ [iceberg-rust]

sdd opened a new issue, #407: URL: https://github.com/apache/iceberg-rust/issues/407 Somehow, between my PR and @viirya's PR getting merged, we've ended up with the filter predicate being duplicated in the `TableScanBuilder` and the `TableScan`. See `predicates` and `filter`:

Re: [PR] feat: make BoundPredicate,Datum serializable [iceberg-rust]

sdd commented on code in PR #406: URL: https://github.com/apache/iceberg-rust/pull/406#discussion_r1644888305 ## crates/iceberg/src/spec/values.rs: ## @@ -2334,9 +2453,20 @@ mod _serde { }, RawLiteralEnum::Float(v) => match ty {

Re: [PR] Fix lower/upper-case not to depend on JVM locale [iceberg]

findepi commented on PR #10521: URL: https://github.com/apache/iceberg/pull/10521#issuecomment-2176672130 thank you @ajantha-bhat @nastra @dimas-b @Fokko for your reviews! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and us

Re: [PR] Remove some dead code, fill some javadocs [iceberg]

findepi commented on PR #10530: URL: https://github.com/apache/iceberg/pull/10530#issuecomment-2176669752 thank you @nastra @amogh-jahagirdar for your review! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

Re: [PR] Core: Skip uncommitted manifest cleanup if no retries have occurred for FastAppend [iceberg]

grantatspothero commented on PR #10523: URL: https://github.com/apache/iceberg/pull/10523#issuecomment-2176593226 > > This is incorrect for MergingSnapshotProducer which merges manifests by writing the unmerged manifests, creating a new merged manifest, and marking the unmerged manifests fo

Re: [PR] Core: Skip uncommitted manifest cleanup if no retries have occurred for FastAppend [iceberg]

amogh-jahagirdar commented on code in PR #10523: URL: https://github.com/apache/iceberg/pull/10523#discussion_r1644764063 ## core/src/main/java/org/apache/iceberg/SnapshotProducer.java: ## @@ -423,21 +423,23 @@ public void commit() { try { LOG.info("Committed snapsho

Re: [I] Race condition in CachingCatalog [iceberg]

singhpk234 commented on issue #10493: URL: https://github.com/apache/iceberg/issues/10493#issuecomment-2176532253 should we include the snapshot id in the cache key to mitigate this situation ? -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] Fix ancestors_of exception for tables with >=2000 snapshots [iceberg-python]

Fokko merged PR #821: URL: https://github.com/apache/iceberg-python/pull/821 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

[PR] Add history inspect table [iceberg-python]

ndrluis opened a new pull request, #828: URL: https://github.com/apache/iceberg-python/pull/828 Reference: #511 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscr

Re: [PR] Fix lower/upper-case not to depend on JVM locale [iceberg]

Fokko commented on PR #10521: URL: https://github.com/apache/iceberg/pull/10521#issuecomment-2176459288 Thanks @findepi for the PR, and thanks @ajantha-bhat, @nastra and @dimas-b for the review 🙌 -- This is an automated message from the Apache Git Service. To respond to the message, plea

Re: [PR] Fix lower/upper-case not to depend on JVM locale [iceberg]

Fokko merged PR #10521: URL: https://github.com/apache/iceberg/pull/10521 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] Support Remove Branch or Tag APIs [iceberg-python]

chinmay-bhat commented on code in PR #822: URL: https://github.com/apache/iceberg-python/pull/822#discussion_r1644663032 ## pyiceberg/table/__init__.py: ## @@ -1978,6 +2013,21 @@ def create_tag(self, snapshot_id: int, tag_name: str, max_ref_age_ms: Optional[i self._req

Re: [PR] Remove some dead code, fill some javadocs [iceberg]

amogh-jahagirdar merged PR #10530: URL: https://github.com/apache/iceberg/pull/10530 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@

Re: [PR] API: implement types timestamp_ns and timestamptz_ns [iceberg]

epgif commented on PR #9008: URL: https://github.com/apache/iceberg/pull/9008#issuecomment-2176351708 > @jacobmarble I'll try to do another round this week once merge conflicts have been resolved. Also I think it would be good to get a review from @rdblue on this one Conflicts resolv

[I] [Bug]: Can't create/update tables in REST nessie catalog via Spark. Iceberg Cannot read field "formatVersion" because "x0" is null [iceberg]

sdasdasd1 opened a new issue, #10533: URL: https://github.com/apache/iceberg/issues/10533 ### Apache Iceberg version 1.5.0 ### Query engine Spark ### Please describe the bug 🐞 ### What happened I cant create or append to any table in REST nessie catal

Re: [I] flink datastream api write iceberg table, finding exception: iceberg Caused by: org.apache.thrift.transport.TTransportException [iceberg]

hashmapybx closed issue #10527: flink datastream api write iceberg table, finding exception: iceberg Caused by: org.apache.thrift.transport.TTransportException URL: https://github.com/apache/iceberg/issues/10527 -- This is an automated message from the Apache Git Service. To respond to the m

Re: [PR] feat: make BoundPredicate,Datum serializable [iceberg-rust]

liurenjie1024 commented on PR #406: URL: https://github.com/apache/iceberg-rust/pull/406#issuecomment-2176223197 cc @sdd Do you have other concerns? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] feat: make BoundPredicate,Datum serializable [iceberg-rust]

liurenjie1024 commented on code in PR #406: URL: https://github.com/apache/iceberg-rust/pull/406#discussion_r1644548426 ## crates/iceberg/src/spec/values.rs: ## @@ -2334,9 +2453,20 @@ mod _serde { }, RawLiteralEnum::Float(v) => match ty {

Re: [I] Upgrade HadoopTableOperations.version from int32 to long64 [iceberg]

jkolash commented on issue #10277: URL: https://github.com/apache/iceberg/issues/10277#issuecomment-2176207807 Closing this issue, as it won't be relevant for snowflake anymore. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [I] Upgrade HadoopTableOperations.version from int32 to long64 [iceberg]

jkolash closed issue #10277: Upgrade HadoopTableOperations.version from int32 to long64 URL: https://github.com/apache/iceberg/issues/10277 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

Re: [PR] Fix ancestors_of exception for tables with >=2000 snapshots [iceberg-python]

ndrluis commented on PR #821: URL: https://github.com/apache/iceberg-python/pull/821#issuecomment-2176069528 @syun64 Thank you for the review. I made the changes as you asked and also made a slight change due to a mypy error regarding the snapshot_by_id type `Optional[int]`. -- This is a

Re: [PR] Data: Add a util to read write partition stats [iceberg]

ajantha-bhat commented on code in PR #10176: URL: https://github.com/apache/iceberg/pull/10176#discussion_r1644433728 ## core/src/main/java/org/apache/iceberg/PartitionStatsUtil.java: ## @@ -0,0 +1,213 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or

Re: [PR] Data: Add a util to read write partition stats [iceberg]

ajantha-bhat commented on code in PR #10176: URL: https://github.com/apache/iceberg/pull/10176#discussion_r1644433728 ## core/src/main/java/org/apache/iceberg/PartitionStatsUtil.java: ## @@ -0,0 +1,213 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or

Re: [PR] Fix ancestors_of exception for tables with >=2000 snapshots [iceberg-python]

ndrluis commented on code in PR #821: URL: https://github.com/apache/iceberg-python/pull/821#discussion_r1644420395 ## pyiceberg/table/snapshots.py: ## @@ -421,8 +421,16 @@ def set_when_positive(properties: Dict[str, str], num: int, property_name: str) def ancestors_of(curre

Re: [PR] Fix ancestors_of exception for tables with >=2000 snapshots [iceberg-python]

syun64 commented on PR #821: URL: https://github.com/apache/iceberg-python/pull/821#issuecomment-2176031538 > Hi @syun64, this is the stacktrace > > ``` > Traceback (most recent call last): > File "/Users/magus/Workspace/oss/iceberg-python/test_view.py", line 18, in >

Re: [PR] Run revapi workflow on workflow/build system changes [iceberg]

findepi commented on PR #10485: URL: https://github.com/apache/iceberg/pull/10485#issuecomment-2176012792 thanks for the merge! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

Re: [PR] Drop support for Java 8 [iceberg]

findepi commented on code in PR #10518: URL: https://github.com/apache/iceberg/pull/10518#discussion_r1644403006 ## baseline.gradle: ## @@ -73,39 +73,57 @@ subprojects { options.errorprone.errorproneArgs.addAll ( // error-prone is slow, don't run on tests/gener

Re: [PR] Fix ancestors_of exception for tables with >=2000 snapshots [iceberg-python]

Fokko commented on code in PR #821: URL: https://github.com/apache/iceberg-python/pull/821#discussion_r1644359888 ## pyiceberg/table/snapshots.py: ## @@ -421,8 +421,16 @@ def set_when_positive(properties: Dict[str, str], num: int, property_name: str) def ancestors_of(current

Re: [PR] Drop support for Java 8 [iceberg]

nastra commented on code in PR #10518: URL: https://github.com/apache/iceberg/pull/10518#discussion_r1644358132 ## baseline.gradle: ## @@ -73,39 +73,57 @@ subprojects { options.errorprone.errorproneArgs.addAll ( // error-prone is slow, don't run on tests/genera

Re: [PR] Flink: move flush operation from prepareSnapshotPreBarrier to snapshotState [iceberg]

pvary commented on PR #10526: URL: https://github.com/apache/iceberg/pull/10526#issuecomment-2175921255 > > @zhongqishang: We have 2 data files with the same data. I suspect that the 1st data file is generated in the 1st checkpoint, and the 2nd data file is generated in the 2nd checkpoint.

Re: [PR] Run revapi workflow on workflow/build system changes [iceberg]

nastra merged PR #10485: URL: https://github.com/apache/iceberg/pull/10485 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] API: implement types timestamp_ns and timestamptz_ns [iceberg]

nastra commented on PR #9008: URL: https://github.com/apache/iceberg/pull/9008#issuecomment-2175917761 @jacobmarble I'll try to do another round this week once merge conflicts have been resolved. Also I think it would be good to get a review from @rdblue on this one -- This is an automa

Re: [PR] Remove some dead code, fill some javadocs [iceberg]

findepi commented on code in PR #10530: URL: https://github.com/apache/iceberg/pull/10530#discussion_r1644281005 ## flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/source/enumerator/EnumerationHistory.java: ## @@ -78,7 +78,11 @@ synchronized boolean hasFullHistory() {

Re: [PR] Data: Add a util to read write partition stats [iceberg]

lirui-apache commented on code in PR #10176: URL: https://github.com/apache/iceberg/pull/10176#discussion_r1644281052 ## core/src/main/java/org/apache/iceberg/PartitionStatsUtil.java: ## @@ -0,0 +1,213 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or

Re: [PR] Drop support for Java 8 [iceberg]

findepi commented on code in PR #10518: URL: https://github.com/apache/iceberg/pull/10518#discussion_r1644279376 ## baseline.gradle: ## @@ -73,39 +73,57 @@ subprojects { options.errorprone.errorproneArgs.addAll ( // error-prone is slow, don't run on tests/gener

Re: [PR] Spark Action to Analyze table [iceberg]

findepi commented on code in PR #10288: URL: https://github.com/apache/iceberg/pull/10288#discussion_r1644268788 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/AnalyzeTableSparkAction.java: ## @@ -47,13 +55,20 @@ public class AnalyzeTableSparkAction extends

Re: [PR] Drop support for Java 8 [iceberg]

nastra commented on code in PR #10518: URL: https://github.com/apache/iceberg/pull/10518#discussion_r1644271939 ## baseline.gradle: ## @@ -73,39 +73,57 @@ subprojects { options.errorprone.errorproneArgs.addAll ( // error-prone is slow, don't run on tests/genera

Re: [PR] Core：Simplify newTableMetadata method in TableMetadata class [iceberg]

nastra merged PR #10528: URL: https://github.com/apache/iceberg/pull/10528 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Remove some dead code, fill some javadocs [iceberg]

nastra commented on code in PR #10530: URL: https://github.com/apache/iceberg/pull/10530#discussion_r1644264209 ## flink/v1.17/flink/src/main/java/org/apache/iceberg/flink/source/enumerator/EnumerationHistory.java: ## @@ -78,7 +78,11 @@ synchronized boolean hasFullHistory() {

Re: [PR] Remove some dead code, fill some javadocs [iceberg]

nastra commented on code in PR #10530: URL: https://github.com/apache/iceberg/pull/10530#discussion_r1644263768 ## flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/source/enumerator/EnumerationHistory.java: ## @@ -78,7 +78,11 @@ synchronized boolean hasFullHistory() {

Re: [PR] Require Assertions to be statically imported [iceberg]

findepi commented on PR #10517: URL: https://github.com/apache/iceberg/pull/10517#issuecomment-2175825275 thanks for your review @nastra ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

Re: [PR] Fix lower/upper-case not to depend on JVM locale [iceberg]

findepi commented on PR #10521: URL: https://github.com/apache/iceberg/pull/10521#issuecomment-2175820686 there is a conflict (`core/src/test/java/org/apache/iceberg/hadoop/TestStaticTable.java`), rebasing -- This is an automated message from the Apache Git Service. To respond to the mess

Re: [PR] Fix lower/upper-case not to depend on JVM locale [iceberg]

findepi commented on code in PR #10521: URL: https://github.com/apache/iceberg/pull/10521#discussion_r1644259514 ## api/src/main/java/org/apache/iceberg/catalog/TableIdentifier.java: ## @@ -80,7 +81,7 @@ public String name() { public TableIdentifier toLowerCase() { Strin

Re: [PR] Require Assertions to be statically imported [iceberg]

nastra merged PR #10517: URL: https://github.com/apache/iceberg/pull/10517 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Require Assertions to be statically imported [iceberg]

nastra commented on PR #10517: URL: https://github.com/apache/iceberg/pull/10517#issuecomment-2175689599 This can be rebased now, since the other 2 PRs have been merged -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

Re: [PR] Import Assertions statically in Flink modules [iceberg]

nastra merged PR #10532: URL: https://github.com/apache/iceberg/pull/10532 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Require Assertions to be statically imported [iceberg]

findepi commented on PR #10517: URL: https://github.com/apache/iceberg/pull/10517#issuecomment-2175690800 rebased -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubsc

Re: [PR] Import Assertions statically in Spark modules [iceberg]

findepi commented on PR #10531: URL: https://github.com/apache/iceberg/pull/10531#issuecomment-2175689411 thanks @nastra for the merge! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

Re: [PR] Import Assertions statically in Spark modules [iceberg]

nastra merged PR #10531: URL: https://github.com/apache/iceberg/pull/10531 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Import Assertions statically in Flink modules [iceberg]

findepi commented on PR #10532: URL: https://github.com/apache/iceberg/pull/10532#issuecomment-2175686072 thanks @pvary @nastra for review and merge! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] Core：Simplify newTableMetadata method in TableMetadata class [iceberg]

nastra commented on code in PR #10528: URL: https://github.com/apache/iceberg/pull/10528#discussion_r1644157445 ## core/src/main/java/org/apache/iceberg/TableMetadata.java: ## @@ -74,11 +74,7 @@ public static TableMetadata newTableMetadata( public static TableMetadata newTabl

Re: [PR] Flink: move flush operation from prepareSnapshotPreBarrier to snapshotState [iceberg]

pvary commented on PR #10526: URL: https://github.com/apache/iceberg/pull/10526#issuecomment-2175624396 @zhongqishang: We have 2 data files with the same data. I suspect that the 1st data file is generated in the 1st checkpoint, and the 2nd data file is generated in the 2nd checkpoint. Coul

Re: [PR] Support building with Java 21 [iceberg]

jbonofre commented on PR #10474: URL: https://github.com/apache/iceberg/pull/10474#issuecomment-2175532753 @findepi thanks. FYI, I'm fixing the `gradle-revapi` plugin and Ed and I will do the release, then, I will rebase the Gradle 8.8 PR and we are good 😄 -- This is an automated message

[PR] Remove some dead code, fill some javadocs [iceberg]

findepi opened a new pull request, #10530: URL: https://github.com/apache/iceberg/pull/10530 Found by https://github.com/apache/iceberg/pull/10518 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Core：Simplify newTableMetadata method in TableMetadata class [iceberg]

nastra commented on code in PR #10528: URL: https://github.com/apache/iceberg/pull/10528#discussion_r1644064255 ## core/src/main/java/org/apache/iceberg/TableMetadata.java: ## @@ -74,11 +74,7 @@ public static TableMetadata newTableMetadata( public static TableMetadata newTabl

Re: [PR] Run Flink tests on Java 17 too [iceberg]

findepi commented on PR #10477: URL: https://github.com/apache/iceberg/pull/10477#issuecomment-2175517712 Thanks @singhpk234 @stevenzwu @pvary @nastra @Fokko for your reviews and thank you @nastra @Fokko for approvals. Is there anything I need to change here? -- This is an automated m

Re: [I] Concerns with `String.toLowerCase()` in default Locale [iceberg]

findepi commented on issue #9276: URL: https://github.com/apache/iceberg/issues/9276#issuecomment-2175487722 > I did a quick scan of the calls to `String.toLowerCase()` in Iceberg codebase The toLowerCase & toUpperCase calls are being fixed in https://github.com/apache/iceberg/pull/1

Re: [PR] Fix lower/upper-case not to depend on JVM locale [iceberg]

findepi commented on PR #10521: URL: https://github.com/apache/iceberg/pull/10521#issuecomment-2175486324 @ajantha-bhat thanks for the link! I am not sure this is a full fix though. There may be other Locale-dependent APIs besides toLowerCase & toUpperCase. Let's keep the issue open f

Re: [PR] Support building with Java 21 [iceberg]

findepi commented on PR #10474: URL: https://github.com/apache/iceberg/pull/10474#issuecomment-2175527255 https://github.com/apache/iceberg/pull/10477, https://github.com/apache/iceberg/pull/10482, https://github.com/apache/iceberg/pull/10485 were extracted from here. Will rebase this PR o

Re: [PR] Run Hive3 tests on Java 11 and 17 too [iceberg]

findepi commented on code in PR #10482: URL: https://github.com/apache/iceberg/pull/10482#discussion_r1644050241 ## .github/workflows/hive-ci.yml: ## @@ -94,14 +94,17 @@ jobs: hive3-tests: runs-on: ubuntu-22.04 +strategy: + matrix: +jvm: [8, 11, 17] R

Re: [I] Concerns with `String.toLowerCase()` in default Locale [iceberg]

findepi commented on issue #9276: URL: https://github.com/apache/iceberg/issues/9276#issuecomment-2175479755 > Does Iceberg support using its libraries in user-defined locales? Not for me to decide this, but i believe that we have basically these options 1. make the code indepe

Re: [I] How to set Spark conf to use Parquet and Iceberg tables using glue catalog without catalog name(spark_catalog)? [iceberg]

salah-djb commented on issue #7748: URL: https://github.com/apache/iceberg/issues/7748#issuecomment-2175422262 Hi there, anyone found a solution for querying non iceberg tables? @akshayakp97 When using SparkSessionCatalog tables are not found (`TABLE_OR_VIEW_NOT_FOUND` error) -- This

Re: [PR] Flink: Maintenance - TriggerManager [iceberg]

pvary commented on code in PR #10484: URL: https://github.com/apache/iceberg/pull/10484#discussion_r1643990192 ## flink/v1.19/flink/src/test/java/org/apache/iceberg/flink/maintenance/operator/TestTriggerManager.java: ## @@ -0,0 +1,587 @@ +/* + * Licensed to the Apache Software F

Re: [PR] Flink: Maintenance - TriggerManager [iceberg]

pvary commented on code in PR #10484: URL: https://github.com/apache/iceberg/pull/10484#discussion_r1643989763 ## flink/v1.19/flink/src/test/java/org/apache/iceberg/flink/maintenance/operator/TestTriggerManager.java: ## @@ -0,0 +1,587 @@ +/* + * Licensed to the Apache Software F

Re: [PR] Flink: Maintenance - TriggerManager [iceberg]

pvary commented on code in PR #10484: URL: https://github.com/apache/iceberg/pull/10484#discussion_r1643989141 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/TriggerManager.java: ## @@ -0,0 +1,295 @@ +/* + * Licensed to the Apache Software Found

Re: [PR] Flink: Maintenance - TriggerManager [iceberg]

pvary commented on code in PR #10484: URL: https://github.com/apache/iceberg/pull/10484#discussion_r1643988437 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/TriggerEvaluator.java: ## @@ -0,0 +1,127 @@ +/* + * Licensed to the Apache Software Fou

Re: [I] flink datastream api write iceberg table, finding exception: iceberg Caused by: org.apache.thrift.transport.TTransportException [iceberg]

hashmapybx commented on issue #10527: URL: https://github.com/apache/iceberg/issues/10527#issuecomment-2175440373 > @hashmapybx: Seems like an issue with the HMS communication. hi, can you help me please. Or the a issue with HMS conf. my. conf is down: properties1.put("type", "iceberg")

Re: [PR] Flink: Maintenance - TriggerManager [iceberg]

pvary commented on code in PR #10484: URL: https://github.com/apache/iceberg/pull/10484#discussion_r1643992382 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/TriggerManager.java: ## @@ -0,0 +1,295 @@ +/* + * Licensed to the Apache Software Found

Re: [PR] Flink: Maintenance - TriggerManager [iceberg]

pvary commented on code in PR #10484: URL: https://github.com/apache/iceberg/pull/10484#discussion_r1643987852 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/TagBasedLockFactory.java: ## @@ -0,0 +1,139 @@ +/* + * Licensed to the Apache Software

Re: [PR] Flink: Maintenance - TriggerManager [iceberg]

pvary commented on code in PR #10484: URL: https://github.com/apache/iceberg/pull/10484#discussion_r1643986681 ## core/src/main/java/org/apache/iceberg/SerializableTable.java: ## @@ -397,6 +398,25 @@ private String errorMsg(String operation) { return String.format("Operatio

Re: [PR] Docs: Add flinkVersion and flinkVersionMajor instead of hardcode [iceberg]

Fokko merged PR #10463: URL: https://github.com/apache/iceberg/pull/10463 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [I] End-of-life Flink 1.16 is still referenced in docs [iceberg]

Fokko closed issue #10412: End-of-life Flink 1.16 is still referenced in docs URL: https://github.com/apache/iceberg/issues/10412 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] Bump griffe from 0.45.3 to 0.46.1 [iceberg-python]

Fokko merged PR #824: URL: https://github.com/apache/iceberg-python/pull/824 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] Bump mypy-boto3-glue from 1.34.126 to 1.34.128 [iceberg-python]

Fokko merged PR #825: URL: https://github.com/apache/iceberg-python/pull/825 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] Bump mkdocs-material from 9.5.26 to 9.5.27 [iceberg-python]

Fokko merged PR #826: URL: https://github.com/apache/iceberg-python/pull/826 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] Require Assertions to be statically imported [iceberg]

nastra commented on PR #10517: URL: https://github.com/apache/iceberg/pull/10517#issuecomment-2175335632 @findepi yeah I think it would make sense to split out the Flink + Spark changes. Everything else can stay in this PR -- This is an automated message from the Apache Git Service. To re

Re: [PR] Bump urllib3 from 1.26.18 to 1.26.19 [iceberg-python]

Fokko merged PR #823: URL: https://github.com/apache/iceberg-python/pull/823 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] spec: Fix formatting of Default values [iceberg]

Fokko commented on PR #10525: URL: https://github.com/apache/iceberg/pull/10525#issuecomment-2175352898 Thanks @szehon-ho and @ajantha-bhat for the review 👍 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

Re: [PR] spec: Fix formatting of Default values [iceberg]

Fokko merged PR #10525: URL: https://github.com/apache/iceberg/pull/10525 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] Core：Simplify newTableMetadata method in TableMetadata class [iceberg]

DaqianLiao commented on code in PR #10528: URL: https://github.com/apache/iceberg/pull/10528#discussion_r1643939337 ## core/src/main/java/org/apache/iceberg/TableMetadata.java: ## @@ -74,11 +74,7 @@ public static TableMetadata newTableMetadata( public static TableMetadata new

Re: [I] How to query NON Iceberg format tables (created in aws Athena) from spark? [iceberg]

salahdjb commented on issue #10524: URL: https://github.com/apache/iceberg/issues/10524#issuecomment-2175328324 Still same error, table is not found -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to