Re: [PR] Override useCommitCoordinator to false [iceberg]

jiantao-vungle commented on PR #9017: URL: https://github.com/apache/iceberg/pull/9017#issuecomment-1818405200 > @jiantao-vungle The problem you encountered is probably the same as the one in this PR. Are you experiencing this issue only in Spark 3.4? @huaxingao, actually, it was enco

Re: [PR] Override useCommitCoordinator to false [iceberg]

Fokko commented on PR #9017: URL: https://github.com/apache/iceberg/pull/9017#issuecomment-1818401583 Looping in @jbonofre here who's preparing the 1.4.3 release. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [I] Failed to create namespace using spark sql based on iceberg hadoop catalog (rest catalog) [iceberg]

Fokko commented on issue #9072: URL: https://github.com/apache/iceberg/issues/9072#issuecomment-1818400054 Hey @TCGOGOGO Can you provide a bit more context here? The Hadoop Catalog and Rest Catalog are two different catalogs, and they should not be combined. -- This is an automated messag

Re: [PR] Core: Adding Serializing TableMetadataLocation in TableMetadataParser [iceberg]

nastra closed pull request #9034: Core: Adding Serializing TableMetadataLocation in TableMetadataParser URL: https://github.com/apache/iceberg/pull/9034 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] Build: Bump software.amazon.awssdk:bom from 2.21.22 to 2.21.26 [iceberg]

nastra merged PR #9105: URL: https://github.com/apache/iceberg/pull/9105 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

Re: [I] write data with a specific tag [iceberg]

nastra commented on issue #9111: URL: https://github.com/apache/iceberg/issues/9111#issuecomment-1818383938 @Jake-00 you can already create a Tag right after writing, but it takes an additional SQL call: https://iceberg.apache.org/docs/latest/spark-ddl/#alter-table--create-tag That b

Re: [I] write data with a specific tag [iceberg]

nastra closed issue #9111: write data with a specific tag URL: https://github.com/apache/iceberg/issues/9111 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-m

Re: [PR] Override useCommitCoordinator to false [iceberg]

huaxingao commented on PR #9017: URL: https://github.com/apache/iceberg/pull/9017#issuecomment-1818356916 @aokolnychyi Do we need to back-port the fix to previous releases? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

Re: [PR] Override useCommitCoordinator to false [iceberg]

huaxingao commented on PR #9017: URL: https://github.com/apache/iceberg/pull/9017#issuecomment-1818355654 @jiantao-vungle The problem you encountered is probably the same as the one in this PR. Are you experiencing this issue only in Spark 3.4? -- This is an automated message from the Apa

Re: [I] Failed to create namespace using spark sql based on iceberg hadoop catalog (rest catalog) [iceberg]

TCGOGOGO commented on issue #9072: URL: https://github.com/apache/iceberg/issues/9072#issuecomment-1818327317 @XiaokunDing Hi xiaokun, could you please help take a look? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[I] Decouple modules for Hive 2 and 3 Versions and make Hive 3 as default [iceberg]

chinnaraolalam opened a new issue, #9113: URL: https://github.com/apache/iceberg/issues/9113 ### Apache Iceberg version 1.3.0 ### Query engine Hive ### Please describe the bug 🐞 Iceberg supports Hive 2 and Hive 3 versions, for the same it has below modu

Re: [PR] Override useCommitCoordinator to false [iceberg]

jiantao-vungle commented on PR #9017: URL: https://github.com/apache/iceberg/pull/9017#issuecomment-1818200739 hi @huaxingao, we tried to execute sql `INSERT` into a Iceberg table, but encountered following exception stack, it's like https://github.com/apache/iceberg/issues/8904, but withou

Re: [PR] Override useCommitCoordinator to false [iceberg]

jiantao-vungle commented on PR #9017: URL: https://github.com/apache/iceberg/pull/9017#issuecomment-1818197335 hi @huaxingao, we tried to execute sql `INSERT` into a Iceberg table, but encountered following exception stack, no Iceberg API calling in the stack, It seems like the same issue a

Re: [PR] Replace i64 with DateTime [iceberg-rust]

liurenjie1024 commented on PR #94: URL: https://github.com/apache/iceberg-rust/pull/94#issuecomment-1818192533 > Is there anything else I need to do to get this merged in? > > I don't (understandably) have permissions to merge this myself. https://user-images.githubusercontent.com/2050

Re: [PR] Replace i64 with DateTime [iceberg-rust]

fqaiser94 commented on PR #94: URL: https://github.com/apache/iceberg-rust/pull/94#issuecomment-1818190059 Is there anything else I need to do to get this merged in? I don't (understandably) have permissions to merge this myself. https://github.com/apache/iceberg-rust/assets/20507

Re: [PR] feat: Implement create table and update table api for rest catalog. [iceberg-rust]

liurenjie1024 commented on PR #97: URL: https://github.com/apache/iceberg-rust/pull/97#issuecomment-1818188492 This is already ready for review. cc @Fokko @Xuanwo @ZENOTME @JanKaul -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

Re: [I] feat: Add support for `UnboundPartitionSpec`. [iceberg-rust]

liurenjie1024 commented on issue #98: URL: https://github.com/apache/iceberg-rust/issues/98#issuecomment-1818147736 Hi, @my-vegetable-has-exploded Sorry for the confusion. `UnboundPartitionSpec` has two important use cases: 1. In [rest api's definition](https://github.com/apache/iceberg/

Re: [I] test: Rest catalog integration test. [iceberg-rust]

liurenjie1024 commented on issue #100: URL: https://github.com/apache/iceberg-rust/issues/100#issuecomment-1818111263 > @liurenjie1024 Thanks for raising this. This made the testing on PyIceberg much easier. Also, I've created a similar PR for duckdb: [duckdb/duckdb_iceberg#27](https://gith

Re: [I] org.apache.iceberg.hive.RuntimeMetaException: Failed to connect to Hive Metastore at [iceberg]

zhangbutao commented on issue #9030: URL: https://github.com/apache/iceberg/issues/9030#issuecomment-1818091848 You can also try to paly Hive4 version which can use iceberg out of box. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [I] org.apache.iceberg.hive.RuntimeMetaException: Failed to connect to Hive Metastore at [iceberg]

zyx199693 commented on issue #9030: URL: https://github.com/apache/iceberg/issues/9030#issuecomment-1818082100 Above Hive 3.0.0, libfb303 was removed from hive-exec.jar due to an issue (HIVE-18459). However, when working with Iceberg tables, libfb303 is necessary. To address this, you can r

Re: [PR] Spark: Add spark.sql.iceberg.locality to control whether fetch locali… [iceberg]

zhongyujiang commented on code in PR #9101: URL: https://github.com/apache/iceberg/pull/9101#discussion_r1398559537 ## spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/SparkSQLProperties.java: ## @@ -63,4 +63,7 @@ private SparkSQLProperties() {} // Controls the WAP bra

Re: [PR] Spark: Fix metadata delete condition check when there are branches [iceberg]

amogh-jahagirdar commented on code in PR #9102: URL: https://github.com/apache/iceberg/pull/9102#discussion_r1398496635 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/SparkRowLevelOperationsTestBase.java: ## @@ -135,7 +135,7 @@ public static Obj

Re: [I] spark insert partition table Data skew [iceberg]

github-actions[bot] commented on issue #7455: URL: https://github.com/apache/iceberg/issues/7455#issuecomment-1818027657 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] spark insert partition table Data skew [iceberg]

github-actions[bot] closed issue #7455: spark insert partition table Data skew URL: https://github.com/apache/iceberg/issues/7455 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [I] Explore: option of creating single delete file to reduce number of delete files in a partition in MOR [iceberg]

github-actions[bot] commented on issue #7690: URL: https://github.com/apache/iceberg/issues/7690#issuecomment-1818027614 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [PR] Spark: Fix metadata delete condition check when there are branches [iceberg]

rdblue commented on PR #9102: URL: https://github.com/apache/iceberg/pull/9102#issuecomment-1818021648 Merged. Thanks, @amogh-jahagirdar! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

Re: [I] Failure to delete with merge-on-read in branch and main branch [iceberg]

rdblue closed issue #7635: Failure to delete with merge-on-read in branch and main branch URL: https://github.com/apache/iceberg/issues/7635 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

Re: [PR] Spark: Fix metadata delete condition check when there are branches [iceberg]

rdblue merged PR #9102: URL: https://github.com/apache/iceberg/pull/9102 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

[I] Time Travel / Snapshot Performance [iceberg]

w14 opened a new issue, #9112: URL: https://github.com/apache/iceberg/issues/9112 The docs discuss time travel but not the performance / size requirements. 1. Does Iceberg store changes by row or whole table? So if you have a 100MiB table and you make 100 separates changes (inserts, u

Re: [PR] Spark: Fix metadata delete condition check when there are branches [iceberg]

amogh-jahagirdar commented on code in PR #9102: URL: https://github.com/apache/iceberg/pull/9102#discussion_r1398496635 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/SparkRowLevelOperationsTestBase.java: ## @@ -135,7 +135,7 @@ public static Obj

Re: [PR] Spark: Fix metadata delete condition check when there are branches [iceberg]

amogh-jahagirdar commented on code in PR #9102: URL: https://github.com/apache/iceberg/pull/9102#discussion_r1398496635 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/SparkRowLevelOperationsTestBase.java: ## @@ -135,7 +135,7 @@ public static Obj

Re: [PR] Spark: Fix metadata delete condition check when there are branches [iceberg]

amogh-jahagirdar commented on code in PR #9102: URL: https://github.com/apache/iceberg/pull/9102#discussion_r1398494928 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/data/TestHelpers.java: ## @@ -804,11 +805,7 @@ public static List dataFiles(Table table) { }

Re: [PR] Spark: Fix metadata delete condition check when there are branches [iceberg]

rdblue commented on code in PR #9102: URL: https://github.com/apache/iceberg/pull/9102#discussion_r1398492356 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/data/TestHelpers.java: ## @@ -804,11 +805,7 @@ public static List dataFiles(Table table) { } public st

Re: [PR] Spark: Fix metadata delete condition check when there are branches [iceberg]

rdblue commented on code in PR #9102: URL: https://github.com/apache/iceberg/pull/9102#discussion_r1398492178 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/SparkTable.java: ## @@ -371,16 +372,12 @@ public void deleteWhere(Predicate[] predicates) { ic

Re: [PR] Spark: Fix metadata delete condition check when there are branches [iceberg]

rdblue commented on code in PR #9102: URL: https://github.com/apache/iceberg/pull/9102#discussion_r1398492074 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/SparkTable.java: ## @@ -327,12 +331,9 @@ private boolean canDeleteUsingMetadata(Expression deleteExpr)

Re: [PR] Spark: Fix metadata delete condition check when there are branches [iceberg]

rdblue commented on code in PR #9102: URL: https://github.com/apache/iceberg/pull/9102#discussion_r1398490588 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/SparkRowLevelOperationsTestBase.java: ## @@ -135,7 +135,7 @@ public static Object[][] pa

Re: [PR] Spark: Fix metadata delete condition check when there are branches [iceberg]

rdblue commented on code in PR #9102: URL: https://github.com/apache/iceberg/pull/9102#discussion_r1398490588 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/SparkRowLevelOperationsTestBase.java: ## @@ -135,7 +135,7 @@ public static Object[][] pa

Re: [I] write data with a specific tag [iceberg]

Jake-00 commented on issue #9111: URL: https://github.com/apache/iceberg/issues/9111#issuecomment-1817870277 > Your suggestion however would imply that you first create a tag and then write data to it, which is not what a tag supports. My suggestion may not be clear enough. Indeed I want

[PR] chore(deps): Update itertools requirement from 0.11 to 0.12 [iceberg-rust]

dependabot[bot] opened a new pull request, #102: URL: https://github.com/apache/iceberg-rust/pull/102 Updates the requirements on [itertools](https://github.com/rust-itertools/itertools) to permit the latest version. Changelog Sourced from https://github.com/rust-itertools/itertoo

[PR] chore(deps): Update opendal requirement from 0.41 to 0.42 [iceberg-rust]

dependabot[bot] opened a new pull request, #101: URL: https://github.com/apache/iceberg-rust/pull/101 Updates the requirements on [opendal](https://github.com/apache/incubator-opendal) to permit the latest version. Release notes Sourced from https://github.com/apache/incubator-ope

Re: [PR] Build: Bump software.amazon.awssdk:bom from 2.21.22 to 2.21.26 [iceberg]

nastra commented on PR #9105: URL: https://github.com/apache/iceberg/pull/9105#issuecomment-1817802234 @dependabot rebase -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] Build: Bump com.fasterxml.jackson.dataformat:jackson-dataformat-xml from 2.9.9 to 2.16.0 [iceberg]

nastra merged PR #9107: URL: https://github.com/apache/iceberg/pull/9107 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

Re: [PR] Build: Bump org.testcontainers:testcontainers from 1.19.1 to 1.19.2 [iceberg]

nastra merged PR #9103: URL: https://github.com/apache/iceberg/pull/9103 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

Re: [PR] Build: Update pydantic requirement from <2.4.0 to <2.6.0 [iceberg]

dependabot[bot] closed pull request #9056: Build: Update pydantic requirement from <2.4.0 to <2.6.0 URL: https://github.com/apache/iceberg/pull/9056 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

Re: [PR] Build: Update pydantic requirement from <2.4.0 to <2.6.0 [iceberg]

dependabot[bot] commented on PR #9056: URL: https://github.com/apache/iceberg/pull/9056#issuecomment-1817801999 Looks like pydantic is no longer a dependency, so this is no longer needed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [PR] Build: Bump com.fasterxml.jackson.core:jackson-annotations from 2.15.3 to 2.16.0 [iceberg]

nastra merged PR #9106: URL: https://github.com/apache/iceberg/pull/9106 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

Re: [PR] Open-API: Remove pydantic pin [iceberg]

nastra merged PR #9110: URL: https://github.com/apache/iceberg/pull/9110 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

Re: [I] It sometimes throws exception java.lang.AssertionError: assertion failed after upgrade to Iceberg 1.3.1 + Spark 3.4.1 [iceberg]

nastra commented on issue #9092: URL: https://github.com/apache/iceberg/issues/9092#issuecomment-1817801003 @jiantao-vungle I would still check what the custom code is doing as it must be doing something that fails that particular assertion that was mentioned further above -- This is an

Re: [I] write data with a specific tag [iceberg]

nastra commented on issue #9111: URL: https://github.com/apache/iceberg/issues/9111#issuecomment-1817800700 > Referring to documentation, I can't find out a way to write data into table along with a specific tag This isn't supported and I don't think there's much value in supporting

Re: [PR] build(deps): bump github.com/aws/aws-sdk-go-v2/config from 1.19.1 to 1.23.0 [iceberg-go]

dependabot[bot] closed pull request #35: build(deps): bump github.com/aws/aws-sdk-go-v2/config from 1.19.1 to 1.23.0 URL: https://github.com/apache/iceberg-go/pull/35 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

Re: [PR] build(deps): bump github.com/aws/aws-sdk-go-v2/config from 1.19.1 to 1.23.0 [iceberg-go]

dependabot[bot] commented on PR #35: URL: https://github.com/apache/iceberg-go/pull/35#issuecomment-1817787503 Superseded by #38. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

[PR] build(deps): bump github.com/aws/aws-sdk-go-v2/config from 1.19.1 to 1.25.3 [iceberg-go]

dependabot[bot] opened a new pull request, #38: URL: https://github.com/apache/iceberg-go/pull/38 Bumps [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2) from 1.19.1 to 1.25.3. Commits https://github.com/aws/aws-sdk-go-v2/commit/0be05fa6cc06d8e81eda10

Re: [PR] build(deps): bump github.com/hamba/avro/v2 from 2.16.0 to 2.17.2 [iceberg-go]

dependabot[bot] closed pull request #33: build(deps): bump github.com/hamba/avro/v2 from 2.16.0 to 2.17.2 URL: https://github.com/apache/iceberg-go/pull/33 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] build(deps): bump github.com/hamba/avro/v2 from 2.16.0 to 2.17.2 [iceberg-go]

dependabot[bot] commented on PR #33: URL: https://github.com/apache/iceberg-go/pull/33#issuecomment-1817787354 Superseded by #37. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

[PR] build(deps): bump github.com/hamba/avro/v2 from 2.16.0 to 2.18.0 [iceberg-go]

dependabot[bot] opened a new pull request, #37: URL: https://github.com/apache/iceberg-go/pull/37 Bumps [github.com/hamba/avro/v2](https://github.com/hamba/avro) from 2.16.0 to 2.18.0. Release notes Sourced from https://github.com/hamba/avro/releases";>github.com/hamba/avro/v2's r

Re: [PR] build(deps): bump github.com/aws/aws-sdk-go-v2 from 1.21.2 to 1.22.2 [iceberg-go]

dependabot[bot] closed pull request #34: build(deps): bump github.com/aws/aws-sdk-go-v2 from 1.21.2 to 1.22.2 URL: https://github.com/apache/iceberg-go/pull/34 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[PR] build(deps): bump github.com/aws/aws-sdk-go-v2 from 1.21.2 to 1.23.0 [iceberg-go]

dependabot[bot] opened a new pull request, #36: URL: https://github.com/apache/iceberg-go/pull/36 Bumps [github.com/aws/aws-sdk-go-v2](https://github.com/aws/aws-sdk-go-v2) from 1.21.2 to 1.23.0. Commits https://github.com/aws/aws-sdk-go-v2/commit/435199fc01ab47020ab36dab07d811

Re: [PR] build(deps): bump github.com/aws/aws-sdk-go-v2 from 1.21.2 to 1.22.2 [iceberg-go]

dependabot[bot] commented on PR #34: URL: https://github.com/apache/iceberg-go/pull/34#issuecomment-1817787018 Superseded by #36. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

[I] write data with a specific tag [iceberg]

Jake-00 opened a new issue, #9111: URL: https://github.com/apache/iceberg/issues/9111 ### Feature Request / Improvement Background: Referring to documentation, I can't find out a way to add tag after writing data into table. Nowadays, the common workflow is to list all snapshots-id f