[PR] Support CreateTableTransaction for Hive and SQL Catalog [iceberg-python]

HonahX opened a new pull request, #611: URL: https://github.com/apache/iceberg-python/pull/611 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

Re: [PR] Flink: Don't fail to serialize IcebergSourceSplit when there is too many delete files [iceberg]

javrasya commented on code in PR #9464: URL: https://github.com/apache/iceberg/pull/9464#discussion_r1568359596 ## flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/source/split/SerializerHelper.java: ## @@ -0,0 +1,186 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] Flink: Don't fail to serialize IcebergSourceSplit when there is too many delete files [iceberg]

javrasya commented on code in PR #9464: URL: https://github.com/apache/iceberg/pull/9464#discussion_r1568359596 ## flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/source/split/SerializerHelper.java: ## @@ -0,0 +1,186 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] feat: Glue Catalog - table operations (3/3) [iceberg-rust]

marvinlanhenke commented on PR #314: URL: https://github.com/apache/iceberg-rust/pull/314#issuecomment-2060583038 @liurenjie1024 I think I fixed all of your suggestions - thanks again for the review. For the 'missing table checks' - I'll file an Issue later, so we can track this and i

Re: [PR] Docs: Update features for Hive 4.0 [iceberg]

ajantha-bhat commented on code in PR #10162: URL: https://github.com/apache/iceberg/pull/10162#discussion_r1568390242 ## docs/docs/hive.md: ## @@ -431,12 +466,120 @@ ALTER TABLE t SET TBLPROPERTIES ('storage_handler'='org.apache.iceberg.mr.hive.H During the migration the data

Re: [PR] Add Pagination To List Apis [iceberg]

nastra commented on code in PR #9782: URL: https://github.com/apache/iceberg/pull/9782#discussion_r1568398951 ## core/src/test/java/org/apache/iceberg/rest/TestRESTCatalog.java: ## @@ -2329,6 +2332,120 @@ public void multipleDiffsAgainstMultipleTablesLastFails() { assertTh

Re: [PR] Add Pagination To List Apis [iceberg]

nastra commented on code in PR #9782: URL: https://github.com/apache/iceberg/pull/9782#discussion_r1568415371 ## core/src/test/java/org/apache/iceberg/rest/TestRESTCatalog.java: ## @@ -2329,6 +2332,120 @@ public void multipleDiffsAgainstMultipleTablesLastFails() { assertTh

Re: [PR] Flink: Don't fail to serialize IcebergSourceSplit when there is too many delete files [iceberg]

pvary commented on PR #9464: URL: https://github.com/apache/iceberg/pull/9464#issuecomment-2060691225 @javrasya: Please fix the failures as well. Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] Flink: Don't fail to serialize IcebergSourceSplit when there is too many delete files [iceberg]

pvary commented on code in PR #9464: URL: https://github.com/apache/iceberg/pull/9464#discussion_r1568432766 ## flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/source/split/SerializerHelper.java: ## @@ -0,0 +1,186 @@ +/* + * Licensed to the Apache Software Foundation (A

Re: [PR] Backport Flink 1.18 JUnit5 migration to Flink 1.17 [iceberg]

tomtongue commented on PR #10163: URL: https://github.com/apache/iceberg/pull/10163#issuecomment-2060751325 @nastra Could you review this PR? If I should migrate more files to this PR, please let me know. -- This is an automated message from the Apache Git Service. To respond to the messa

Re: [PR] feat: Glue Catalog - table operations (3/3) [iceberg-rust]

marvinlanhenke commented on PR #314: URL: https://github.com/apache/iceberg-rust/pull/314#issuecomment-2060905965 > For the 'missing table checks' - I'll file an Issue later, so we can track this and implement in separate PRs. #337 -- This is an automated message from the Apache Gi

Re: [PR] Docs: Update features for Hive 4.0 [iceberg]

SourabhBadhya commented on code in PR #10162: URL: https://github.com/apache/iceberg/pull/10162#discussion_r1568611714 ## docs/docs/hive.md: ## @@ -431,12 +466,120 @@ ALTER TABLE t SET TBLPROPERTIES ('storage_handler'='org.apache.iceberg.mr.hive.H During the migration the data

Re: [PR] Docs: Update features for Hive 4.0 [iceberg]

SourabhBadhya commented on PR #10162: URL: https://github.com/apache/iceberg/pull/10162#issuecomment-2060924512 @pvary Requesting your review on this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] Add Refs metadata table [iceberg-python]

Fokko merged PR #602: URL: https://github.com/apache/iceberg-python/pull/602 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] Add Refs metadata table [iceberg-python]

Fokko commented on PR #602: URL: https://github.com/apache/iceberg-python/pull/602#issuecomment-2061026458 Thanks @geruh for working on this 🙌 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[PR] chore: update roadmap [iceberg-rust]

marvinlanhenke opened a new pull request, #336: URL: https://github.com/apache/iceberg-rust/pull/336 partly #330. - updated the roadmap to reflect the current status (to the best of my knowledge) @liurenjie1024 Can you PTAL at the `reader` section I'm not sure, I could have

Re: [PR] Add Refs metadata table [iceberg-python]

Fokko commented on code in PR #602: URL: https://github.com/apache/iceberg-python/pull/602#discussion_r1568666707 ## tests/integration/test_inspect_table.py: ## @@ -274,6 +274,64 @@ def test_inspect_entries_partitioned(spark: SparkSession, session_catalog: Catal @pytest.mark

Re: [PR] Backport Flink 1.18 JUnit5 migration to Flink 1.17 [iceberg]

nastra merged PR #10163: URL: https://github.com/apache/iceberg/pull/10163 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] add `InclusiveProjection` Visitor [iceberg-rust]

marvinlanhenke commented on code in PR #335: URL: https://github.com/apache/iceberg-rust/pull/335#discussion_r1568705676 ## crates/iceberg/src/expr/visitors/inclusive_projection.rs: ## @@ -0,0 +1,371 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more c

[I] Cannot insert table created by spark temp into iceberg table [iceberg]

hellonewplayer opened a new issue, #10164: URL: https://github.com/apache/iceberg/issues/10164 ### Apache Iceberg version None ### Query engine Spark ### Please describe the bug 🐞 Helo, I got a similar issue to https://github.com/apache/iceberg/issues/510.

Re: [PR] StructType field `optional` by default [iceberg-python]

MehulBatra commented on code in PR #592: URL: https://github.com/apache/iceberg-python/pull/592#discussion_r1568694815 ## tests/test_types.py: ## @@ -560,13 +560,13 @@ def test_deserialization_struct() -> None: def test_str_struct(simple_struct: StructType) -> None: -as

Re: [PR] StructType field `optional` by default [iceberg-python]

MehulBatra commented on PR #592: URL: https://github.com/apache/iceberg-python/pull/592#issuecomment-2061051645 > Looks good @MehulBatra Left one small suggestion. > > Could you also remove `required=False` from `mkdocs/docs/api.md`. Since this is now default, I think we should remove

Re: [I] Add metadata tables [iceberg-python]

Fokko commented on issue #511: URL: https://github.com/apache/iceberg-python/issues/511#issuecomment-2061028334 @Gowthami03B checking in if you're still interested in contributing the `files` table. -- This is an automated message from the Apache Git Service. To respond to the message, pl

Re: [PR] StructType field `optional` by default [iceberg-python]

MehulBatra commented on code in PR #592: URL: https://github.com/apache/iceberg-python/pull/592#discussion_r1568694815 ## tests/test_types.py: ## @@ -560,13 +560,13 @@ def test_deserialization_struct() -> None: def test_str_struct(simple_struct: StructType) -> None: -as

Re: [PR] StructType field `optional` by default [iceberg-python]

Fokko commented on code in PR #592: URL: https://github.com/apache/iceberg-python/pull/592#discussion_r1568690611 ## tests/test_types.py: ## @@ -560,13 +560,13 @@ def test_deserialization_struct() -> None: def test_str_struct(simple_struct: StructType) -> None: -assert

Re: [PR] Flink: Don't fail to serialize IcebergSourceSplit when there is too many delete files [iceberg]

javrasya commented on PR #9464: URL: https://github.com/apache/iceberg/pull/9464#issuecomment-2061065359 Done @pvary , I guess the triggers are requiring an approval to kick in. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] Incremental Append Scan [iceberg-python]

Fokko commented on code in PR #533: URL: https://github.com/apache/iceberg-python/pull/533#discussion_r1568698784 ## pyiceberg/table/__init__.py: ## @@ -1594,6 +1617,197 @@ def to_ray(self) -> ray.data.dataset.Dataset: return ray.data.from_arrow(self.to_arrow()) +cl

Re: [PR] Flink: Don't fail to serialize IcebergSourceSplit when there is too many delete files [iceberg]

javrasya commented on code in PR #9464: URL: https://github.com/apache/iceberg/pull/9464#discussion_r1568704619 ## flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/source/split/SerializerHelper.java: ## @@ -0,0 +1,200 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] Sanitized special character column name before writing to parquet [iceberg-python]

Fokko commented on code in PR #590: URL: https://github.com/apache/iceberg-python/pull/590#discussion_r1568710826 ## pyiceberg/io/pyarrow.py: ## @@ -1769,27 +1769,33 @@ def data_file_statistics_from_parquet_metadata( def write_file(io: FileIO, table_metadata: TableMetadata,

Re: [I] [BUG] Valid column characters fail on to_arrow() or to_pandas() ArrowInvalid: No match for FieldRef.Name [iceberg-python]

Fokko closed issue #584: [BUG] Valid column characters fail on to_arrow() or to_pandas() ArrowInvalid: No match for FieldRef.Name URL: https://github.com/apache/iceberg-python/issues/584 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to G

Re: [PR] Sanitized special character column name before writing to parquet [iceberg-python]

Fokko merged PR #590: URL: https://github.com/apache/iceberg-python/pull/590 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] Sanitized special character column name before writing to parquet [iceberg-python]

Fokko commented on PR #590: URL: https://github.com/apache/iceberg-python/pull/590#issuecomment-2061073883 Thanks for working on this @kevinjqliu and thanks @HonahX for the review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

Re: [PR] Hive: turn off the stats gathering when iceberg.hive.keep.stats is false [iceberg]

deniskuzZ commented on PR #10148: URL: https://github.com/apache/iceberg/pull/10148#issuecomment-2061152342 @stargrey102, have you checked the same in Hive-4.0? see HIVE-27355, https://github.com/apache/hive/pull/4348 -- This is an automated message from the Apache Git Service. To resp

Re: [PR] add `InclusiveProjection` Visitor [iceberg-rust]

Fokko commented on code in PR #335: URL: https://github.com/apache/iceberg-rust/pull/335#discussion_r1568763855 ## crates/iceberg/src/expr/visitors/inclusive_projection.rs: ## @@ -0,0 +1,371 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributo

Re: [PR] add `InclusiveProjection` Visitor [iceberg-rust]

Fokko commented on code in PR #335: URL: https://github.com/apache/iceberg-rust/pull/335#discussion_r1568751884 ## crates/iceberg/src/expr/visitors/inclusive_projection.rs: ## @@ -0,0 +1,371 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributo

[I] Iceberg may occur data duplication when use flink to write data to iceberg and commit failed [iceberg]

zlzhang0122 opened a new issue, #10165: URL: https://github.com/apache/iceberg/issues/10165 ### Apache Iceberg version 1.3.0 ### Query engine Spark ### Please describe the bug 🐞 Iceberg may occur data duplication when use flink to write data to iceberg and

Re: [I] Add metadata tables [iceberg-python]

Gowthami03B commented on issue #511: URL: https://github.com/apache/iceberg-python/issues/511#issuecomment-2061211477 > @Gowthami03B checking in if you're still interested in contributing the `files` table. @Fokko yes, I will be sending over a PR shortly. -- This is an automated me

Re: [PR] Fix CI [iceberg-python]

Fokko merged PR #613: URL: https://github.com/apache/iceberg-python/pull/613 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] Backport Flink 1.18 JUnit5 migration to Flink 1.17 [iceberg]

tomtongue commented on PR #10163: URL: https://github.com/apache/iceberg/pull/10163#issuecomment-2061336515 Sure, thanks for the headsup. Will submit a PR for Flink 1.9 including backport to other versions. -- This is an automated message from the Apache Git Service. To respond to the mes

Re: [I] flink autoscaler: how set write-parallelism ? [iceberg]

pvary commented on issue #10147: URL: https://github.com/apache/iceberg/issues/10147#issuecomment-2061388525 Could it be, that the table is partitioned and all of the new data is targeting a single partition? If you start the sink with higher writer parallelism, how does the data dis

Re: [PR] JDBC Catalog:Change SQL like escape character from '\\' to '!' [iceberg]

amogh-jahagirdar commented on PR #9407: URL: https://github.com/apache/iceberg/pull/9407#issuecomment-2061567560 I've added this to the 1.5.1 milestone. If we validate that it works on all the backends as expected, then I think we can include in the patch release (we'll also discuss this in

Re: [PR] JDBC Catalog:Change SQL like escape character from '\\' to '!' [iceberg]

jbonofre commented on PR #9407: URL: https://github.com/apache/iceberg/pull/9407#issuecomment-2061586682 I tested with MySQL 8.3.0 (both server and JDBC driver) and I confirm the `!` notation works. I used MySQL 8.3.0 docker image, create the schema and updated the `GET_NAMESPACE_SQL` st

Re: [PR] JDBC Catalog:Change SQL like escape character from '\\' to '!' [iceberg]

jbonofre commented on PR #9407: URL: https://github.com/apache/iceberg/pull/9407#issuecomment-2061615460 I also tested with PostgreSQL 42.7.3 and it works fine. In order to fix the conflict, I propose to create a clean PR. -- This is an automated message from the Apache Git Service.

Re: [PR] JDBC Catalog:Change SQL like escape character from '\\' to '!' [iceberg]

jbonofre commented on code in PR #9407: URL: https://github.com/apache/iceberg/pull/9407#discussion_r1569078743 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcUtil.java: ## @@ -136,7 +136,7 @@ final class JdbcUtil { + TABLE_NAMESPACE + " = ? OR "

[PR] Core: fix namespace SQL statement using ESCAPE character working with MySQL, PostgreSQL, ... [iceberg]

jbonofre opened a new pull request, #10167: URL: https://github.com/apache/iceberg/pull/10167 Co-Author (special thanks to): xuchuanqiu -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

Re: [PR] Core: fix namespace SQL statement using ESCAPE character working with MySQL, PostgreSQL, ... [iceberg]

jbonofre commented on PR #10167: URL: https://github.com/apache/iceberg/pull/10167#issuecomment-2061638296 @nastra @amogh-jahagirdar superseding #9407 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

Re: [PR] JDBC Catalog:Change SQL like escape character from '\\' to '!' [iceberg]

jbonofre commented on PR #9407: URL: https://github.com/apache/iceberg/pull/9407#issuecomment-2061639934 Replaced by #10167 to merge the conflict. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Core: fix namespace SQL statement using ESCAPE character working with MySQL, PostgreSQL, ... [iceberg]

jbonofre commented on PR #10167: URL: https://github.com/apache/iceberg/pull/10167#issuecomment-2061641506 This closes #10056 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

Re: [PR] JDBC Catalog:Change SQL like escape character from '\\' to '!' [iceberg]

amogh-jahagirdar commented on PR #9407: URL: https://github.com/apache/iceberg/pull/9407#issuecomment-2061642341 Thanks @jbonofre ! Discussed offline, we'll ensure that @xuchuanqiu gets co-authorship/attribution. Thanks @xuchuanqiu -- This is an automated message from the Apache Git Serv

Re: [I] [BUG] Valid column characters fail on to_arrow() or to_pandas() ArrowInvalid: No match for FieldRef.Name [iceberg-python]

kevinjqliu commented on issue #584: URL: https://github.com/apache/iceberg-python/issues/584#issuecomment-2061699557 @gwindes The fixes for the read side (#597) and write side (#590) are merged. Thanks for reporting this issue! -- This is an automated message from the Apache Git Service.

Re: [PR] Fix open-api /rename/views success return code [iceberg]

nastra merged PR #10166: URL: https://github.com/apache/iceberg/pull/10166 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [I] Iceberg Rest OpenAPI Spec views/rename should return 204 [iceberg]

nastra closed issue #10135: Iceberg Rest OpenAPI Spec views/rename should return 204 URL: https://github.com/apache/iceberg/issues/10135 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] StructType field `optional` by default [iceberg-python]

Fokko commented on code in PR #592: URL: https://github.com/apache/iceberg-python/pull/592#discussion_r1569166271 ## tests/expressions/test_literals.py: ## @@ -277,8 +277,7 @@ def test_decimal_to_decimal_conversion() -> None: def test_timestamp_to_date() -> None: epoch_lit

Re: [PR] StructType field `optional` by default [iceberg-python]

Fokko commented on code in PR #592: URL: https://github.com/apache/iceberg-python/pull/592#discussion_r1569167115 ## tests/expressions/test_literals.py: ## @@ -277,8 +277,7 @@ def test_decimal_to_decimal_conversion() -> None: def test_timestamp_to_date() -> None: epoch_lit

Re: [PR] StructType field `optional` by default [iceberg-python]

Fokko commented on PR #592: URL: https://github.com/apache/iceberg-python/pull/592#issuecomment-2061773917 @MehulBatra I've removed an unrelated change, I think https://github.com/apache/iceberg-python/pull/612/ addresses the issue that you ran into -- This is an automated message from t

Re: [PR] StructType field `optional` by default [iceberg-python]

MehulBatra commented on PR #592: URL: https://github.com/apache/iceberg-python/pull/592#issuecomment-2061776890 > @MehulBatra I've removed an unrelated change, I think #612 addresses the issue that you ran into sounds good! -- This is an automated message from the Apache Git Servic

Re: [PR] Docs: Update features for Hive 4.0 [iceberg]

pvary commented on code in PR #10162: URL: https://github.com/apache/iceberg/pull/10162#discussion_r1569171967 ## docs/docs/hive.md: ## @@ -34,6 +34,32 @@ Iceberg compatibility with Hive 2.x and Hive 3.1.2/3 supports the following feat !!! warning DML operations work only

[I] Restrict generated locations to URI syntax [iceberg]

dimas-b opened a new issue, #10168: URL: https://github.com/apache/iceberg/issues/10168 ### Feature Request / Improvement It is [possible](https://github.com/projectnessie/nessie/issues/8328) to make Iceberg produce S3 (as an example) locations that violate URI syntax. Granted

Re: [PR] Add Pagination To List Apis [iceberg]

rahil-c commented on code in PR #9782: URL: https://github.com/apache/iceberg/pull/9782#discussion_r1569180280 ## core/src/test/java/org/apache/iceberg/rest/TestRESTCatalog.java: ## @@ -2329,6 +2332,120 @@ public void multipleDiffsAgainstMultipleTablesLastFails() { assertT

Re: [PR] Add Pagination To List Apis [iceberg]

rahil-c commented on code in PR #9782: URL: https://github.com/apache/iceberg/pull/9782#discussion_r1569192411 ## core/src/test/java/org/apache/iceberg/rest/TestRESTCatalog.java: ## @@ -2329,6 +2332,120 @@ public void multipleDiffsAgainstMultipleTablesLastFails() { assertT

Re: [PR] StructType field `optional` by default [iceberg-python]

Fokko merged PR #592: URL: https://github.com/apache/iceberg-python/pull/592 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] StructType field `optional` by default [iceberg-python]

Fokko commented on PR #592: URL: https://github.com/apache/iceberg-python/pull/592#issuecomment-2061828760 Thanks for fixing this @MehulBatra Much appreciated! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

Re: [I] flink:FlinkSink support dynamically changed schema [iceberg]

pvary commented on issue #4190: URL: https://github.com/apache/iceberg/issues/4190#issuecomment-2061836288 I think it is not trivial to implement this feature, as the schema of the RowData objects which are the input of the Sink is finalized when the job graph is created. To change the sche

Re: [PR] Core: fix namespace SQL statement using ESCAPE character working with MySQL, PostgreSQL, ... [iceberg]

amogh-jahagirdar merged PR #10167: URL: https://github.com/apache/iceberg/pull/10167 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@

Re: [I] [feature request] Allow engines to time travel [iceberg-python]

kevinjqliu commented on issue #600: URL: https://github.com/apache/iceberg-python/issues/600#issuecomment-2062244023 +1, I agree with you. Passing the snapshot-id should be an engine-specific implementation detail. I was thinking about the Spark/Trino syntax of `AS OF / `. I interpr

[PR] Core: Fix namespace SQL statement using ESCAPE character that works with MySQL/PostgreSQL (#10167) [iceberg]

jbonofre opened a new pull request, #10169: URL: https://github.com/apache/iceberg/pull/10169 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e

Re: [PR] Core: Fix namespace SQL statement using ESCAPE character that works with MySQL/PostgreSQL (#10167) [iceberg]

jbonofre commented on PR #10169: URL: https://github.com/apache/iceberg/pull/10169#issuecomment-2062278521 @nastra @amogh-jahagirdar hey guys, here's the backport on `1.5.x` branch. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

Re: [PR] Core: Calling rewrite_position_delete_files fails on tables with more than 1k columns [iceberg]

szehon-ho commented on PR #10020: URL: https://github.com/apache/iceberg/pull/10020#issuecomment-2062330828 Hi @bk-mz we discussed this a bit in the last Iceberg community sync. The motivation here is to fix the position_deletes metadata table. Any other fix as you mentioned is either

Re: [I] Flink: Decouple the iceberg integration work from hadoop libraries [iceberg]

Fokko commented on issue #3117: URL: https://github.com/apache/iceberg/issues/3117#issuecomment-2062342437 I have a draft over here: https://github.com/apache/iceberg/pull/7369 but it breaks quite a few APIs. -- This is an automated message from the Apache Git Service. To respond to the m

Re: [PR] Build: Bump Guava to 32.1.1 [iceberg]

fdavis99 commented on PR #7973: URL: https://github.com/apache/iceberg/pull/7973#issuecomment-2062371744 This isn't listed in https://github.com/apache/iceberg/releases (nor is #7971), and release notes for 1.4.1 > Dependencies don't mention bumping Guava. Makes it difficult to track when C

Re: [PR] Core: Fix namespace SQL statement using ESCAPE character that works with MySQL/PostgreSQL (#10167) [iceberg]

amogh-jahagirdar merged PR #10169: URL: https://github.com/apache/iceberg/pull/10169 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@

Re: [I] [feature request] Allow engines to time travel [iceberg-python]

corleyma commented on issue #600: URL: https://github.com/apache/iceberg-python/issues/600#issuecomment-2062571021 Still, an api like `Table.as_of(snapshot_id/timestamp) -> Snapshot` would be useful, even if reading requires then passing the correct arguments to `Table.scan`. In general it

Re: [PR] Hive: turn off the stats gathering when iceberg.hive.keep.stats is false [iceberg]

stargrey102 commented on PR #10148: URL: https://github.com/apache/iceberg/pull/10148#issuecomment-2062688748 @deniskuzZ thank you for the link. HiveOperationsBase uses HiveMetastore client when creating the Iceberg table: https://github.com/apache/iceberg/blob/main/hive-metastore/src/main

Re: [I] Web 404 Not Found, Many page jump links cannot be opened [iceberg]

github-actions[bot] commented on issue #2561: URL: https://github.com/apache/iceberg/issues/2561#issuecomment-2062757954 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] Error when dropping partition fields for the same source field more than once. [iceberg]

github-actions[bot] commented on issue #2569: URL: https://github.com/apache/iceberg/issues/2569#issuecomment-2062757982 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] Flink batch job cannot read the iceberg table that writing few chang-log events in format v2. [iceberg]

github-actions[bot] commented on issue #2571: URL: https://github.com/apache/iceberg/issues/2571#issuecomment-2062757992 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] Support creating flink table without creating an explicit hive/hadoop catalog [iceberg]

github-actions[bot] commented on issue #2572: URL: https://github.com/apache/iceberg/issues/2572#issuecomment-2062758005 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] Hive: insert overwrite table xxx partition (pt='xxx') on iceberg table [iceberg]

github-actions[bot] commented on issue #2576: URL: https://github.com/apache/iceberg/issues/2576#issuecomment-2062758019 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] Hive: cannot find field xxx at Map operator initialization failed when iceberg table join hive table [iceberg]

github-actions[bot] commented on issue #2567: URL: https://github.com/apache/iceberg/issues/2567#issuecomment-2062757966 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] Cannot unset hive.engine.enabled once enabled if Iceberg Jars are not on the HMS Classpath [iceberg]

github-actions[bot] commented on issue #2578: URL: https://github.com/apache/iceberg/issues/2578#issuecomment-2062758032 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] Allow SchemaToType class to be public so we can extends and override `record` method [iceberg]

github-actions[bot] commented on issue #2580: URL: https://github.com/apache/iceberg/issues/2580#issuecomment-2062758047 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] Restrict generated locations to URI syntax [iceberg]

ajantha-bhat commented on issue #10168: URL: https://github.com/apache/iceberg/issues/10168#issuecomment-2062777454 Assigned the milestone "Iceberg V3 spec" milestone as discussed on community sync. -- This is an automated message from the Apache Git Service. To respond to the message, p

Re: [PR] JDBC Catalog:Change SQL like escape character from '\\' to '!' [iceberg]

amogh-jahagirdar commented on PR #9407: URL: https://github.com/apache/iceberg/pull/9407#issuecomment-2062812347 I'll close this since #9407 has been merged, thanks @xuchuanqiu @jbonofre ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [PR] JDBC Catalog:Change SQL like escape character from '\\' to '!' [iceberg]

amogh-jahagirdar closed pull request #9407: JDBC Catalog:Change SQL like escape character from '\\' to '!' URL: https://github.com/apache/iceberg/pull/9407 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [I] [feature request] Allow engines to time travel [iceberg-python]

kevinjqliu commented on issue #600: URL: https://github.com/apache/iceberg-python/issues/600#issuecomment-2062828600 > an api like Table.as_of(snapshot_id/timestamp) -> Snapshot would be useful Yea, it's helpful in situations where we need to manipulate the Table state and get back th

[PR] Spark 3.5: Fix system function pushdown in CoW row-level commands [iceberg]

amogh-jahagirdar opened a new pull request, #10170: URL: https://github.com/apache/iceberg/pull/10170 Cherry picks #9873 to the 1.5.x branch -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

[PR] files metadata table [iceberg-python]

Gowthami03B opened a new pull request, #614: URL: https://github.com/apache/iceberg-python/pull/614 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscr

Re: [I] Add metadata tables [iceberg-python]

Gowthami03B commented on issue #511: URL: https://github.com/apache/iceberg-python/issues/511#issuecomment-2062975370 [https://github.com/apache/iceberg-python/pull/614](url) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [I] Add metadata tables [iceberg-python]

Gowthami03B commented on issue #511: URL: https://github.com/apache/iceberg-python/issues/511#issuecomment-2062975602 @https://github.com/apache/iceberg-python/pull/614 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

Re: [PR] Sql catalog [iceberg-rust]

himadripal commented on PR #229: URL: https://github.com/apache/iceberg-rust/pull/229#issuecomment-2062991628 @JanKaul WDYT? I think this PR is ready for review, I can add the update and delete in a separate PR. -- This is an automated message from the Apache Git Service. To respond to t

[I] TestDataFrameWrites#testFaultToleranceOnWrite failed due to exception in cleaning up temporary directory [iceberg]

manuzhang opened a new issue, #10172: URL: https://github.com/apache/iceberg/issues/10172 ### Apache Iceberg version main (development) ### Query engine Spark ### Please describe the bug 🐞 ``` TestDataFrameWrites > testFaultToleranceOnWrite() > format = p

Re: [I] Timestamp/Day transform returns Date as required type while days is actually stored integer [iceberg]

manuzhang commented on issue #10159: URL: https://github.com/apache/iceberg/issues/10159#issuecomment-2063024043 @zinking You may check [this comment](https://github.com/apache/iceberg/issues/279#issuecomment-519620975) for the background. -- This is an automated message from the Apache

Re: [PR] chore: update roadmap [iceberg-rust]

sdd commented on code in PR #336: URL: https://github.com/apache/iceberg-rust/pull/336#discussion_r1570012813 ## README.md: ## @@ -50,19 +50,19 @@ expand to other service. Reader | Feature| Status | -|

[PR] Exclude `docutils!=0.21` as a dependency [iceberg-python]

Fokko opened a new pull request, #615: URL: https://github.com/apache/iceberg-python/pull/615 This release is bodged, and causes Poetry to fail when it tries to fetch the tar: https://github.com/python-poetry/poetry/issues/9293#issuecomment-2048205226 It is being tracked: ht

Re: [PR] Exclude `docutils!=0.21` as a dependency [iceberg-python]

Fokko merged PR #615: URL: https://github.com/apache/iceberg-python/pull/615 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] Exclude `docutils!=0.21` as a dependency [iceberg-python]

Fokko commented on PR #615: URL: https://github.com/apache/iceberg-python/pull/615#issuecomment-2063071933 @HonahX You're quick 🚀 I just posted a message on the devlist with more context. > Shall we also backport it to 0.6.1 (and a new rc)? Sounds good to me 👍 -- This is a

Re: [PR] (1.5.x Cherry-pick) Spark 3.4: Fix system function pushdown in CoW row-level commands (#10119) [iceberg]

nastra merged PR #10171: URL: https://github.com/apache/iceberg/pull/10171 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

[PR] Backport: Exclude `docutils!=0.21` as a dependency (#615) [iceberg-python]

Fokko opened a new pull request, #616: URL: https://github.com/apache/iceberg-python/pull/616 This release is bodged, and causes Poetry to fail when it tries to fetch the tar: https://github.com/python-poetry/poetry/issues/9293#issuecomment-2048205226 It is being tracked: ht

Re: [PR] Docs: Update features for Hive 4.0 [iceberg]

SourabhBadhya commented on code in PR #10162: URL: https://github.com/apache/iceberg/pull/10162#discussion_r1570052432 ## docs/docs/hive.md: ## @@ -34,6 +34,32 @@ Iceberg compatibility with Hive 2.x and Hive 3.1.2/3 supports the following feat !!! warning DML operations w

Re: [PR] (1.5.x cherry-pick) Spark 3.5: Fix system function pushdown in CoW row-level commands [iceberg]

nastra merged PR #10170: URL: https://github.com/apache/iceberg/pull/10170 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

  1   2   >