Re: [PR] Migrate WAP, Metrics and FilterFiles in Core to JUnit5 [iceberg]

tomtongue commented on code in PR #10039: URL: https://github.com/apache/iceberg/pull/10039#discussion_r1542341876 ## core/src/test/java/org/apache/iceberg/TestMetrics.java: ## @@ -766,24 +771,25 @@ protected void assertCounts(int fieldId, Long valueCount, Long nullValueCount,

Re: [PR] Migrate WAP, Metrics and FilterFiles in Core to JUnit5 [iceberg]

tomtongue commented on code in PR #10039: URL: https://github.com/apache/iceberg/pull/10039#discussion_r1542341876 ## core/src/test/java/org/apache/iceberg/TestMetrics.java: ## @@ -766,24 +771,25 @@ protected void assertCounts(int fieldId, Long valueCount, Long nullValueCount,

Re: [PR] Migrate WAP, Metrics and FilterFiles in Core to JUnit5 [iceberg]

tomtongue commented on code in PR #10039: URL: https://github.com/apache/iceberg/pull/10039#discussion_r1542341876 ## core/src/test/java/org/apache/iceberg/TestMetrics.java: ## @@ -766,24 +771,25 @@ protected void assertCounts(int fieldId, Long valueCount, Long nullValueCount,

Re: [PR] Hive: Add View support for HIVE catalog [iceberg]

nk1506 commented on code in PR #9852: URL: https://github.com/apache/iceberg/pull/9852#discussion_r1542321268 ## hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java: ## @@ -250,29 +368,53 @@ public void renameTable(TableIdentifier from, TableIdentifier origina

Re: [PR] CI: Run Markdown links checker only when `{docs,site}/**` changes [iceberg]

manuzhang commented on PR #10049: URL: https://github.com/apache/iceberg/pull/10049#issuecomment-2024378877 I think we need to enable with `check-modified-files-only: 'yes'` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] Hive: Use EnvironmentContext instead of Hive Locks to provide transactional commits after HIVE-26882 [iceberg]

chenwyi2 commented on PR #6570: URL: https://github.com/apache/iceberg/pull/6570#issuecomment-2024332961 https://issues.apache.org/jira/browse/HIVE-26882 has something wrong, should we has some documentation to info user? -- This is an automated message from the Apache Git Service. To res

Re: [PR] Support identifier warehouses [iceberg-rust]

liurenjie1024 commented on code in PR #308: URL: https://github.com/apache/iceberg-rust/pull/308#discussion_r1542207188 ## crates/catalog/rest/src/catalog.rs: ## @@ -617,7 +617,13 @@ impl RestCatalog { props.extend(config); } -let file_io = match

Re: [I] Implement `project` for `Transform`. [iceberg-rust]

liurenjie1024 commented on issue #264: URL: https://github.com/apache/iceberg-rust/issues/264#issuecomment-2024236422 > @liurenjie1024 @ZENOTME ...the `fn project(...)` is getting kinda extensive (lengthy). I think it'd be appropriate to introduce some helper functions here instead of writi

[PR] Introduce two properties for reading the connection timeout and socke… [iceberg]

harishch1998 opened a new pull request, #10053: URL: https://github.com/apache/iceberg/pull/10053 This PR includes code changes to make the REST HTTPClient's socket and connection timeout values configurable. Introducing two new properties to help achieve this. -- This is an automated m

[PR] Extend HTTPClient Builder to allow setting a proxy server [iceberg]

harishch1998 opened a new pull request, #10052: URL: https://github.com/apache/iceberg/pull/10052 This PR includes code changes to extend the capability of the existing rest HTTPClient to allow setting a proxy server with basic username/password authentication. The requests made by the HTTP

Re: [PR] Spark 3.5: Parallelize reading files in snapshot and migrate procedures [iceberg]

manuzhang commented on code in PR #10037: URL: https://github.com/apache/iceberg/pull/10037#discussion_r1542198454 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestMigrateTableProcedure.java: ## @@ -232,4 +232,25 @@ public void testMigrateEmpt

Re: [PR] feat: Read Parquet data file with projection [iceberg-rust]

viirya commented on PR #245: URL: https://github.com/apache/iceberg-rust/pull/245#issuecomment-2024206522 Thank you @liurenjie1024 for review. I will address these comments soon. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [I] Partition data to path, and path to data conversions are not consistent with each other [iceberg]

github-actions[bot] commented on issue #2043: URL: https://github.com/apache/iceberg/issues/2043#issuecomment-2024182329 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Remove calls to the deprecated Metrics constructors [iceberg]

github-actions[bot] commented on issue #2044: URL: https://github.com/apache/iceberg/issues/2044#issuecomment-2024182352 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Remove calls to the deprecated Metrics constructors [iceberg]

github-actions[bot] closed issue #2044: Remove calls to the deprecated Metrics constructors URL: https://github.com/apache/iceberg/issues/2044 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

Re: [I] Partition data to path, and path to data conversions are not consistent with each other [iceberg]

github-actions[bot] closed issue #2043: Partition data to path, and path to data conversions are not consistent with each other URL: https://github.com/apache/iceberg/issues/2043 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

Re: [I] Out of order fields are corrupting data with Insert Into SQL on Spark 3.0 [iceberg]

github-actions[bot] commented on issue #2041: URL: https://github.com/apache/iceberg/issues/2041#issuecomment-2024182316 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Partial data ingestion to Iceberg in failing with Spark 3.0.x [iceberg]

github-actions[bot] commented on issue #2040: URL: https://github.com/apache/iceberg/issues/2040#issuecomment-2024182303 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Partial data ingestion to Iceberg in failing with Spark 3.0.x [iceberg]

github-actions[bot] closed issue #2040: Partial data ingestion to Iceberg in failing with Spark 3.0.x URL: https://github.com/apache/iceberg/issues/2040 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [I] Support default value semantics in Iceberg [iceberg]

github-actions[bot] closed issue #2039: Support default value semantics in Iceberg URL: https://github.com/apache/iceberg/issues/2039 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

Re: [I] Support default value semantics in Iceberg [iceberg]

github-actions[bot] commented on issue #2039: URL: https://github.com/apache/iceberg/issues/2039#issuecomment-2024182287 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Iceberg specific files (manifest files, manifest list file, metadata file) are not created, only Flink specific manifest file is created [iceberg]

github-actions[bot] closed issue #2033: Iceberg specific files (manifest files, manifest list file, metadata file) are not created, only Flink specific manifest file is created URL: https://github.com/apache/iceberg/issues/2033 -- This is an automated message from the Apache Git Service. To

Re: [I] Iceberg specific files (manifest files, manifest list file, metadata file) are not created, only Flink specific manifest file is created [iceberg]

github-actions[bot] commented on issue #2033: URL: https://github.com/apache/iceberg/issues/2033#issuecomment-2024182268 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] The data generated by Actions cannot be serialized [iceberg]

github-actions[bot] closed issue #2028: The data generated by Actions cannot be serialized URL: https://github.com/apache/iceberg/issues/2028 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

Re: [I] The data generated by Actions cannot be serialized [iceberg]

github-actions[bot] commented on issue #2028: URL: https://github.com/apache/iceberg/issues/2028#issuecomment-2024182255 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Flink : table column comment missing [iceberg]

github-actions[bot] closed issue #1995: Flink : table column comment missing URL: https://github.com/apache/iceberg/issues/1995 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [I] Flink : The Iceberg metadata is not updated after the SavePoint recovery task [iceberg]

github-actions[bot] closed issue #2023: Flink : The Iceberg metadata is not updated after the SavePoint recovery task URL: https://github.com/apache/iceberg/issues/2023 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [I] Flink : table column comment missing [iceberg]

github-actions[bot] commented on issue #1995: URL: https://github.com/apache/iceberg/issues/1995#issuecomment-2024182223 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] no 0.10.0 for iceberg-hive [iceberg]

github-actions[bot] commented on issue #1992: URL: https://github.com/apache/iceberg/issues/1992#issuecomment-2024182208 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] no 0.10.0 for iceberg-hive [iceberg]

github-actions[bot] closed issue #1992: no 0.10.0 for iceberg-hive URL: https://github.com/apache/iceberg/issues/1992 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubsc

Re: [I] Can we compact data files by some partition? [iceberg]

github-actions[bot] commented on issue #1973: URL: https://github.com/apache/iceberg/issues/1973#issuecomment-2024182175 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Flink : query large iceberg tables slowly [iceberg]

github-actions[bot] closed issue #1983: Flink : query large iceberg tables slowly URL: https://github.com/apache/iceberg/issues/1983 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

Re: [I] Can we compact data files by some partition? [iceberg]

github-actions[bot] closed issue #1973: Can we compact data files by some partition? URL: https://github.com/apache/iceberg/issues/1973 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [I] Read options to read append and delete with overwrite snapshots in Incremental reads [iceberg]

github-actions[bot] commented on issue #1949: URL: https://github.com/apache/iceberg/issues/1949#issuecomment-2024182135 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Flink : query large iceberg tables slowly [iceberg]

github-actions[bot] commented on issue #1983: URL: https://github.com/apache/iceberg/issues/1983#issuecomment-2024182191 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Flink: Commit the existing manifests to iceberg txn in IcebergFilesCommitter. [iceberg]

github-actions[bot] commented on issue #1959: URL: https://github.com/apache/iceberg/issues/1959#issuecomment-2024182156 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Flink: Commit the existing manifests to iceberg txn in IcebergFilesCommitter. [iceberg]

github-actions[bot] closed issue #1959: Flink: Commit the existing manifests to iceberg txn in IcebergFilesCommitter. URL: https://github.com/apache/iceberg/issues/1959 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [I] Read options to read append and delete with overwrite snapshots in Incremental reads [iceberg]

github-actions[bot] closed issue #1949: Read options to read append and delete with overwrite snapshots in Incremental reads URL: https://github.com/apache/iceberg/issues/1949 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

Re: [I] Add an action to compact delete files [iceberg]

github-actions[bot] commented on issue #1027: URL: https://github.com/apache/iceberg/issues/1027#issuecomment-2024182118 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [PR] feat: Support customized header in Rest catalog client [iceberg-rust]

flyrain commented on PR #306: URL: https://github.com/apache/iceberg-rust/pull/306#issuecomment-2024066123 Thanks for @whynick1 for working on this. Thanks @liurenjie1024 @Fokko for the reviews! -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [I] Support customized header in Rest catalog client [iceberg-rust]

flyrain closed issue #292: Support customized header in Rest catalog client URL: https://github.com/apache/iceberg-rust/issues/292 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

Re: [PR] feat: Support customized header in Rest catalog client [iceberg-rust]

flyrain merged PR #306: URL: https://github.com/apache/iceberg-rust/pull/306 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

[I] Document `advisory-partition-size` configuration for Spark's Adaptive Query Execution [iceberg]

johnnywalker opened a new issue, #10051: URL: https://github.com/apache/iceberg/issues/10051 ### Feature Request / Improvement While working with Iceberg on a local Spark cluster, I repeatedly encountered heap size errors when using CTAS/RTAS with `PARTITIONED BY`. These errors baffl

Re: [PR] Flink: implement range partitioner for map data statistics [iceberg]

stevenzwu merged PR #9321: URL: https://github.com/apache/iceberg/pull/9321 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.a

Re: [PR] Flink: implement range partitioner for map data statistics [iceberg]

stevenzwu commented on PR #9321: URL: https://github.com/apache/iceberg/pull/9321#issuecomment-2023956526 thanks @pvary and @yegangy0718 for the code review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] [WIP] Implement project for Transform. #264 [iceberg-rust]

marvinlanhenke closed pull request #269: [WIP] Implement project for Transform. #264 URL: https://github.com/apache/iceberg-rust/pull/269 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] feat: Project transform [iceberg-rust]

marvinlanhenke commented on PR #309: URL: https://github.com/apache/iceberg-rust/pull/309#issuecomment-2023840212 @liurenjie1024 @ZENOTME @sdd PTAL -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[PR] feat: Project transform [iceberg-rust]

marvinlanhenke opened a new pull request, #309: URL: https://github.com/apache/iceberg-rust/pull/309 ### Which issue does this PR close? Closes #264 ### Rationale for this change The ability to project row_filter to `Transform` partition. This will unblock the Manifest & Par

Re: [PR] Hive: Add View support for HIVE catalog [iceberg]

nk1506 commented on code in PR #9852: URL: https://github.com/apache/iceberg/pull/9852#discussion_r1541896158 ## hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java: ## @@ -250,29 +368,53 @@ public void renameTable(TableIdentifier from, TableIdentifier origina

Re: [PR] Build: Bump org.glassfish.jaxb:jaxb-runtime from 2.3.3 to 2.3.9 [iceberg]

Fokko commented on PR #9988: URL: https://github.com/apache/iceberg/pull/9988#issuecomment-2023772579 @dependabot rebase Seems like an intermittent issue -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

Re: [PR] Build: Bump com.esotericsoftware:kryo from 4.0.2 to 4.0.3 [iceberg]

Fokko merged PR #9984: URL: https://github.com/apache/iceberg/pull/9984 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apach

Re: [PR] Hive: Add View support for HIVE catalog [iceberg]

nk1506 commented on code in PR #9852: URL: https://github.com/apache/iceberg/pull/9852#discussion_r1541864321 ## hive-metastore/src/main/java/org/apache/iceberg/hive/HiveViewOperations.java: ## @@ -0,0 +1,351 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

Re: [PR] Hive: Add View support for HIVE catalog [iceberg]

nk1506 commented on code in PR #9852: URL: https://github.com/apache/iceberg/pull/9852#discussion_r1541864321 ## hive-metastore/src/main/java/org/apache/iceberg/hive/HiveViewOperations.java: ## @@ -0,0 +1,351 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

Re: [PR] Hive: Add View support for HIVE catalog [iceberg]

nk1506 commented on code in PR #9852: URL: https://github.com/apache/iceberg/pull/9852#discussion_r1541854460 ## hive-metastore/src/main/java/org/apache/iceberg/hive/HiveViewOperations.java: ## @@ -0,0 +1,351 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

Re: [PR] Hive: Add View support for HIVE catalog [iceberg]

szehon-ho commented on code in PR #9852: URL: https://github.com/apache/iceberg/pull/9852#discussion_r1541619872 ## hive-metastore/src/main/java/org/apache/iceberg/hive/HiveViewOperations.java: ## @@ -0,0 +1,351 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under o

Re: [PR] Hive: Add View support for HIVE catalog [iceberg]

szehon-ho commented on code in PR #9852: URL: https://github.com/apache/iceberg/pull/9852#discussion_r1541619023 ## hive-metastore/src/main/java/org/apache/iceberg/hive/HiveViewOperations.java: ## @@ -0,0 +1,351 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under o

Re: [PR] Hive: Add View support for HIVE catalog [iceberg]

szehon-ho commented on code in PR #9852: URL: https://github.com/apache/iceberg/pull/9852#discussion_r1541618026 ## hive-metastore/src/main/java/org/apache/iceberg/hive/HiveViewOperations.java: ## @@ -0,0 +1,351 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under o

Re: [PR] Hive: Add View support for HIVE catalog [iceberg]

szehon-ho commented on code in PR #9852: URL: https://github.com/apache/iceberg/pull/9852#discussion_r1541613871 ## hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java: ## @@ -250,29 +368,53 @@ public void renameTable(TableIdentifier from, TableIdentifier orig

Re: [PR] OpenAPI: Express server capabilities via /config endpoint [iceberg]

geruh commented on PR #9940: URL: https://github.com/apache/iceberg/pull/9940#issuecomment-2023422080 If I'm correct this approach provides a way to group catalog functionalities, and allow the server to explicitly define the support or not. This is kind of similar to what both the append a

Re: [PR] feat: Support customized header in Rest catalog client [iceberg-rust]

flyrain commented on code in PR #306: URL: https://github.com/apache/iceberg-rust/pull/306#discussion_r1541552671 ## crates/catalog/rest/src/catalog.rs: ## @@ -963,6 +993,76 @@ mod tests { ); } +#[tokio::test] +async fn test_get_default_headers() { +

Re: [PR] feat: Support customized header in Rest catalog client [iceberg-rust]

flyrain commented on code in PR #306: URL: https://github.com/apache/iceberg-rust/pull/306#discussion_r1541552671 ## crates/catalog/rest/src/catalog.rs: ## @@ -963,6 +993,76 @@ mod tests { ); } +#[tokio::test] +async fn test_get_default_headers() { +

Re: [PR] feat: Support customized header in Rest catalog client [iceberg-rust]

whynick1 commented on code in PR #306: URL: https://github.com/apache/iceberg-rust/pull/306#discussion_r1541504517 ## crates/catalog/rest/src/catalog.rs: ## @@ -103,8 +104,7 @@ impl RestCatalogConfig { ]) } -fn try_create_rest_client(&self) -> Result { -

Re: [PR] feat: Support customized header in Rest catalog client [iceberg-rust]

whynick1 commented on code in PR #306: URL: https://github.com/apache/iceberg-rust/pull/306#discussion_r1541494804 ## crates/catalog/rest/src/catalog.rs: ## @@ -963,6 +993,76 @@ mod tests { ); } +#[tokio::test] +async fn test_get_default_headers() { +

Re: [PR] Build: Bump software.amazon.awssdk:bom from 2.24.5 to 2.25.16 [iceberg]

dependabot[bot] commented on PR #10035: URL: https://github.com/apache/iceberg/pull/10035#issuecomment-2023257954 Superseded by #10050. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

Re: [PR] Build: Bump software.amazon.awssdk:bom from 2.24.5 to 2.25.16 [iceberg]

dependabot[bot] closed pull request #10035: Build: Bump software.amazon.awssdk:bom from 2.24.5 to 2.25.16 URL: https://github.com/apache/iceberg/pull/10035 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[PR] Build: Bump software.amazon.awssdk:bom from 2.24.5 to 2.25.18 [iceberg]

dependabot[bot] opened a new pull request, #10050: URL: https://github.com/apache/iceberg/pull/10050 Bumps software.amazon.awssdk:bom from 2.24.5 to 2.25.18. [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=softw

Re: [PR] Build: Bump com.azure:azure-sdk-bom from 1.2.20 to 1.2.21 [iceberg]

Fokko merged PR #9857: URL: https://github.com/apache/iceberg/pull/9857 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apach

Re: [PR] Build: Bump orc from 1.9.2 to 1.9.3 [iceberg]

dongjoon-hyun commented on PR #10033: URL: https://github.com/apache/iceberg/pull/10033#issuecomment-2023241276 Thank YOU, @Fokko . -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

Re: [PR] Build: Bump orc from 1.9.2 to 1.9.3 [iceberg]

Fokko commented on PR #10033: URL: https://github.com/apache/iceberg/pull/10033#issuecomment-2023238634 Awesome, thanks for the quick reply @dongjoon-hyun -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] Build: Bump orc from 1.9.2 to 1.9.3 [iceberg]

Fokko merged PR #10033: URL: https://github.com/apache/iceberg/pull/10033 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] Build: Bump kafka from 3.6.1 to 3.7.0 [iceberg]

bryanck merged PR #9855: URL: https://github.com/apache/iceberg/pull/9855 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] Build: Bump kafka from 3.6.1 to 3.7.0 [iceberg]

bryanck commented on PR #9855: URL: https://github.com/apache/iceberg/pull/9855#issuecomment-2023227924 Thanks for keep this up to date @Fokko -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[PR] CI: Run Markdown links checker only when `{docs,site}/**` changes [iceberg]

Fokko opened a new pull request, #10049: URL: https://github.com/apache/iceberg/pull/10049 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-ma

Re: [PR] Build: Bump arrow from 15.0.1 to 15.0.2 [iceberg]

Fokko merged PR #10034: URL: https://github.com/apache/iceberg/pull/10034 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] Build: Bump io.netty:netty-buffer from 4.1.107.Final to 4.1.108.Final [iceberg]

Fokko merged PR #10032: URL: https://github.com/apache/iceberg/pull/10032 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] Build: Bump org.xerial:sqlite-jdbc from 3.45.1.0 to 3.45.2.0 [iceberg]

Fokko commented on PR #9974: URL: https://github.com/apache/iceberg/pull/9974#issuecomment-2023179076 Thanks for approval @jbonofre -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [PR] Build: Bump org.xerial:sqlite-jdbc from 3.45.1.0 to 3.45.2.0 [iceberg]

Fokko merged PR #9974: URL: https://github.com/apache/iceberg/pull/9974 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apach

Re: [PR] Spark 3.4: Fail on recursive cycle in view [iceberg]

nastra merged PR #10048: URL: https://github.com/apache/iceberg/pull/10048 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Build: Bump arrow from 15.0.1 to 15.0.2 [iceberg]

jbonofre commented on PR #10034: URL: https://github.com/apache/iceberg/pull/10034#issuecomment-2023040218 Tested without problem. NB: arrow 16.0.0 will be available soon. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] Migrate WAP, Metrics and FilterFiles in Core to JUnit5 [iceberg]

nastra commented on code in PR #10039: URL: https://github.com/apache/iceberg/pull/10039#discussion_r1541315898 ## core/src/test/java/org/apache/iceberg/TestMetrics.java: ## @@ -766,24 +771,25 @@ protected void assertCounts(int fieldId, Long valueCount, Long nullValueCount, M

Re: [PR] Build: Bump io.netty:netty-buffer from 4.1.107.Final to 4.1.108.Final [iceberg]

jbonofre commented on PR #10032: URL: https://github.com/apache/iceberg/pull/10032#issuecomment-2023021767 Tested locally without problem. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

Re: [PR] Spark 3.5: Parallelize reading files in snapshot and migrate procedures [iceberg]

nastra commented on code in PR #10037: URL: https://github.com/apache/iceberg/pull/10037#discussion_r1541275556 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestMigrateTableProcedure.java: ## @@ -232,4 +232,25 @@ public void testMigrateEmptyTa

[I] Support Partial deletes [iceberg-python]

Fokko opened a new issue, #554: URL: https://github.com/apache/iceberg-python/issues/554 ### Feature Request / Improvement Today we only support full deletes (overwrite). Supporting partial deletes can be achieved at several places in Iceberg: - Pure delete operations: - D

Re: [PR] feat: Implement the conversion from Iceberg Schema to Arrow Schema [iceberg-rust]

Fokko commented on code in PR #277: URL: https://github.com/apache/iceberg-rust/pull/277#discussion_r1541095666 ## crates/iceberg/src/arrow.rs: ## @@ -106,3 +114,224 @@ impl ArrowReader { ProjectionMask::all() } } + +/// The key of column id in the metadata of arr

Re: [PR] feat: Support customized header in Rest catalog client [iceberg-rust]

Fokko commented on code in PR #306: URL: https://github.com/apache/iceberg-rust/pull/306#discussion_r1541090474 ## crates/catalog/rest/src/catalog.rs: ## @@ -963,6 +993,76 @@ mod tests { ); } +#[tokio::test] +async fn test_get_default_headers() { +

Re: [I] Implement `project` for `Transform`. [iceberg-rust]

marvinlanhenke commented on issue #264: URL: https://github.com/apache/iceberg-rust/issues/264#issuecomment-2022741526 @liurenjie1024 @ZENOTME ...the `fn project(...)` is getting kinda extensive (lengthy). I think it'd be appropriate to introduce some helper functions here instead of wri

Re: [PR] Spark 3.5: Parallelize reading files in snapshot and migrate procedures [iceberg]

manuzhang commented on code in PR #10037: URL: https://github.com/apache/iceberg/pull/10037#discussion_r1541039378 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestMigrateTableProcedure.java: ## @@ -232,4 +232,25 @@ public void testMigrateEmpt

Re: [PR] Spark 3.5: Parallelize reading files in snapshot and migrate procedures [iceberg]

nastra commented on code in PR #10037: URL: https://github.com/apache/iceberg/pull/10037#discussion_r1540956163 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestMigrateTableProcedure.java: ## @@ -232,4 +232,25 @@ public void testMigrateEmptyTa

Re: [I] Support Page Skipping in Iceberg Parquet Reader [iceberg]

zhongyujiang commented on issue #193: URL: https://github.com/apache/iceberg/issues/193#issuecomment-2022556785 Hi guys, I have implemented a ID-based `ParquetColumnIndexFilter` on the Iceberg side, equivalent to the ColumnIndexFilter in Parquet-MR. I have also updated the `ParquetReader` t

[PR] Fix the Avro tests [iceberg-python]

Fokko opened a new pull request, #552: URL: https://github.com/apache/iceberg-python/pull/552 When writing V1, the sequence-numbers should be None. For V2 they will be written and read into the original value. -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] Build: disable link-check for existing medium blog posts [iceberg]

manuzhang commented on PR #10042: URL: https://github.com/apache/iceberg/pull/10042#issuecomment-2022204285 Sure, do we want to run link check on main branch or just PRs for changed docs? -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [PR] Spark 3.5: Parallelize reading files in snapshot and migrate procedures [iceberg]

manuzhang commented on code in PR #10037: URL: https://github.com/apache/iceberg/pull/10037#discussion_r1540660926 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestMigrateTableProcedure.java: ## @@ -232,4 +232,25 @@ public void testMigrateEmpt

Re: [PR] Build: disable link-check for existing medium blog posts [iceberg]

nastra commented on PR #10042: URL: https://github.com/apache/iceberg/pull/10042#issuecomment-2022179463 @manuzhang it looks like there are a few more medium links that haven't been covered. Can you please follow-up on those as well? -- This is an automated message from the Apache Git Ser

Re: [PR] Spark 3.5: Parallelize reading files in snapshot and migrate procedures [iceberg]

nastra commented on code in PR #10037: URL: https://github.com/apache/iceberg/pull/10037#discussion_r1540633541 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestMigrateTableProcedure.java: ## @@ -232,4 +232,25 @@ public void testMigrateEmptyTa

Re: [PR] Spark 3.5: Parallelize reading files in snapshot and migrate procedures [iceberg]

nastra commented on code in PR #10037: URL: https://github.com/apache/iceberg/pull/10037#discussion_r1540633541 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestMigrateTableProcedure.java: ## @@ -232,4 +232,25 @@ public void testMigrateEmptyTa

Re: [I] [JDBC Catalog] Table commit fails if iceberg_type field is NULL [iceberg]

jbonofre commented on issue #10046: URL: https://github.com/apache/iceberg/issues/10046#issuecomment-2022123136 I'm investigating and working on this one. Thanks ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

[I] [JDBC Catalog] Table commit fails if iceberg_type is NULL [iceberg]

vakarisbk opened a new issue, #10046: URL: https://github.com/apache/iceberg/issues/10046 ### Apache Iceberg version 1.5.0 (latest release) ### Query engine Spark ### Please describe the bug 🐞 Table commit fails with an error `org.apache.iceberg.exceptions.

Re: [PR] Build: disable link-check for existing medium blog posts [iceberg]

nastra merged PR #10042: URL: https://github.com/apache/iceberg/pull/10042 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Add entries metadata table [iceberg-python]

Fokko commented on PR #551: URL: https://github.com/apache/iceberg-python/pull/551#issuecomment-2022116483 This will also make it easier to check the sequence numbers for https://github.com/apache/iceberg-python/pull/363 -- This is an automated message from the Apache Git Service. To resp

Re: [PR] Add entries metadata table [iceberg-python]

Fokko commented on code in PR #551: URL: https://github.com/apache/iceberg-python/pull/551#discussion_r1540591861 ## tests/integration/test_writes.py: ## @@ -672,69 +643,3 @@ def test_table_properties_raise_for_none_value( session_catalog, identifier, {"format-versi

  1   2   >