Re: [PR] Build: Bump io.delta:delta-standalone_2.12 from 3.1.0 to 3.2.0 [iceberg]

2024-05-23 Thread via GitHub
Fokko commented on PR #10321: URL: https://github.com/apache/iceberg/pull/10321#issuecomment-2126383960 @dependabot rebase -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

[I] Run RevAPI without Gradle [iceberg]

2024-05-23 Thread via GitHub
Fokko opened a new issue, #10368: URL: https://github.com/apache/iceberg/issues/10368 ### Feature Request / Improvement By decoupling RevAPI from Gradle we can upgrade to a later version of Gradle. A later version of Gradle is required to unblock #10209. The 1.14.0 version of P

Re: [I] Upgrade to gradle 8.7 [iceberg]

2024-05-23 Thread via GitHub
Fokko commented on issue #8485: URL: https://github.com/apache/iceberg/issues/8485#issuecomment-2126395440 @jbonofre Gentle ping on this one! A later version of Gradle is required to unblock #10209. The 1.14.0 version of Parquet ships with a newer version of Jackson, with JDK21-specif

Re: [PR] Build: Bump mkdocs-material from 9.5.21 to 9.5.23 [iceberg]

2024-05-23 Thread via GitHub
Fokko merged PR #10353: URL: https://github.com/apache/iceberg/pull/10353 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] Build: Bump org.springframework:spring-web from 5.3.34 to 5.3.35 [iceberg]

2024-05-23 Thread via GitHub
Fokko merged PR #10354: URL: https://github.com/apache/iceberg/pull/10354 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] Build: Bump software.amazon.awssdk:bom from 2.25.50 to 2.25.57 [iceberg]

2024-05-23 Thread via GitHub
Fokko merged PR #10367: URL: https://github.com/apache/iceberg/pull/10367 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] Bump requests from 2.32.1 to 2.32.2 [iceberg-python]

2024-05-23 Thread via GitHub
Fokko merged PR #759: URL: https://github.com/apache/iceberg-python/pull/759 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [I] Upgrade to gradle 8.7 [iceberg]

2024-05-23 Thread via GitHub
jbonofre commented on issue #8485: URL: https://github.com/apache/iceberg/issues/8485#issuecomment-2126402772 Back on it. Sorry for the delay. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] Bump griffe from 0.45.0 to 0.45.1 [iceberg-python]

2024-05-23 Thread via GitHub
Fokko merged PR #760: URL: https://github.com/apache/iceberg-python/pull/760 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] Bump mypy-boto3-glue from 1.34.88 to 1.34.110 [iceberg-python]

2024-05-23 Thread via GitHub
Fokko merged PR #761: URL: https://github.com/apache/iceberg-python/pull/761 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [I] Upgrade to gradle 8.7 [iceberg]

2024-05-23 Thread via GitHub
Fokko commented on issue #8485: URL: https://github.com/apache/iceberg/issues/8485#issuecomment-2126405479 @jbonofre No problem, we're all busy. I've split out the RevAPI change in https://github.com/apache/iceberg/issues/10368 -- This is an automated message from the Apache Git Service.

Re: [I] Upgrade to gradle 8.7 [iceberg]

2024-05-23 Thread via GitHub
jbonofre commented on issue #8485: URL: https://github.com/apache/iceberg/issues/8485#issuecomment-2126407254 Yeah. I saw that. Thanks. I'm working on both right now. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

Re: [PR] Add `InclusiveMetricsEvaluator` [iceberg-rust]

2024-05-23 Thread via GitHub
Fokko commented on code in PR #347: URL: https://github.com/apache/iceberg-rust/pull/347#discussion_r1611140628 ## crates/iceberg/src/expr/visitors/inclusive_metrics_evaluator.rs: ## @@ -0,0 +1,2255 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more co

Re: [PR] Add Files metadata table [iceberg-python]

2024-05-23 Thread via GitHub
geruh commented on code in PR #614: URL: https://github.com/apache/iceberg-python/pull/614#discussion_r1611151435 ## pyiceberg/table/__init__.py: ## @@ -3537,6 +3537,106 @@ def update_partitions_map( schema=table_schema, ) +def files(self, snapshot_id

Re: [PR] Add Files metadata table [iceberg-python]

2024-05-23 Thread via GitHub
geruh commented on code in PR #614: URL: https://github.com/apache/iceberg-python/pull/614#discussion_r1611151435 ## pyiceberg/table/__init__.py: ## @@ -3537,6 +3537,106 @@ def update_partitions_map( schema=table_schema, ) +def files(self, snapshot_id

Re: [PR] Add `InclusiveMetricsEvaluator` [iceberg-rust]

2024-05-23 Thread via GitHub
Fokko commented on code in PR #347: URL: https://github.com/apache/iceberg-rust/pull/347#discussion_r1611145207 ## crates/iceberg/src/expr/visitors/inclusive_metrics_evaluator.rs: ## @@ -0,0 +1,2255 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more co

Re: [PR] Make current-snapshot-id optional while maintaining backwards compatibility [iceberg-rust]

2024-05-23 Thread via GitHub
Fokko commented on PR #374: URL: https://github.com/apache/iceberg-rust/pull/374#issuecomment-2126463830 Sorry for the late reply as I was touching grass. We're trying to solve two problems: - Don't produce `-1` since it is erroneous unless there is a snapshot with the ID `-1`.

Re: [PR] feat: make file scan task serializable [iceberg-rust]

2024-05-23 Thread via GitHub
Fokko commented on code in PR #377: URL: https://github.com/apache/iceberg-rust/pull/377#discussion_r1611209532 ## crates/iceberg/src/scan.rs: ## @@ -463,18 +464,19 @@ impl ManifestEvaluatorCache { } /// A task to scan part of file. -#[derive(Debug)] +#[derive(Debug, Clone,

Re: [PR] Bump mkdocstrings-python from 1.10.2 to 1.10.3 [iceberg-python]

2024-05-23 Thread via GitHub
Fokko merged PR #762: URL: https://github.com/apache/iceberg-python/pull/762 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] feat: support append data file and add e2e test [iceberg-rust]

2024-05-23 Thread via GitHub
Fokko commented on code in PR #349: URL: https://github.com/apache/iceberg-rust/pull/349#discussion_r1611236654 ## crates/iceberg/src/transaction.rs: ## @@ -121,6 +166,270 @@ impl<'a> Transaction<'a> { } } +/// FastAppendAction is a transaction action for fast append dat

Re: [PR] feat: support append data file and add e2e test [iceberg-rust]

2024-05-23 Thread via GitHub
Fokko commented on code in PR #349: URL: https://github.com/apache/iceberg-rust/pull/349#discussion_r1611244871 ## crates/iceberg/src/transaction.rs: ## @@ -121,6 +166,270 @@ impl<'a> Transaction<'a> { } } +/// FastAppendAction is a transaction action for fast append dat

Re: [PR] feat: support append data file and add e2e test [iceberg-rust]

2024-05-23 Thread via GitHub
Fokko commented on code in PR #349: URL: https://github.com/apache/iceberg-rust/pull/349#discussion_r1611247468 ## crates/e2e_test/tests/append_data_file_test.rs: ## @@ -0,0 +1,212 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license

Re: [PR] Hive: Use base table metadata to create HiveLock [iceberg]

2024-05-23 Thread via GitHub
lirui-apache commented on PR #10016: URL: https://github.com/apache/iceberg/pull/10016#issuecomment-2126541809 Hi @pvary @szehon-ho I just rebased the PR, do you think it's good to go? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] feat: support append data file and add e2e test [iceberg-rust]

2024-05-23 Thread via GitHub
Fokko commented on code in PR #349: URL: https://github.com/apache/iceberg-rust/pull/349#discussion_r1611259929 ## crates/iceberg/src/io.rs: ## @@ -368,6 +368,9 @@ impl Storage { new_props.insert("root".to_string(), DEFAULT_ROOT_PATH.to_string()); match schem

Re: [PR] feat: support append data file and add e2e test [iceberg-rust]

2024-05-23 Thread via GitHub
Fokko commented on code in PR #349: URL: https://github.com/apache/iceberg-rust/pull/349#discussion_r1611261437 ## crates/iceberg/src/transaction.rs: ## @@ -121,6 +166,270 @@ impl<'a> Transaction<'a> { } } +/// FastAppendAction is a transaction action for fast append dat

Re: [PR] feat: support append data file and add e2e test [iceberg-rust]

2024-05-23 Thread via GitHub
Fokko commented on code in PR #349: URL: https://github.com/apache/iceberg-rust/pull/349#discussion_r1611263732 ## crates/iceberg/src/spec/datatypes.rs: ## @@ -260,6 +279,29 @@ impl<'de> Deserialize<'de> for PrimitiveType { } } +impl PrimitiveType { +/// Check whethe

Re: [PR] feat: support append data file and add e2e test [iceberg-rust]

2024-05-23 Thread via GitHub
Fokko commented on code in PR #349: URL: https://github.com/apache/iceberg-rust/pull/349#discussion_r1611264064 ## crates/iceberg/src/spec/datatypes.rs: ## @@ -260,6 +279,29 @@ impl<'de> Deserialize<'de> for PrimitiveType { } } +impl PrimitiveType { +/// Check whethe

Re: [PR] feat: support append data file and add e2e test [iceberg-rust]

2024-05-23 Thread via GitHub
ZENOTME commented on code in PR #349: URL: https://github.com/apache/iceberg-rust/pull/349#discussion_r1611265301 ## crates/e2e_test/tests/append_data_file_test.rs: ## @@ -0,0 +1,212 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor licens

Re: [PR] feat: support append data file and add e2e test [iceberg-rust]

2024-05-23 Thread via GitHub
Fokko commented on code in PR #349: URL: https://github.com/apache/iceberg-rust/pull/349#discussion_r1611273092 ## crates/iceberg/src/transaction.rs: ## @@ -121,6 +190,313 @@ impl<'a> Transaction<'a> { } } +/// FastAppendAction is a transaction action for fast append dat

Re: [PR] feat: support append data file and add e2e test [iceberg-rust]

2024-05-23 Thread via GitHub
ZENOTME commented on code in PR #349: URL: https://github.com/apache/iceberg-rust/pull/349#discussion_r1611275943 ## crates/iceberg/src/transaction.rs: ## @@ -121,6 +166,270 @@ impl<'a> Transaction<'a> { } } +/// FastAppendAction is a transaction action for fast append d

Re: [PR] feat: support append data file and add e2e test [iceberg-rust]

2024-05-23 Thread via GitHub
Fokko commented on code in PR #349: URL: https://github.com/apache/iceberg-rust/pull/349#discussion_r1611283989 ## crates/iceberg/src/transaction.rs: ## @@ -121,6 +190,313 @@ impl<'a> Transaction<'a> { } } +/// FastAppendAction is a transaction action for fast append dat

Re: [PR] feat: support append data file and add e2e test [iceberg-rust]

2024-05-23 Thread via GitHub
Fokko commented on code in PR #349: URL: https://github.com/apache/iceberg-rust/pull/349#discussion_r1611285227 ## crates/e2e_test/tests/append_data_file_test.rs: ## @@ -0,0 +1,234 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license

Re: [PR] feat: support append data file and add e2e test [iceberg-rust]

2024-05-23 Thread via GitHub
Fokko commented on code in PR #349: URL: https://github.com/apache/iceberg-rust/pull/349#discussion_r1611289703 ## crates/iceberg/src/io.rs: ## @@ -368,6 +368,9 @@ impl Storage { new_props.insert("root".to_string(), DEFAULT_ROOT_PATH.to_string()); match schem

Re: [PR] Incremental Append Scan [iceberg-python]

2024-05-23 Thread via GitHub
Fokko commented on PR #533: URL: https://github.com/apache/iceberg-python/pull/533#issuecomment-2126662798 @hililiwei I'm sorry, this also fell off my radar. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

Re: [I] PyIceberg Near-Term Roadmap [iceberg-python]

2024-05-23 Thread via GitHub
Fokko commented on issue #736: URL: https://github.com/apache/iceberg-python/issues/736#issuecomment-2126667610 Sorry for the late reply, I was touching grass. > @kevinjqliu @Fokko Where would something like the Iceberg Spark create_changelog_view procedure fit in this roadmap? Is tha

Re: [PR] Add manifests metadata table [iceberg-python]

2024-05-23 Thread via GitHub
Fokko merged PR #717: URL: https://github.com/apache/iceberg-python/pull/717 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] Fix: Table-Exists if Server returns 204 [iceberg-python]

2024-05-23 Thread via GitHub
Fokko merged PR #739: URL: https://github.com/apache/iceberg-python/pull/739 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [I] Rest Catalog: table_exist does not succeed on 204 response code [iceberg-python]

2024-05-23 Thread via GitHub
Fokko closed issue #735: Rest Catalog: table_exist does not succeed on 204 response code URL: https://github.com/apache/iceberg-python/issues/735 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] feat: Adding literals [iceberg-go]

2024-05-23 Thread via GitHub
Fokko commented on code in PR #76: URL: https://github.com/apache/iceberg-go/pull/76#discussion_r1611387836 ## literals.go: ## @@ -0,0 +1,777 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// di

Re: [PR] feat: Adding literals [iceberg-go]

2024-05-23 Thread via GitHub
Fokko commented on code in PR #76: URL: https://github.com/apache/iceberg-go/pull/76#discussion_r1611390524 ## literals.go: ## @@ -0,0 +1,777 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// di

Re: [PR] build(deps): bump github.com/aws/aws-sdk-go-v2/service/glue from 1.80.1 to 1.80.3 [iceberg-go]

2024-05-23 Thread via GitHub
Fokko merged PR #84: URL: https://github.com/apache/iceberg-go/pull/84 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache

Re: [PR] build(deps): bump github.com/hamba/avro/v2 from 2.21.1 to 2.22.0 [iceberg-go]

2024-05-23 Thread via GitHub
Fokko merged PR #85: URL: https://github.com/apache/iceberg-go/pull/85 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache

Re: [PR] build(deps): bump github.com/aws/aws-sdk-go-v2/config from 1.26.6 to 1.27.15 [iceberg-go]

2024-05-23 Thread via GitHub
Fokko merged PR #86: URL: https://github.com/apache/iceberg-go/pull/86 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache

Re: [PR] feat: Adding literals [iceberg-go]

2024-05-23 Thread via GitHub
Fokko commented on code in PR #76: URL: https://github.com/apache/iceberg-go/pull/76#discussion_r1611395602 ## literals.go: ## @@ -0,0 +1,777 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// di

[PR] Flink 1.19: Fix flaky `TestIcebergSourceFailover > testBoundedWithSavepoint` [iceberg]

2024-05-23 Thread via GitHub
manuzhang opened a new pull request, #10370: URL: https://github.com/apache/iceberg/pull/10370 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

Re: [PR] Update fsspec.py to respect s3.signer.uri property [iceberg-python]

2024-05-23 Thread via GitHub
Fokko commented on PR #741: URL: https://github.com/apache/iceberg-python/pull/741#issuecomment-2127020569 @c-thiel Since it is in Java, I think it is fair to add it here. Two things: - Can you mention it in the docs as well? - Should we introduce a constant? -- This is an autom

Re: [PR] define snowflake catalog [iceberg-python]

2024-05-23 Thread via GitHub
questsul commented on code in PR #687: URL: https://github.com/apache/iceberg-python/pull/687#discussion_r1611636395 ## pyiceberg/catalog/snowflake_catalog.py: ## @@ -0,0 +1,237 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agree

Re: [PR] feat: add `ExpressionEvaluator` [iceberg-rust]

2024-05-23 Thread via GitHub
liurenjie1024 commented on code in PR #363: URL: https://github.com/apache/iceberg-rust/pull/363#discussion_r1611733596 ## crates/iceberg/src/expr/visitors/expression_evaluator.rs: ## @@ -0,0 +1,819 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more co

Re: [PR] feat: add `ExpressionEvaluator` [iceberg-rust]

2024-05-23 Thread via GitHub
liurenjie1024 commented on code in PR #363: URL: https://github.com/apache/iceberg-rust/pull/363#discussion_r1611740946 ## crates/iceberg/src/expr/visitors/expression_evaluator.rs: ## @@ -0,0 +1,819 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more co

Re: [PR] feat: add `ExpressionEvaluator` [iceberg-rust]

2024-05-23 Thread via GitHub
liurenjie1024 commented on code in PR #363: URL: https://github.com/apache/iceberg-rust/pull/363#discussion_r1611740946 ## crates/iceberg/src/expr/visitors/expression_evaluator.rs: ## @@ -0,0 +1,819 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more co

Re: [PR] Flink 1.19: Fix flaky `TestIcebergSourceFailover > testBoundedWithSavepoint` [iceberg]

2024-05-23 Thread via GitHub
manuzhang closed pull request #10370: Flink 1.19: Fix flaky `TestIcebergSourceFailover > testBoundedWithSavepoint` URL: https://github.com/apache/iceberg/pull/10370 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] Implement BoundPredicateVisitor trait for ManifestFilterVisitor [iceberg-rust]

2024-05-23 Thread via GitHub
liurenjie1024 commented on code in PR #367: URL: https://github.com/apache/iceberg-rust/pull/367#discussion_r1611756781 ## crates/iceberg/src/expr/visitors/manifest_evaluator.rs: ## @@ -103,98 +106,245 @@ impl BoundPredicateVisitor for ManifestFilterVisitor<'_> { refer

Re: [I] bug: `PrimitiveLiteral` and `Literal` should not be `Ord`. [iceberg-rust]

2024-05-23 Thread via GitHub
liurenjie1024 commented on issue #378: URL: https://github.com/apache/iceberg-rust/issues/378#issuecomment-2127281854 I think this is blocking prs with filter related, such as #363 and #367 -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [PR] Update fsspec.py to respect s3.signer.uri property [iceberg-python]

2024-05-23 Thread via GitHub
c-thiel commented on PR #741: URL: https://github.com/apache/iceberg-python/pull/741#issuecomment-2127322592 @Fokko I added docs and the constant. The constant is a very good idea - I hope we will be able to use remote signing with FileIO as well eventually. Right now only the fsspec impl

Re: [PR] Flink 1.19: Fix flaky `TestIcebergSourceFailover > testBoundedWithSavepoint` [iceberg]

2024-05-23 Thread via GitHub
manuzhang closed pull request #10370: Flink 1.19: Fix flaky `TestIcebergSourceFailover > testBoundedWithSavepoint` URL: https://github.com/apache/iceberg/pull/10370 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] Flink 1.19: Fix flaky `TestIcebergSourceFailover > testBoundedWithSavepoint` [iceberg]

2024-05-23 Thread via GitHub
stevenzwu commented on code in PR #10370: URL: https://github.com/apache/iceberg/pull/10370#discussion_r1611949141 ## flink/v1.19/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceFailover.java: ## @@ -124,7 +124,7 @@ public void testBoundedWithSavepoint() thr

Re: [PR] Flink 1.19: Fix flaky `TestIcebergSourceFailover > testBoundedWithSavepoint` [iceberg]

2024-05-23 Thread via GitHub
stevenzwu commented on code in PR #10370: URL: https://github.com/apache/iceberg/pull/10370#discussion_r1611949141 ## flink/v1.19/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceFailover.java: ## @@ -124,7 +124,7 @@ public void testBoundedWithSavepoint() thr

Re: [PR] Cherrypick and Publish Changes Snapshot Management Operations [iceberg-python]

2024-05-23 Thread via GitHub
chinmay-bhat commented on code in PR #750: URL: https://github.com/apache/iceberg-python/pull/750#discussion_r1612036245 ## tests/table/test_init.py: ## @@ -676,6 +676,15 @@ def test_update_metadata_set_snapshot_ref(table_v2: Table) -> None: ) +@pytest.mark.integration

Re: [PR] Flink 1.19: Fix flaky `TestIcebergSourceFailover > testBoundedWithSavepoint` [iceberg]

2024-05-23 Thread via GitHub
stevenzwu commented on code in PR #10370: URL: https://github.com/apache/iceberg/pull/10370#discussion_r1612082934 ## flink/v1.19/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceFailover.java: ## @@ -124,7 +124,7 @@ public void testBoundedWithSavepoint() thr

Re: [PR] Fix aggregate pushdown when optional DataFile stats are null [iceberg]

2024-05-23 Thread via GitHub
jkolash commented on code in PR #10273: URL: https://github.com/apache/iceberg/pull/10273#discussion_r1612093999 ## api/src/test/java/org/apache/iceberg/expressions/TestAggregateEvaluator.java: ## @@ -95,6 +95,24 @@ public class TestAggregateEvaluator { FILE, MISSING_SOME_N

Re: [PR] docs: installation of the new `iceberg_catalog_rest` added to the docs [iceberg-rust]

2024-05-23 Thread via GitHub
nishant-sachdeva commented on code in PR #355: URL: https://github.com/apache/iceberg-rust/pull/355#discussion_r1612121397 ## website/src/install.md: ## @@ -23,10 +23,11 @@ Cargo 1.75.0 or later is required to build. -Add `iceberg` into `Cargo.toml` dependencies: +Add `iceb

[I] [feat] Ability to read/write table using `version-hint.txt` [iceberg-python]

2024-05-23 Thread via GitHub
kevinjqliu opened a new issue, #763: URL: https://github.com/apache/iceberg-python/issues/763 ### Feature Request / Improvement Although not in the official spec, `version-hint.txt` can be useful to read an iceberg table without a catalog. This is useful when considering an ic

Re: [I] move this bug report to iceberg python [iceberg-python]

2024-05-23 Thread via GitHub
kevinjqliu closed issue #589: move this bug report to iceberg python URL: https://github.com/apache/iceberg-python/issues/589 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

Re: [I] move this bug report to iceberg python [iceberg-python]

2024-05-23 Thread via GitHub
kevinjqliu commented on issue #589: URL: https://github.com/apache/iceberg-python/issues/589#issuecomment-2127794976 closing in favor of #763 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

Re: [PR] Fix aggregate pushdown when optional DataFile stats are null [iceberg]

2024-05-23 Thread via GitHub
szehon-ho commented on PR #10273: URL: https://github.com/apache/iceberg/pull/10273#issuecomment-2127814740 Thanks, will merge once tests pass -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] feat: add `ExpressionEvaluator` [iceberg-rust]

2024-05-23 Thread via GitHub
marvinlanhenke commented on code in PR #363: URL: https://github.com/apache/iceberg-rust/pull/363#discussion_r1612174468 ## crates/iceberg/src/expr/visitors/expression_evaluator.rs: ## @@ -0,0 +1,819 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more c

Re: [PR] feat: add `ExpressionEvaluator` [iceberg-rust]

2024-05-23 Thread via GitHub
marvinlanhenke commented on code in PR #363: URL: https://github.com/apache/iceberg-rust/pull/363#discussion_r1612178234 ## crates/iceberg/src/expr/visitors/expression_evaluator.rs: ## @@ -0,0 +1,819 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more c

Re: [PR] feat: add `ExpressionEvaluator` [iceberg-rust]

2024-05-23 Thread via GitHub
marvinlanhenke commented on code in PR #363: URL: https://github.com/apache/iceberg-rust/pull/363#discussion_r1612174468 ## crates/iceberg/src/expr/visitors/expression_evaluator.rs: ## @@ -0,0 +1,819 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more c

Re: [PR] Introduces the new IcebergSink based on the new V2 Flink Sink Abstraction [iceberg]

2024-05-23 Thread via GitHub
pvary commented on code in PR #10179: URL: https://github.com/apache/iceberg/pull/10179#discussion_r1612188278 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/sink/IcebergSink.java: ## @@ -0,0 +1,780 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under o

Re: [PR] Introduces the new IcebergSink based on the new V2 Flink Sink Abstraction [iceberg]

2024-05-23 Thread via GitHub
pvary commented on code in PR #10179: URL: https://github.com/apache/iceberg/pull/10179#discussion_r1612194617 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/sink/IcebergSink.java: ## @@ -0,0 +1,780 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under o

Re: [PR] Hive: Use base table metadata to create HiveLock [iceberg]

2024-05-23 Thread via GitHub
pvary merged PR #10016: URL: https://github.com/apache/iceberg/pull/10016 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [I] HiveLock is chosen according to new table metadata [iceberg]

2024-05-23 Thread via GitHub
pvary closed issue #10006: HiveLock is chosen according to new table metadata URL: https://github.com/apache/iceberg/issues/10006 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] Hive: Use base table metadata to create HiveLock [iceberg]

2024-05-23 Thread via GitHub
pvary commented on PR #10016: URL: https://github.com/apache/iceberg/pull/10016#issuecomment-2127873404 Merged to main. Thanks @lirui-apache for the PR and @szehon-ho for the review! -- This is an automated message from the Apache Git Service. To respond to the message, please log on t

Re: [I] The need for adding hadoop-aws dependencies for `add_files` procedure [iceberg]

2024-05-23 Thread via GitHub
hpylypets-ua commented on issue #8322: URL: https://github.com/apache/iceberg/issues/8322#issuecomment-2127885277 So what's the plan to fix this or any workarounds? How can I migrate existing files on the s3 to the iceberg aws glue catalog backed with s3? it fails as indicated

Re: [PR] Flink 1.19: Fix flaky `TestIcebergSourceFailover > testBoundedWithSavepoint` [iceberg]

2024-05-23 Thread via GitHub
pvary commented on code in PR #10370: URL: https://github.com/apache/iceberg/pull/10370#discussion_r1612213520 ## flink/v1.19/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceFailover.java: ## @@ -124,7 +124,7 @@ public void testBoundedWithSavepoint() throws

Re: [PR] Fix aggregate pushdown when optional DataFile stats are null [iceberg]

2024-05-23 Thread via GitHub
szehon-ho merged PR #10273: URL: https://github.com/apache/iceberg/pull/10273 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [PR] Fix aggregate pushdown when optional DataFile stats are null [iceberg]

2024-05-23 Thread via GitHub
szehon-ho commented on PR #10273: URL: https://github.com/apache/iceberg/pull/10273#issuecomment-2127907725 Thanks @jkolash ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[PR] Bump duckdb from 0.10.2 to 0.10.3 [iceberg-python]

2024-05-23 Thread via GitHub
dependabot[bot] opened a new pull request, #764: URL: https://github.com/apache/iceberg-python/pull/764 Bumps [duckdb](https://github.com/duckdb/duckdb) from 0.10.2 to 0.10.3. Release notes Sourced from https://github.com/duckdb/duckdb/releases";>duckdb's releases. v0.10.3 Bu

[PR] Bump griffe from 0.45.1 to 0.45.2 [iceberg-python]

2024-05-23 Thread via GitHub
dependabot[bot] opened a new pull request, #765: URL: https://github.com/apache/iceberg-python/pull/765 Bumps [griffe](https://github.com/mkdocstrings/griffe) from 0.45.1 to 0.45.2. Release notes Sourced from https://github.com/mkdocstrings/griffe/releases";>griffe's releases.

Re: [I] remove orphan file question [iceberg]

2024-05-23 Thread via GitHub
JunseoChoJJ commented on issue #10363: URL: https://github.com/apache/iceberg/issues/10363#issuecomment-2128273438 @manuzhang thanks a lot -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[I] Introduce a parameter to control whether the flink writer is linked with the previous operator [iceberg]

2024-05-23 Thread via GitHub
huyuanfeng2018 opened a new issue, #10371: URL: https://github.com/apache/iceberg/issues/10371 ### Feature Request / Improvement The flink task written to iceberg can set write-parallelism, which allows it to have different parallelism degrees from the previous operator. However, when

Re: [I] Introduce a parameter to control whether the flink writer is linked with the previous operator [iceberg]

2024-05-23 Thread via GitHub
pvary commented on issue #10371: URL: https://github.com/apache/iceberg/issues/10371#issuecomment-2128535529 Could you use one of the ways described [here](https://nightlies.apache.org/flink/flink-docs-master/docs/dev/datastream/operators/overview/#task-chaining-and-resource-groups) to arch

Re: [PR] Flink 1.19: Fix flaky `TestIcebergSourceFailover > testBoundedWithSavepoint` [iceberg]

2024-05-23 Thread via GitHub
stevenzwu commented on code in PR #10370: URL: https://github.com/apache/iceberg/pull/10370#discussion_r1612756537 ## flink/v1.19/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceFailover.java: ## @@ -124,7 +124,7 @@ public void testBoundedWithSavepoint() thr

Re: [PR] Flink 1.19: Fix flaky `TestIcebergSourceFailover > testBoundedWithSavepoint` [iceberg]

2024-05-23 Thread via GitHub
stevenzwu commented on code in PR #10370: URL: https://github.com/apache/iceberg/pull/10370#discussion_r1612760150 ## flink/v1.19/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceFailover.java: ## @@ -124,7 +124,7 @@ public void testBoundedWithSavepoint() thr

Re: [I] bug: `StructAccessor` should return `Result>` [iceberg-rust]

2024-05-23 Thread via GitHub
liurenjie1024 commented on issue #379: URL: https://github.com/apache/iceberg-rust/issues/379#issuecomment-2128557481 cc @sdd Would you take a look at this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] feat: add `ExpressionEvaluator` [iceberg-rust]

2024-05-23 Thread via GitHub
liurenjie1024 commented on code in PR #363: URL: https://github.com/apache/iceberg-rust/pull/363#discussion_r1612791416 ## crates/iceberg/src/expr/visitors/expression_evaluator.rs: ## @@ -0,0 +1,819 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more co

Re: [PR] Flink 1.19: Fix flaky `TestIcebergSourceFailover > testBoundedWithSavepoint` [iceberg]

2024-05-23 Thread via GitHub
pvary commented on code in PR #10370: URL: https://github.com/apache/iceberg/pull/10370#discussion_r1612804231 ## flink/v1.19/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceFailover.java: ## @@ -124,7 +124,7 @@ public void testBoundedWithSavepoint() throws

Re: [PR] Hive: Use base table metadata to create HiveLock [iceberg]

2024-05-23 Thread via GitHub
lirui-apache commented on PR #10016: URL: https://github.com/apache/iceberg/pull/10016#issuecomment-2128649510 Thanks @pvary @szehon-ho for reviewing and merging! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] Core: rewrite should drop delete files by data sequence number partition wise [iceberg]

2024-05-23 Thread via GitHub
zinking closed pull request #9454: Core: rewrite should drop delete files by data sequence number partition wise URL: https://github.com/apache/iceberg/pull/9454 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo