Re: [PR] Flink: Maintenance - MonitorSource [iceberg]

pvary commented on code in PR #10308: URL: https://github.com/apache/iceberg/pull/10308#discussion_r1623164100 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/MonitorSource.java: ## @@ -0,0 +1,187 @@ +/* + * Licensed to the Apache Software Founda

Re: [PR] Add accessor for Schema identifier_field_ids [iceberg-rust]

liurenjie1024 commented on code in PR #388: URL: https://github.com/apache/iceberg-rust/pull/388#discussion_r1623135632 ## crates/iceberg/src/spec/schema.rs: ## @@ -327,6 +327,12 @@ impl Schema { &self.r#struct } +/// Returns [`identifier_field_ids`]. +#[

Re: [PR] Make current-snapshot-id optional while maintaining backwards compatibility [iceberg-rust]

liurenjie1024 commented on PR #374: URL: https://github.com/apache/iceberg-rust/pull/374#issuecomment-2143264331 > @liurenjie1024 @sdd, sure, I can work on #375 and use a config file to set the value. Let me know if you have a different approach in mind, thanks! @s-akhtar-baig I think

Re: [PR] fix: make PrimitiveLiteral and Literal not be Ord [iceberg-rust]

liurenjie1024 merged PR #386: URL: https://github.com/apache/iceberg-rust/pull/386 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [I] bug: `PrimitiveLiteral` and `Literal` should not be `Ord`. [iceberg-rust]

liurenjie1024 closed issue #378: bug: `PrimitiveLiteral` and `Literal` should not be `Ord`. URL: https://github.com/apache/iceberg-rust/issues/378 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [I] to_pandas() API which converts iceberg table scan to a pd.DataFrame will lost datetime data type and row order [iceberg-python]

github-actions[bot] commented on issue #132: URL: https://github.com/apache/iceberg-python/issues/132#issuecomment-2143160485 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity oc

Re: [PR] Support Appends with TimeTransform Partitions [iceberg-python]

Fokko merged PR #784: URL: https://github.com/apache/iceberg-python/pull/784 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

[PR] Support Appends with TimeTransform Partitions [iceberg-python]

syun64 opened a new pull request, #784: URL: https://github.com/apache/iceberg-python/pull/784 reopening https://github.com/apache/iceberg-python/pull/703 due to issues with git diff detection on the PR -- This is an automated message from the Apache Git Service. To respond to the message

Re: [PR] Support Appends with TimeTransform Partitions [iceberg-python]

syun64 closed pull request #703: Support Appends with TimeTransform Partitions URL: https://github.com/apache/iceberg-python/pull/703 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

Re: [PR] Support Appends with TimeTransform Partitions [iceberg-python]

syun64 commented on PR #703: URL: https://github.com/apache/iceberg-python/pull/703#issuecomment-2142845753 I'll close this PR and reopen it and see if that helps -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] Support Appends with TimeTransform Partitions [iceberg-python]

syun64 commented on PR #703: URL: https://github.com/apache/iceberg-python/pull/703#issuecomment-2142844385 @Fokko hmmm yeah I've seen this happen some times and I don't quite understand it - when you do a diff between the two branches, it shows the right changes: https://github.com/apache

Re: [PR] Support Appends with TimeTransform Partitions [iceberg-python]

Fokko commented on PR #703: URL: https://github.com/apache/iceberg-python/pull/703#issuecomment-2142832199 @syun64 It looks like something is off. I see a lot of unrelated changes 🤔 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Gi

Re: [I] UUID type support in Spark is incomplete? [iceberg]

igniris87 commented on issue #4038: URL: https://github.com/apache/iceberg/issues/4038#issuecomment-2142831324 hey nastra how can I use this fix https://github.com/apache/iceberg/pull/7399 using pyspark and aws glue? -- This is an automated message from the Apache Git Service. To respond

Re: [PR] Spark Action to Analyze table [iceberg]

amogh-jahagirdar commented on code in PR #10288: URL: https://github.com/apache/iceberg/pull/10288#discussion_r1622832030 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/AnalyzeTableSparkAction.java: ## @@ -0,0 +1,150 @@ +/* + * Licensed to the Apache Software

Re: [PR] Spark Action to Analyze table [iceberg]

amogh-jahagirdar commented on code in PR #10288: URL: https://github.com/apache/iceberg/pull/10288#discussion_r1622832030 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/AnalyzeTableSparkAction.java: ## @@ -0,0 +1,150 @@ +/* + * Licensed to the Apache Software

Re: [PR] Flink: Maintenance - MonitorSource [iceberg]

stevenzwu commented on code in PR #10308: URL: https://github.com/apache/iceberg/pull/10308#discussion_r1622721206 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/SingleThreadedIteratorSource.java: ## @@ -0,0 +1,195 @@ +/* + * Licensed to the Apa

Re: [PR] Spark Action to Analyze table [iceberg]

karuppayya commented on code in PR #10288: URL: https://github.com/apache/iceberg/pull/10288#discussion_r1622775896 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/AnalyzeTableSparkAction.java: ## @@ -0,0 +1,150 @@ +/* + * Licensed to the Apache Software Found

Re: [PR] Make current-snapshot-id optional while maintaining backwards compatibility [iceberg-rust]

s-akhtar-baig commented on PR #374: URL: https://github.com/apache/iceberg-rust/pull/374#issuecomment-2142649684 @liurenjie1024 @sdd, sure, I can work on https://github.com/apache/iceberg-rust/issues/375 and use a config file to set the value. Let me know if you have a different approach in

Re: [PR] Implement BoundPredicateVisitor trait for ManifestFilterVisitor [iceberg-rust]

s-akhtar-baig commented on code in PR #367: URL: https://github.com/apache/iceberg-rust/pull/367#discussion_r1622704735 ## crates/iceberg/src/expr/visitors/manifest_evaluator.rs: ## @@ -103,98 +106,264 @@ impl BoundPredicateVisitor for ManifestFilterVisitor<'_> { refer

Re: [PR] Implement BoundPredicateVisitor trait for ManifestFilterVisitor [iceberg-rust]

s-akhtar-baig commented on code in PR #367: URL: https://github.com/apache/iceberg-rust/pull/367#discussion_r1622706982 ## crates/iceberg/src/expr/visitors/manifest_evaluator.rs: ## @@ -103,98 +106,264 @@ impl BoundPredicateVisitor for ManifestFilterVisitor<'_> { refer

Re: [PR] Implement BoundPredicateVisitor trait for ManifestFilterVisitor [iceberg-rust]

s-akhtar-baig commented on code in PR #367: URL: https://github.com/apache/iceberg-rust/pull/367#discussion_r1622705122 ## crates/iceberg/src/expr/visitors/manifest_evaluator.rs: ## @@ -103,98 +106,264 @@ impl BoundPredicateVisitor for ManifestFilterVisitor<'_> { refer

Re: [PR] Implement BoundPredicateVisitor trait for ManifestFilterVisitor [iceberg-rust]

s-akhtar-baig commented on code in PR #367: URL: https://github.com/apache/iceberg-rust/pull/367#discussion_r1622704735 ## crates/iceberg/src/expr/visitors/manifest_evaluator.rs: ## @@ -103,98 +106,264 @@ impl BoundPredicateVisitor for ManifestFilterVisitor<'_> { refer

Re: [PR] Flink: Maintenance - MonitorSource [iceberg]

stevenzwu commented on code in PR #10308: URL: https://github.com/apache/iceberg/pull/10308#discussion_r1622689279 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/MonitorSource.java: ## @@ -0,0 +1,200 @@ +/* + * Licensed to the Apache Software Fo

Re: [PR] Flink: Maintenance - MonitorSource [iceberg]

stevenzwu commented on code in PR #10308: URL: https://github.com/apache/iceberg/pull/10308#discussion_r1622689279 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/MonitorSource.java: ## @@ -0,0 +1,200 @@ +/* + * Licensed to the Apache Software Fo

Re: [PR] Flink: Maintenance - MonitorSource [iceberg]

stevenzwu commented on code in PR #10308: URL: https://github.com/apache/iceberg/pull/10308#discussion_r1622689279 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/MonitorSource.java: ## @@ -0,0 +1,200 @@ +/* + * Licensed to the Apache Software Fo

Re: [PR] Flink: Maintenance - MonitorSource [iceberg]

stevenzwu commented on code in PR #10308: URL: https://github.com/apache/iceberg/pull/10308#discussion_r1622578621 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/MonitorSource.java: ## @@ -0,0 +1,200 @@ +/* + * Licensed to the Apache Software Fo

Re: [PR] Flink: Maintenance - MonitorSource [iceberg]

stevenzwu commented on code in PR #10308: URL: https://github.com/apache/iceberg/pull/10308#discussion_r1622578621 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/MonitorSource.java: ## @@ -0,0 +1,200 @@ +/* + * Licensed to the Apache Software Fo

Re: [I] Iceberg Rest Catalog does not honor "s3.signer.uri" property [iceberg-python]

Fokko closed issue #740: Iceberg Rest Catalog does not honor "s3.signer.uri" property URL: https://github.com/apache/iceberg-python/issues/740 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

Re: [PR] Update fsspec.py to respect s3.signer.uri property [iceberg-python]

Fokko merged PR #741: URL: https://github.com/apache/iceberg-python/pull/741 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] Flink: Maintenance - MonitorSource [iceberg]

stevenzwu commented on code in PR #10308: URL: https://github.com/apache/iceberg/pull/10308#discussion_r1622579093 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/MonitorSource.java: ## @@ -0,0 +1,200 @@ +/* + * Licensed to the Apache Software Fo

Re: [PR] Flink: Maintenance - MonitorSource [iceberg]

stevenzwu commented on code in PR #10308: URL: https://github.com/apache/iceberg/pull/10308#discussion_r1622579093 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/MonitorSource.java: ## @@ -0,0 +1,200 @@ +/* + * Licensed to the Apache Software Fo

Re: [PR] Flink: Maintenance - MonitorSource [iceberg]

stevenzwu commented on code in PR #10308: URL: https://github.com/apache/iceberg/pull/10308#discussion_r1622578621 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/MonitorSource.java: ## @@ -0,0 +1,200 @@ +/* + * Licensed to the Apache Software Fo

Re: [PR] Update fsspec.py to respect s3.signer.uri property [iceberg-python]

Fokko commented on PR #741: URL: https://github.com/apache/iceberg-python/pull/741#issuecomment-2142322807 @c-thiel The idea is to support both. The problem is that with Arrow it is hard to inject in the signing process, since most of the code is pushed down to the C level, and there are no

Re: [PR] Cherrypick and Publish Changes Snapshot Management Operations [iceberg-python]

chinmay-bhat commented on code in PR #750: URL: https://github.com/apache/iceberg-python/pull/750#discussion_r1622308580 ## tests/table/test_init.py: ## @@ -676,6 +676,15 @@ def test_update_metadata_set_snapshot_ref(table_v2: Table) -> None: ) +@pytest.mark.integration

Re: [PR] Spark Action to Analyze table [iceberg]

jeesou commented on code in PR #10288: URL: https://github.com/apache/iceberg/pull/10288#discussion_r1622261695 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/AnalyzeTableSparkAction.java: ## @@ -0,0 +1,150 @@ +/* + * Licensed to the Apache Software Foundatio

Re: [PR] Spark Action to Analyze table [iceberg]

jeesou commented on code in PR #10288: URL: https://github.com/apache/iceberg/pull/10288#discussion_r1622261695 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/AnalyzeTableSparkAction.java: ## @@ -0,0 +1,150 @@ +/* + * Licensed to the Apache Software Foundatio

Re: [I] Questions on some requirement in view spec [iceberg]

ajantha-bhat commented on issue #10410: URL: https://github.com/apache/iceberg/issues/10410#issuecomment-2141766249 cc: @jbonofre, @snazy, @dimas-b -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[I] Questions on some requirement in view spec [iceberg]

ajantha-bhat opened a new issue, #10410: URL: https://github.com/apache/iceberg/issues/10410 ### Query engine NA ### Question 1. Should the `summary` be optional? https://iceberg.apache.org/view-spec/#versions Because the contents of summary is not strictly define

Re: [I] bug: Iceberg opendal missing s3 support [iceberg-rust]

liurenjie1024 commented on issue #385: URL: https://github.com/apache/iceberg-rust/issues/385#issuecomment-2141754544 > Maybe we can add features like `storage-s3`, `storage-gcs` at the iceberg side. +1 -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] Support getting snapshot at or right before the given timestamp [iceberg-python]

chinmay-bhat commented on code in PR #748: URL: https://github.com/apache/iceberg-python/pull/748#discussion_r1622175948 ## pyiceberg/table/__init__.py: ## @@ -1290,6 +1291,17 @@ def snapshot_by_name(self, name: str) -> Optional[Snapshot]: return self.snapshot_by_i

Re: [PR] Support getting snapshot at or right before the given timestamp [iceberg-python]

chinmay-bhat commented on code in PR #748: URL: https://github.com/apache/iceberg-python/pull/748#discussion_r1622176755 ## pyiceberg/table/__init__.py: ## @@ -1290,6 +1291,19 @@ def snapshot_by_name(self, name: str) -> Optional[Snapshot]: return self.snapshot_by_i

Re: [PR] Support getting snapshot at or right before the given timestamp [iceberg-python]

chinmay-bhat commented on code in PR #748: URL: https://github.com/apache/iceberg-python/pull/748#discussion_r1622175948 ## pyiceberg/table/__init__.py: ## @@ -1290,6 +1291,17 @@ def snapshot_by_name(self, name: str) -> Optional[Snapshot]: return self.snapshot_by_i

Re: [I] bug: Iceberg opendal missing s3 support [iceberg-rust]

liurenjie1024 commented on issue #385: URL: https://github.com/apache/iceberg-rust/issues/385#issuecomment-2141620053 Good catch! Thanks @c-thiel for the finding! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] Support creating tags by adding `set_ref_snapshot` API [iceberg-python]

chinmay-bhat commented on PR #728: URL: https://github.com/apache/iceberg-python/pull/728#issuecomment-2141599731 Thank you Honah for the review! :) I've updated the tests! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

[I] Broken links in Spark Writes documentation [iceberg]

gphilipp opened a new issue, #10409: URL: https://github.com/apache/iceberg/issues/10409 ### Apache Iceberg version 1.5.2 (latest release) ### Query engine Spark ### Please describe the bug 🐞 I'm not an Iceberg user, just browsing the doc around. I've no

Re: [PR] [Build] Add a script to execute revapi without gradle plugin [iceberg]

Fokko commented on code in PR #10386: URL: https://github.com/apache/iceberg/pull/10386#discussion_r1621962589 ## dev/revapi: ## @@ -0,0 +1,88 @@ +#!/usr/bin/env bash + +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. S

Re: [PR] [Build] Add a script to execute revapi without gradle plugin [iceberg]

Fokko commented on code in PR #10386: URL: https://github.com/apache/iceberg/pull/10386#discussion_r1621960019 ## .github/workflows/api-binary-compatibility.yml: ## @@ -17,45 +17,17 @@ # under the License. # -name: "API Binary Compatibility Checks" -on: - push: -branche

Re: [PR] [Build] Add a script to execute revapi without gradle plugin [iceberg]

Fokko commented on PR #10386: URL: https://github.com/apache/iceberg/pull/10386#issuecomment-2141479501 @jbonofre I gave it another swing, looks good. One thing I noticed when I tried to trigger rev-API, it didn't report the breaking change. Example is in https://github.com/apache/iceberg/p

Re: [PR] Support creating tags by adding `set_ref_snapshot` API [iceberg-python]

chinmay-bhat commented on code in PR #728: URL: https://github.com/apache/iceberg-python/pull/728#discussion_r1621937279 ## tests/table/test_init.py: ## @@ -652,6 +652,58 @@ def test_update_metadata_add_snapshot(table_v2: Table) -> None: assert new_metadata.last_updated_ms

Re: [PR] feat: Support viewfs scheme along side with hdfs [iceberg-python]

Fokko merged PR #777: URL: https://github.com/apache/iceberg-python/pull/777 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] Support creating tags by adding `set_ref_snapshot` API [iceberg-python]

HonahX commented on code in PR #728: URL: https://github.com/apache/iceberg-python/pull/728#discussion_r1621822758 ## pyiceberg/table/__init__.py: ## @@ -277,6 +279,50 @@ def __init__(self, table: Table, autocommit: bool = False): self._autocommit = autocommit

Re: [PR] Support CreateTableTransaction for HiveCatalog [iceberg-python]

HonahX merged PR #683: URL: https://github.com/apache/iceberg-python/pull/683 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [PR] Support creating tags by adding `set_ref_snapshot` API [iceberg-python]

chinmay-bhat commented on code in PR #728: URL: https://github.com/apache/iceberg-python/pull/728#discussion_r1621876496 ## pyiceberg/table/__init__.py: ## @@ -277,6 +279,50 @@ def __init__(self, table: Table, autocommit: bool = False): self._autocommit = autocommit

Re: [PR] Support CreateTableTransaction for SqlCatalog [iceberg-python]

HonahX commented on PR #684: URL: https://github.com/apache/iceberg-python/pull/684#issuecomment-2141385387 Thanks for reviewing! @Fokko @syun64 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

Re: [I] bug: Iceberg opendal missing s3 support [iceberg-rust]

c-thiel commented on issue #385: URL: https://github.com/apache/iceberg-rust/issues/385#issuecomment-2141384589 > BTW how the integration tests passed when s3 is not included? I think at least [this test](https://github.com/apache/iceberg-rust/blob/048eac801f09ca58e610a020db2e29bd95c22db0/c

Re: [PR] Support CreateTableTransaction for SqlCatalog [iceberg-python]

HonahX merged PR #684: URL: https://github.com/apache/iceberg-python/pull/684 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [PR] Support getting a snapshot right before the given timestamp [iceberg-python]

HonahX commented on code in PR #748: URL: https://github.com/apache/iceberg-python/pull/748#discussion_r1621869633 ## pyiceberg/table/__init__.py: ## @@ -1290,6 +1291,17 @@ def snapshot_by_name(self, name: str) -> Optional[Snapshot]: return self.snapshot_by_id(ref.

Re: [PR] Support getting a snapshot right before the given timestamp [iceberg-python]

HonahX commented on code in PR #748: URL: https://github.com/apache/iceberg-python/pull/748#discussion_r1621866561 ## pyiceberg/table/__init__.py: ## @@ -1290,6 +1291,19 @@ def snapshot_by_name(self, name: str) -> Optional[Snapshot]: return self.snapshot_by_id(ref.

Re: [PR] Support getting a snapshot right before the given timestamp [iceberg-python]

HonahX commented on code in PR #748: URL: https://github.com/apache/iceberg-python/pull/748#discussion_r1621866561 ## pyiceberg/table/__init__.py: ## @@ -1290,6 +1291,19 @@ def snapshot_by_name(self, name: str) -> Optional[Snapshot]: return self.snapshot_by_id(ref.

Re: [PR] Support getting a snapshot right before the given timestamp [iceberg-python]

HonahX commented on code in PR #748: URL: https://github.com/apache/iceberg-python/pull/748#discussion_r1621866561 ## pyiceberg/table/__init__.py: ## @@ -1290,6 +1291,19 @@ def snapshot_by_name(self, name: str) -> Optional[Snapshot]: return self.snapshot_by_id(ref.

Re: [PR] Support getting a snapshot right before the given timestamp [iceberg-python]

HonahX commented on code in PR #748: URL: https://github.com/apache/iceberg-python/pull/748#discussion_r1621866561 ## pyiceberg/table/__init__.py: ## @@ -1290,6 +1291,19 @@ def snapshot_by_name(self, name: str) -> Optional[Snapshot]: return self.snapshot_by_id(ref.