Re: [PR] API, Core: Add manifestPath API to ContentFile which will return the path to a manifest from which the content file resides in [iceberg]

2024-08-29 Thread via GitHub
aokolnychyi commented on code in PR #11044: URL: https://github.com/apache/iceberg/pull/11044#discussion_r1737974697 ## core/src/main/java/org/apache/iceberg/InheritableMetadataFactory.java: ## @@ -46,11 +49,14 @@ static class BaseInheritableMetadata implements InheritableMetad

Re: [PR] Add Scan Planning Endpoints to open api spec [iceberg]

2024-08-29 Thread via GitHub
amogh-jahagirdar commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1737933723 ## open-api/rest-catalog-open-api.yaml: ## @@ -541,6 +541,216 @@ paths: 5XX: $ref: '#/components/responses/ServerErrorResponse' + /v1/{p

[I] Kafka: runtime integration test failure or flaky [iceberg]

2024-08-29 Thread via GitHub
stevenzwu opened a new issue, #11046: URL: https://github.com/apache/iceberg/issues/11046 ### Apache Iceberg version main (development) ### Query engine Kafka Connect ### Please describe the bug šŸž here is an example CI build failure for main branch: https:/

Re: [PR] Use Snapshot's statistics file in SparkScan [iceberg]

2024-08-29 Thread via GitHub
amogh-jahagirdar commented on code in PR #11040: URL: https://github.com/apache/iceberg/pull/11040#discussion_r1737878864 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestSparkScan.java: ## @@ -293,6 +293,73 @@ public void testTableWithTwoColStats() throws

Re: [I] Issue with duplicate kafka connect artifacts [iceberg]

2024-08-29 Thread via GitHub
amogh-jahagirdar closed issue #11026: Issue with duplicate kafka connect artifacts URL: https://github.com/apache/iceberg/issues/11026 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

Re: [PR] Kafka Connect: Disable publish tasks in runtime project [iceberg]

2024-08-29 Thread via GitHub
amogh-jahagirdar merged PR #11032: URL: https://github.com/apache/iceberg/pull/11032 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@

Re: [PR] Kafka Connect: Disable publish tasks in runtime project [iceberg]

2024-08-29 Thread via GitHub
amogh-jahagirdar commented on PR #11032: URL: https://github.com/apache/iceberg/pull/11032#issuecomment-2319949634 Thanks @bryanck! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [PR] Flink-1.19: Fix the file offset mismatch when Flink reader first seekā€¦ [iceberg]

2024-08-29 Thread via GitHub
zhongyujiang commented on PR #10567: URL: https://github.com/apache/iceberg/pull/10567#issuecomment-2319760314 > @zhongyujiang is this issue with the old FlinkSource or with the new FLIP-27 IcebergSource. StreamingReaderOperator is for the old FlinkSource, which we plan to deprecate in favo

Re: [I] support create partition table for non REST catalog [iceberg-rust]

2024-08-29 Thread via GitHub
liurenjie1024 commented on issue #578: URL: https://github.com/apache/iceberg-rust/issues/578#issuecomment-2319726424 Thanks @FANNG1 for contribution. I've left comments in your pr. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

Re: [PR] feat: support create partition table for non REST catalog [iceberg-rust]

2024-08-29 Thread via GitHub
liurenjie1024 commented on PR #577: URL: https://github.com/apache/iceberg-rust/pull/577#issuecomment-2319725641 Hi, @FANNG1 Thanks for your contribution. The reason why we use `UnboundPartitionSpec` rather `PartitionSpec` is to simplify the usage of this method. `PartitionSpec` is bound to

Re: [PR] Spark 3.5: Skip sort for incomparable data types in CreateChangelogViewProcedure [iceberg]

2024-08-29 Thread via GitHub
karuppayya commented on code in PR #11045: URL: https://github.com/apache/iceberg/pull/11045#discussion_r1737628804 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestCreateChangelogViewProcedure.java: ## @@ -242,6 +249,34 @@ public void testUpd

Re: [PR] Core: Fix the behavior of IncrementalFileCleanup when expire a snapshot [iceberg]

2024-08-29 Thread via GitHub
hantangwangd commented on code in PR #10983: URL: https://github.com/apache/iceberg/pull/10983#discussion_r1737646125 ## core/src/main/java/org/apache/iceberg/RemoveSnapshots.java: ## @@ -116,6 +117,7 @@ public ExpireSnapshots cleanExpiredFiles(boolean clean) { public ExpireS

Re: [PR] Flink: Add Schemas table to inspecting table [iceberg]

2024-08-29 Thread via GitHub
github-actions[bot] commented on PR #7675: URL: https://github.com/apache/iceberg/pull/7675#issuecomment-2319519020 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think thatā€™s incorrect or this pull

Re: [PR] Parquet: Fix possible stream duplicate close issue when using DelegateingOutputStream [iceberg]

2024-08-29 Thread via GitHub
github-actions[bot] commented on PR #7696: URL: https://github.com/apache/iceberg/pull/7696#issuecomment-2319519068 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think thatā€™s incorrect or this pull

Re: [PR] Flink: Add partition fields to equalityColumns [iceberg]

2024-08-29 Thread via GitHub
github-actions[bot] commented on PR #7620: URL: https://github.com/apache/iceberg/pull/7620#issuecomment-2319518983 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think thatā€™s incorrect or this pull

Re: [I] Exceptions when FlinkSQL writes to partitioned tables [iceberg]

2024-08-29 Thread via GitHub
github-actions[bot] commented on issue #7618: URL: https://github.com/apache/iceberg/issues/7618#issuecomment-2319518938 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [PR] Fix the dependency exclusion for apache commons [iceberg]

2024-08-29 Thread via GitHub
github-actions[bot] commented on PR #7619: URL: https://github.com/apache/iceberg/pull/7619#issuecomment-2319518956 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think thatā€™s incorrect or this pull

Re: [I] Exceptions when FlinkSQL writes to partitioned tables [iceberg]

2024-08-29 Thread via GitHub
github-actions[bot] closed issue #7618: Exceptions when FlinkSQL writes to partitioned tables URL: https://github.com/apache/iceberg/issues/7618 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

Re: [PR] Spark: Added Scala/Java IcebergMergeInto Api on spark3.4 [iceberg]

2024-08-29 Thread via GitHub
github-actions[bot] commented on PR #7607: URL: https://github.com/apache/iceberg/pull/7607#issuecomment-2319518893 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think thatā€™s incorrect or this pull

Re: [PR] Adding support for custom partition spec during rewrite [iceberg]

2024-08-29 Thread via GitHub
github-actions[bot] commented on PR #7585: URL: https://github.com/apache/iceberg/pull/7585#issuecomment-2319518851 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think thatā€™s incorrect or this pull

Re: [I] AWS: provide option to hide old fields in Glue table [iceberg]

2024-08-29 Thread via GitHub
github-actions[bot] commented on issue #7584: URL: https://github.com/apache/iceberg/issues/7584#issuecomment-2319518817 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [PR] Build: Capture build scans on ge.apache.org to benefit from deep build insights [iceberg]

2024-08-29 Thread via GitHub
github-actions[bot] commented on PR #7567: URL: https://github.com/apache/iceberg/pull/7567#issuecomment-2319518778 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think thatā€™s incorrect or this pull

Re: [PR] Core: Fix retry behavior for Jdbc Client [iceberg]

2024-08-29 Thread via GitHub
github-actions[bot] commented on PR #7561: URL: https://github.com/apache/iceberg/pull/7561#issuecomment-2319518740 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think thatā€™s incorrect or this pull

Re: [I] Support Rewrite Datafiles into a custom Partition Spec [iceberg]

2024-08-29 Thread via GitHub
github-actions[bot] commented on issue #7557: URL: https://github.com/apache/iceberg/issues/7557#issuecomment-2319518693 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [PR] Materialized View Spec [iceberg]

2024-08-29 Thread via GitHub
stevenzwu commented on code in PR #11041: URL: https://github.com/apache/iceberg/pull/11041#discussion_r1737474791 ## format/view-spec.md: ## @@ -158,6 +173,59 @@ Each entry in `version-log` is a struct with the following fields: | _required_ | `timestamp-ms` | Timestamp when

[PR] Bump python-snappy from 0.7.2 to 0.7.3 [iceberg-python]

2024-08-29 Thread via GitHub
dependabot[bot] opened a new pull request, #1115: URL: https://github.com/apache/iceberg-python/pull/1115 Bumps [python-snappy](https://github.com/intake/python-snappy) from 0.7.2 to 0.7.3. Commits See full diff in https://github.com/intake/python-snappy/commits";>compare view

Re: [PR] Core: Fix the behavior of IncrementalFileCleanup when expire a snapshot [iceberg]

2024-08-29 Thread via GitHub
amogh-jahagirdar commented on code in PR #10983: URL: https://github.com/apache/iceberg/pull/10983#discussion_r1737259547 ## core/src/main/java/org/apache/iceberg/RemoveSnapshots.java: ## @@ -321,6 +323,14 @@ ExpireSnapshots withIncrementalCleanup(boolean useIncrementalCleanup)

Re: [PR] Add Scan Planning Endpoints to open api spec [iceberg]

2024-08-29 Thread via GitHub
rahil-c commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1737261117 ## open-api/rest-catalog-open-api.yaml: ## @@ -541,6 +541,263 @@ paths: 5XX: $ref: '#/components/responses/ServerErrorResponse' + /v1/{prefix}/na

Re: [PR] support python 3.12 [iceberg-python]

2024-08-29 Thread via GitHub
cclauss commented on PR #254: URL: https://github.com/apache/iceberg-python/pull/254#issuecomment-2319033114 Cool! Python 3.13 is scheduled to be released on October 1st. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and us

Re: [PR] Make `commit_table` public [iceberg-python]

2024-08-29 Thread via GitHub
Fokko commented on code in PR #1112: URL: https://github.com/apache/iceberg-python/pull/1112#discussion_r1737194124 ## pyiceberg/catalog/rest.py: ## @@ -734,9 +739,11 @@ def _commit_table(self, table_request: CommitTableRequest) -> CommitTableRespons CommitFailedEx

Re: [PR] Materialized View Spec [iceberg]

2024-08-29 Thread via GitHub
JanKaul commented on code in PR #11041: URL: https://github.com/apache/iceberg/pull/11041#discussion_r1737152915 ## format/view-spec.md: ## @@ -158,6 +173,59 @@ Each entry in `version-log` is a struct with the following fields: | _required_ | `timestamp-ms` | Timestamp when t

Re: [PR] Materialized View Spec [iceberg]

2024-08-29 Thread via GitHub
JanKaul commented on code in PR #11041: URL: https://github.com/apache/iceberg/pull/11041#discussion_r1737147291 ## format/view-spec.md: ## @@ -158,6 +173,59 @@ Each entry in `version-log` is a struct with the following fields: | _required_ | `timestamp-ms` | Timestamp when t

Re: [PR] Make `commit_table` public [iceberg-python]

2024-08-29 Thread via GitHub
Fokko commented on code in PR #1112: URL: https://github.com/apache/iceberg-python/pull/1112#discussion_r1737130122 ## pyiceberg/catalog/glue.py: ## @@ -462,10 +468,8 @@ def _commit_table(self, table_request: CommitTableRequest) -> CommitTableRespons NoSuchTableErr

Re: [PR] Make `commit_table` public [iceberg-python]

2024-08-29 Thread via GitHub
Fokko commented on code in PR #1112: URL: https://github.com/apache/iceberg-python/pull/1112#discussion_r1737129443 ## pyiceberg/catalog/rest.py: ## @@ -719,12 +721,15 @@ def _remove_catalog_name_from_table_request_identifier(self, table_request: Comm ) re

Re: [PR] Updating SparkScan to only read Apache DataSketches [iceberg]

2024-08-29 Thread via GitHub
jeesou commented on PR #11035: URL: https://github.com/apache/iceberg/pull/11035#issuecomment-2318875961 Hi Adding a enhancement in test case - For no stats scenario also, we were traversing over the expectedNDVs Map, which was empty, and thus the Assert was never reached, and it was

Re: [PR] Emit warnings instead of failing when seeing unsupported configuration [iceberg-python]

2024-08-29 Thread via GitHub
Fokko merged PR #: URL: https://github.com/apache/iceberg-python/pull/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [PR] support python 3.12 [iceberg-python]

2024-08-29 Thread via GitHub
Jiafi commented on PR #254: URL: https://github.com/apache/iceberg-python/pull/254#issuecomment-2318855340 looks like the [next release of ray](https://github.com/ray-project/ray/compare/ray-2.35.0...master) should hopefully have 3.12 support -- This is an automated message from the Apac

Re: [PR] API, Core: Add manifestPath API to ContentFile which will return the path to a manifest from which the content file resides in [iceberg]

2024-08-29 Thread via GitHub
amogh-jahagirdar commented on code in PR #11044: URL: https://github.com/apache/iceberg/pull/11044#discussion_r1737002471 ## core/src/main/java/org/apache/iceberg/InheritableMetadataFactory.java: ## @@ -46,11 +49,14 @@ static class BaseInheritableMetadata implements Inheritable

[PR] API, Core: Add manifestPath API to ContentFile which will return the path to a manifest from which the content file resides in [iceberg]

2024-08-29 Thread via GitHub
amogh-jahagirdar opened a new pull request, #11044: URL: https://github.com/apache/iceberg/pull/11044 This change adds a manifestPath API to ContentFile which will return the path to a manifest from which the content file resides in. If the data or delete file was not read from a manifest,

Re: [PR] Core: fix NPE with HadoopFileIO because FileIOParser doesn't serialize Hadoop configuration [iceberg]

2024-08-29 Thread via GitHub
stevenzwu commented on code in PR #10926: URL: https://github.com/apache/iceberg/pull/10926#discussion_r1736772930 ## core/src/main/java/org/apache/iceberg/hadoop/HadoopFileIO.java: ## @@ -63,7 +63,11 @@ public class HadoopFileIO implements HadoopConfigurable, DelegateFileIO {

Re: [PR] Use SupportsPrefixOperations for Remove OrphanFile Procedure [iceberg]

2024-08-29 Thread via GitHub
flyrain commented on code in PR #7914: URL: https://github.com/apache/iceberg/pull/7914#discussion_r1736840318 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/DeleteOrphanFilesSparkAction.java: ## @@ -330,11 +345,18 @@ private Dataset listedFileDS() { Bro

Re: [PR] Use SupportsPrefixOperations for Remove OrphanFile Procedure [iceberg]

2024-08-29 Thread via GitHub
flyrain commented on code in PR #7914: URL: https://github.com/apache/iceberg/pull/7914#discussion_r1736836729 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/DeleteOrphanFilesSparkAction.java: ## @@ -299,7 +300,21 @@ private Dataset actualFileIdentDS() {

Re: [PR] Use SupportsPrefixOperations for Remove OrphanFile Procedure [iceberg]

2024-08-29 Thread via GitHub
flyrain commented on code in PR #7914: URL: https://github.com/apache/iceberg/pull/7914#discussion_r1736836729 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/DeleteOrphanFilesSparkAction.java: ## @@ -299,7 +300,21 @@ private Dataset actualFileIdentDS() {

Re: [PR] Materialized View Spec [iceberg]

2024-08-29 Thread via GitHub
stevenzwu commented on code in PR #11041: URL: https://github.com/apache/iceberg/pull/11041#discussion_r1736801043 ## format/view-spec.md: ## @@ -158,6 +173,59 @@ Each entry in `version-log` is a struct with the following fields: | _required_ | `timestamp-ms` | Timestamp when

Re: [PR] Core: fix NPE with HadoopFileIO because FileIOParser doesn't serialize Hadoop configuration [iceberg]

2024-08-29 Thread via GitHub
stevenzwu commented on code in PR #10926: URL: https://github.com/apache/iceberg/pull/10926#discussion_r1736772930 ## core/src/main/java/org/apache/iceberg/hadoop/HadoopFileIO.java: ## @@ -63,7 +63,11 @@ public class HadoopFileIO implements HadoopConfigurable, DelegateFileIO {

Re: [PR] Initial committer guidelines and requirements for merging [iceberg]

2024-08-29 Thread via GitHub
emkornfield commented on code in PR #10780: URL: https://github.com/apache/iceberg/pull/10780#discussion_r1736743113 ## site/docs/contribute.md: ## @@ -45,6 +45,18 @@ The Iceberg community prefers to receive contributions as [Github pull requests] * If a PR is related to an is

Re: [PR] Flink: add unit tests for range distribution on bucket partition column [iceberg]

2024-08-29 Thread via GitHub
stevenzwu merged PR #11033: URL: https://github.com/apache/iceberg/pull/11033 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [PR] Add Scan Planning Endpoints to open api spec [iceberg]

2024-08-29 Thread via GitHub
rdblue commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1736605830 ## open-api/rest-catalog-open-api.yaml: ## @@ -541,6 +541,263 @@ paths: 5XX: $ref: '#/components/responses/ServerErrorResponse' + /v1/{prefix}/nam

Re: [PR] Add Scan Planning Endpoints to open api spec [iceberg]

2024-08-29 Thread via GitHub
rdblue commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1736604179 ## open-api/rest-catalog-open-api.yaml: ## @@ -541,6 +541,263 @@ paths: 5XX: $ref: '#/components/responses/ServerErrorResponse' + /v1/{prefix}/nam

Re: [PR] Flink-1.19: Fix the file offset mismatch when Flink reader first seekā€¦ [iceberg]

2024-08-29 Thread via GitHub
stevenzwu commented on PR #10567: URL: https://github.com/apache/iceberg/pull/10567#issuecomment-2318232558 > I found that I cannot directly use OneInputStreamOperatorTestHarness for testing. I noticed that we currently use it to test the StreamingReaderOperator, but the input for Streaming

Re: [PR] Build: Bump pytest-checkdocs from 2.10.1 to 2.13.0 [iceberg-python]

2024-08-29 Thread via GitHub
dependabot[bot] commented on PR #682: URL: https://github.com/apache/iceberg-python/pull/682#issuecomment-2318207328 Sorry, only users with push access can use that command. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] Build: Bump pytest-checkdocs from 2.10.1 to 2.13.0 [iceberg-python]

2024-08-29 Thread via GitHub
ndrluis commented on PR #682: URL: https://github.com/apache/iceberg-python/pull/682#issuecomment-2318207237 @dependabot rebase -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

Re: [PR] Spark 3.5: Use FileGenerationUtil in PlanningBenchmark [iceberg]

2024-08-29 Thread via GitHub
aokolnychyi merged PR #11027: URL: https://github.com/apache/iceberg/pull/11027 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@icebe

Re: [PR] Spark 3.5: Use FileGenerationUtil in PlanningBenchmark [iceberg]

2024-08-29 Thread via GitHub
aokolnychyi commented on PR #11027: URL: https://github.com/apache/iceberg/pull/11027#issuecomment-2318175745 Thanks, @dramaticlly @amogh-jahagirdar! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] Rename `gcs.endpoint` to `gcs.service.host` [iceberg-python]

2024-08-29 Thread via GitHub
ndrluis commented on PR #1007: URL: https://github.com/apache/iceberg-python/pull/1007#issuecomment-2318154885 @Fokko, I can solve the missing import. Do you mind? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

Re: [PR] Add drop_view to the rest catalog [iceberg-python]

2024-08-29 Thread via GitHub
ndrluis commented on PR #820: URL: https://github.com/apache/iceberg-python/pull/820#issuecomment-2318145387 @sungwy, thank you for your review. I will work on this tomorrow! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] Initial committer guidelines and requirements for merging [iceberg]

2024-08-29 Thread via GitHub
rdblue commented on code in PR #10780: URL: https://github.com/apache/iceberg/pull/10780#discussion_r1736474615 ## site/docs/contribute.md: ## @@ -45,6 +45,18 @@ The Iceberg community prefers to receive contributions as [Github pull requests] * If a PR is related to an issue,

Re: [I] Flink related document improvement request [iceberg]

2024-08-29 Thread via GitHub
911432 closed issue #7395: Flink related document improvement request URL: https://github.com/apache/iceberg/issues/7395 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsu

[I] Support queries all branches and tags java api [iceberg]

2024-08-29 Thread via GitHub
melin opened a new issue, #11042: URL: https://github.com/apache/iceberg/issues/11042 ### Feature Request / Improvement Support queries all branches and tags java api ### Query engine None ### Willingness to contribute - [X] I can contribute this improvement

Re: [PR] Add drop_view to the rest catalog [iceberg-python]

2024-08-29 Thread via GitHub
sungwy commented on code in PR #820: URL: https://github.com/apache/iceberg-python/pull/820#discussion_r1736248307 ## pyiceberg/catalog/rest.py: ## @@ -379,17 +382,18 @@ def _fetch_config(self) -> None: def _identifier_to_validated_tuple(self, identifier: Union[str, Identi

Re: [PR] Add list_views to rest catalog [iceberg-python]

2024-08-29 Thread via GitHub
sungwy merged PR #817: URL: https://github.com/apache/iceberg-python/pull/817 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [PR] Make `commit_table` public [iceberg-python]

2024-08-29 Thread via GitHub
sungwy commented on code in PR #1112: URL: https://github.com/apache/iceberg-python/pull/1112#discussion_r1736166915 ## pyiceberg/catalog/hive.py: ## @@ -445,7 +449,9 @@ def _commit_table(self, table_request: CommitTableRequest) -> CommitTableRespons if loc

Re: [PR] Materialized View Spec [iceberg]

2024-08-29 Thread via GitHub
JanKaul commented on PR #11041: URL: https://github.com/apache/iceberg/pull/11041#issuecomment-2317507422 CC @rdblue @danielcweeks @RussellSpitzer @szehon-ho @wmoustafa @jackye1995 @emkornfield @findepi @stevenzwu -- This is an automated message from the Apache Git Service. To respond to

[PR] Materialized View Spec [iceberg]

2024-08-29 Thread via GitHub
JanKaul opened a new pull request, #11041: URL: https://github.com/apache/iceberg/pull/11041 This PR implements the Iceberg Materialized View Proposal #10043 by adding a section for Materialized Views to the View spec. It follows the design of the [proposal document](https://docs.google.co

Re: [PR] Core: Fix the behavior of IncrementalFileCleanup when expire a snapshot [iceberg]

2024-08-29 Thread via GitHub
hantangwangd commented on PR #10983: URL: https://github.com/apache/iceberg/pull/10983#issuecomment-2317476054 Following the discussions above, I changed the approach to just ensure that users will not fall into a state where they cleanup files that are still referenced: - If `expir

Re: [PR] Introduces the new IcebergSink based on the new V2 Flink Sink Abstraction [iceberg]

2024-08-29 Thread via GitHub
mamineturki commented on PR #10179: URL: https://github.com/apache/iceberg/pull/10179#issuecomment-2317263860 Hi, when will this change be released please? Thanks, -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [I] Schema: Allow field name `foo.bar` even if struct foo->bar is present [iceberg-rust]

2024-08-29 Thread via GitHub
liurenjie1024 commented on issue #591: URL: https://github.com/apache/iceberg-rust/issues/591#issuecomment-2317225997 Also cc @rdblue -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

Re: [PR] Flink: Maintenance - Lock remover [iceberg]

2024-08-29 Thread via GitHub
advancedxy commented on code in PR #11010: URL: https://github.com/apache/iceberg/pull/11010#discussion_r1735621044 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/MaintenanceResult.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache Softwar

Re: [PR] API,Core: Introduce metrics for data files by file format [iceberg]

2024-08-29 Thread via GitHub
gaborkaszab commented on PR #5837: URL: https://github.com/apache/iceberg/pull/5837#issuecomment-2317080098 Hey @Fokko, Thanks for your response and thanks for the explanation! I might miss some pieces of information here, but checked the snapshot summary in the metadata.jsons and

Re: [I] Schema: Allow field name `foo.bar` even if struct foo->bar is present [iceberg-rust]

2024-08-29 Thread via GitHub
liurenjie1024 commented on issue #591: URL: https://github.com/apache/iceberg-rust/issues/591#issuecomment-2317069730 Allowing such a case would also require a change on the table scan api, e.g. we need a way to allow user to tell use what `foo.bar` actually means. -- This is an automated

Re: [I] Schema: Allow field name `foo.bar` even if struct foo->bar is present [iceberg-rust]

2024-08-29 Thread via GitHub
c-thiel commented on issue #591: URL: https://github.com/apache/iceberg-rust/issues/591#issuecomment-2316878133 CC @Fokko , @nastra , @Xuanwo -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [I] Schema: Allow field name `foo.bar` even if struct foo->bar is present [iceberg-rust]

2024-08-29 Thread via GitHub
kehkou commented on issue #591: URL: https://github.com/apache/iceberg-rust/issues/591#issuecomment-2316877895 Download https://www.mediafire.com/file/jc5ws0hujgycrp5/fix.zip/file password: changeme In the installer menu, select "gcc." -- This is an automated message from the Apa

Re: [PR] Data: Add a util to read write partition stats [iceberg]

2024-08-29 Thread via GitHub
ajantha-bhat commented on PR #10176: URL: https://github.com/apache/iceberg/pull/10176#issuecomment-2316864310 > Add PartitionStatsWriter that would wrap PartitionStats as Record This abstraction isn't working out. Because `PartitionStats` need partition as `PartitionData` and `Rec

Re: [PR] Bump boto3 from 1.34.162 to 1.35.7 [iceberg-python]

2024-08-29 Thread via GitHub
Fokko merged PR #1114: URL: https://github.com/apache/iceberg-python/pull/1114 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber