Re: [PR] feat: Add equality delete writer [iceberg-rust]

2024-05-29 Thread via GitHub
Dysprosium0626 commented on PR #372: URL: https://github.com/apache/iceberg-rust/pull/372#issuecomment-2136687629 @ZENOTME @sdd PTAL! Thanks a lot! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Flink 1.19: Fix flaky `TestIcebergSourceFailover > testBoundedWithSavepoint` [iceberg]

2024-05-29 Thread via GitHub
pvary commented on PR #10370: URL: https://github.com/apache/iceberg/pull/10370#issuecomment-2136744665 Created #10393 with the `CountDownLatch` solution -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] Flink 1.19: Fix flaky TestIcebergSourceFailover > testBoundedWithSavepoint [iceberg]

2024-05-29 Thread via GitHub
pvary commented on PR #10393: URL: https://github.com/apache/iceberg/pull/10393#issuecomment-2136745703 CC: @manuzhang -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To u

Re: [PR] Flink 1.19: Fix flaky `TestIcebergSourceFailover > testBoundedWithSavepoint` [iceberg]

2024-05-29 Thread via GitHub
manuzhang closed pull request #10370: Flink 1.19: Fix flaky `TestIcebergSourceFailover > testBoundedWithSavepoint` URL: https://github.com/apache/iceberg/pull/10370 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] Flink 1.19: Fix flaky `TestIcebergSourceFailover > testBoundedWithSavepoint` [iceberg]

2024-05-29 Thread via GitHub
manuzhang commented on PR #10370: URL: https://github.com/apache/iceberg/pull/10370#issuecomment-2136807372 Superseded by #10393 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

[PR] feat: Support viewfs scheme along side with hdfs [iceberg-python]

2024-05-29 Thread via GitHub
yothinix opened a new pull request, #777: URL: https://github.com/apache/iceberg-python/pull/777 # Change proposed - Add support for metadata point to `viewfs` path in Hadoop/HDFS environment -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] build(deps): bump github.com/aws/aws-sdk-go-v2/config from 1.27.15 to 1.27.16 [iceberg-go]

2024-05-29 Thread via GitHub
Fokko merged PR #90: URL: https://github.com/apache/iceberg-go/pull/90 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache

Re: [PR] build(deps): bump github.com/aws/aws-sdk-go-v2/credentials from 1.17.15 to 1.17.16 [iceberg-go]

2024-05-29 Thread via GitHub
Fokko commented on PR #89: URL: https://github.com/apache/iceberg-go/pull/89#issuecomment-2137103247 @dependabot rebase -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To u

Re: [PR] Docs: add metrics-reporting back [iceberg]

2024-05-29 Thread via GitHub
manuzhang commented on PR #10377: URL: https://github.com/apache/iceberg/pull/10377#issuecomment-2137114187 @nastra Should this fix be deployed to nightly docs? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

Re: [PR] build(deps): bump github.com/aws/aws-sdk-go-v2/credentials from 1.17.15 to 1.17.16 [iceberg-go]

2024-05-29 Thread via GitHub
dependabot[bot] commented on PR #89: URL: https://github.com/apache/iceberg-go/pull/89#issuecomment-2137120308 Looks like github.com/aws/aws-sdk-go-v2/credentials is up-to-date now, so this is no longer needed. -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] build(deps): bump github.com/aws/aws-sdk-go-v2/credentials from 1.17.15 to 1.17.16 [iceberg-go]

2024-05-29 Thread via GitHub
dependabot[bot] closed pull request #89: build(deps): bump github.com/aws/aws-sdk-go-v2/credentials from 1.17.15 to 1.17.16 URL: https://github.com/apache/iceberg-go/pull/89 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] Add Files metadata table [iceberg-python]

2024-05-29 Thread via GitHub
Fokko commented on PR #614: URL: https://github.com/apache/iceberg-python/pull/614#issuecomment-2137130504 Sorry for now following up on this @Gowthami03B Could you rebase so we can get this in? Thanks! -- This is an automated message from the Apache Git Service. To respond to the message

Re: [PR] Support CreateTableTransaction for HiveCatalog [iceberg-python]

2024-05-29 Thread via GitHub
Fokko commented on code in PR #683: URL: https://github.com/apache/iceberg-python/pull/683#discussion_r1618684588 ## tests/integration/test_writes/test_writes.py: ## @@ -609,17 +610,18 @@ def test_write_and_evolve(session_catalog: Catalog, format_version: int) -> None @pyt

Re: [PR] Compare `Schema` and `StructType` fields irrespective of ordering [iceberg-python]

2024-05-29 Thread via GitHub
Fokko commented on code in PR #700: URL: https://github.com/apache/iceberg-python/pull/700#discussion_r1618699755 ## tests/integration/test_rest_schema.py: ## @@ -1730,19 +1730,17 @@ def test_move_nested_field_after_first(catalog: Catalog) -> None: with tbl.update_schema()

Re: [PR] Compare `Schema` and `StructType` fields irrespective of ordering [iceberg-python]

2024-05-29 Thread via GitHub
Fokko commented on code in PR #700: URL: https://github.com/apache/iceberg-python/pull/700#discussion_r1618699755 ## tests/integration/test_rest_schema.py: ## @@ -1730,19 +1730,17 @@ def test_move_nested_field_after_first(catalog: Catalog) -> None: with tbl.update_schema()

Re: [PR] Flink: refactor sink shuffling statistics collection [iceberg]

2024-05-29 Thread via GitHub
pvary commented on code in PR #10331: URL: https://github.com/apache/iceberg/pull/10331#discussion_r1618733028 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/AggregatedStatisticsTracker.java: ## @@ -30,71 +42,99 @@ * {@link AggregatedStatistics} recei

Re: [PR] Add a Spark procedure to collect NDV [iceberg]

2024-05-29 Thread via GitHub
rice668 commented on code in PR #6582: URL: https://github.com/apache/iceberg/pull/6582#discussion_r1618744991 ## spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/procedures/DistinctCountProcedure.java: ## @@ -0,0 +1,191 @@ +/* + * Licensed to the Apache Software Foundati

Re: [PR] Add a Spark procedure to collect NDV [iceberg]

2024-05-29 Thread via GitHub
rice668 commented on code in PR #6582: URL: https://github.com/apache/iceberg/pull/6582#discussion_r1618744991 ## spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/procedures/DistinctCountProcedure.java: ## @@ -0,0 +1,191 @@ +/* + * Licensed to the Apache Software Foundati

Re: [PR] Flink: refactor sink shuffling statistics collection [iceberg]

2024-05-29 Thread via GitHub
pvary commented on code in PR #10331: URL: https://github.com/apache/iceberg/pull/10331#discussion_r1618733028 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/AggregatedStatisticsTracker.java: ## @@ -30,71 +42,99 @@ * {@link AggregatedStatistics} recei

Re: [PR] Flink: refactor sink shuffling statistics collection [iceberg]

2024-05-29 Thread via GitHub
pvary commented on code in PR #10331: URL: https://github.com/apache/iceberg/pull/10331#discussion_r1618890956 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/SketchUtil.java: ## @@ -0,0 +1,148 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] Flink: refactor sink shuffling statistics collection [iceberg]

2024-05-29 Thread via GitHub
pvary commented on code in PR #10331: URL: https://github.com/apache/iceberg/pull/10331#discussion_r1618893892 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/SketchUtil.java: ## @@ -0,0 +1,148 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] Flink: refactor sink shuffling statistics collection [iceberg]

2024-05-29 Thread via GitHub
pvary commented on code in PR #10331: URL: https://github.com/apache/iceberg/pull/10331#discussion_r1618897759 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/StatisticsUtil.java: ## @@ -0,0 +1,101 @@ +/* + * Licensed to the Apache Software Foundation (A

Re: [PR] Flink 1.19: Fix flaky TestIcebergSourceFailover > testBoundedWithSavepoint [iceberg]

2024-05-29 Thread via GitHub
pvary commented on PR #10393: URL: https://github.com/apache/iceberg/pull/10393#issuecomment-2137479894 Seems, like it is working now. Run 3k test locally, and ~3*6 tests here. Needed to add the `SPLIT_FILE_OPEN_COST` to the `TestIcebergSourceFailoverWithWatermarkExtractor` as it was m

Re: [PR] Add a Spark procedure to collect NDV [iceberg]

2024-05-29 Thread via GitHub
huaxingao commented on code in PR #6582: URL: https://github.com/apache/iceberg/pull/6582#discussion_r1619092149 ## spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/procedures/DistinctCountProcedure.java: ## @@ -0,0 +1,191 @@ +/* + * Licensed to the Apache Software Founda

[PR] fix: make PrimitiveLiteral and Literal not be Ord [iceberg-rust]

2024-05-29 Thread via GitHub
ZENOTME opened a new pull request, #386: URL: https://github.com/apache/iceberg-rust/pull/386 This PR fix #378. 1. It makes PrimitiveLiteral and Literal not be PartialOrd, Ord. And only Datum PartialOrd. 2. After 1, it also changes the Manifest and ManifestList to use Datum in `lowe

Re: [PR] fix: make PrimitiveLiteral and Literal not be Ord [iceberg-rust]

2024-05-29 Thread via GitHub
ZENOTME commented on PR #386: URL: https://github.com/apache/iceberg-rust/pull/386#issuecomment-2137737297 cc @liurenjie1024 @Xuanwo @Fokko @marvinlanhenke -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

[PR] Deploy site on nightly docs change [iceberg]

2024-05-29 Thread via GitHub
manuzhang opened a new pull request, #10394: URL: https://github.com/apache/iceberg/pull/10394 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

Re: [PR] Docs: add metrics-reporting back [iceberg]

2024-05-29 Thread via GitHub
manuzhang commented on PR #10377: URL: https://github.com/apache/iceberg/pull/10377#issuecomment-2137743614 I created https://github.com/apache/iceberg/pull/10394 to deploy site on nightly docs changes. -- This is an automated message from the Apache Git Service. To respond to the message

Re: [PR] feat: support lower_bound&&upper_bound for parquet writer [iceberg-rust]

2024-05-29 Thread via GitHub
ZENOTME commented on PR #383: URL: https://github.com/apache/iceberg-rust/pull/383#issuecomment-2137745265 This PR is blocked by #386 now. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

Re: [PR] [Build] Add a script to execute revapi without gradle plugin [iceberg]

2024-05-29 Thread via GitHub
jbonofre commented on code in PR #10386: URL: https://github.com/apache/iceberg/pull/10386#discussion_r1619129932 ## build.gradle: ## @@ -139,35 +137,6 @@ subprojects { apply plugin: 'java-library' - if (project.name in REVAPI_PROJECTS) { -apply plugin: 'com.palantir

Re: [PR] [Build] Add a script to execute revapi without gradle plugin [iceberg]

2024-05-29 Thread via GitHub
jbonofre commented on code in PR #10386: URL: https://github.com/apache/iceberg/pull/10386#discussion_r1619131651 ## build.gradle: ## @@ -139,35 +137,6 @@ subprojects { apply plugin: 'java-library' - if (project.name in REVAPI_PROJECTS) { -apply plugin: 'com.palantir

Re: [PR] [Build] Add a script to execute revapi without gradle plugin [iceberg]

2024-05-29 Thread via GitHub
jbonofre commented on code in PR #10386: URL: https://github.com/apache/iceberg/pull/10386#discussion_r1619134828 ## dev/revapi: ## @@ -0,0 +1,81 @@ +#!/usr/bin/env bash + +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements.

[I] Iceberg REST Spec Extensions for Access Decision Exchange [iceberg]

2024-05-29 Thread via GitHub
jackye1995 opened a new issue, #10395: URL: https://github.com/apache/iceberg/issues/10395 ### Proposed Change Devlist Discussion: https://lists.apache.org/thread/4swop72zgcr8rrmwvb51rlk0vnb8joyz Data access control is a critical aspect for data warehouse and lake house system

Re: [PR] Flink 1.19: Fix flaky TestIcebergSourceFailover > testBoundedWithSavepoint [iceberg]

2024-05-29 Thread via GitHub
stevenzwu commented on code in PR #10393: URL: https://github.com/apache/iceberg/pull/10393#discussion_r1619139980 ## flink/v1.19/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceFailoverWithWatermarkExtractor.java: ## @@ -48,10 +52,17 @@ public class TestIc

Re: [PR] Spark 3.5: Only traverse ancestors of current snapshot when building changelog scan [iceberg]

2024-05-29 Thread via GitHub
manuzhang closed pull request #10252: Spark 3.5: Only traverse ancestors of current snapshot when building changelog scan URL: https://github.com/apache/iceberg/pull/10252 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

Re: [PR] Flink 1.19: Fix flaky TestIcebergSourceFailover > testBoundedWithSavepoint [iceberg]

2024-05-29 Thread via GitHub
pvary commented on code in PR #10393: URL: https://github.com/apache/iceberg/pull/10393#discussion_r1619213639 ## flink/v1.19/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceFailoverWithWatermarkExtractor.java: ## @@ -48,10 +52,17 @@ public class TestIceber

Re: [PR] Flink 1.19: Fix flaky TestIcebergSourceFailover > testBoundedWithSavepoint [iceberg]

2024-05-29 Thread via GitHub
pvary commented on code in PR #10393: URL: https://github.com/apache/iceberg/pull/10393#discussion_r1619214564 ## flink/v1.19/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceFailoverWithWatermarkExtractor.java: ## @@ -48,10 +52,17 @@ public class TestIceber

Re: [I] Javadoc issues [iceberg]

2024-05-29 Thread via GitHub
manuzhang commented on issue #10378: URL: https://github.com/apache/iceberg/issues/10378#issuecomment-2137889353 Submitted https://github.com/apache/iceberg/pull/10397 for No. 3. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] Flink 1.19: Fix flaky TestIcebergSourceFailover > testBoundedWithSavepoint [iceberg]

2024-05-29 Thread via GitHub
stevenzwu commented on code in PR #10393: URL: https://github.com/apache/iceberg/pull/10393#discussion_r1619230274 ## flink/v1.19/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceFailoverWithWatermarkExtractor.java: ## @@ -48,10 +52,17 @@ public class TestIc

Re: [PR] Flink 1.19: Fix flaky TestIcebergSourceFailover > testBoundedWithSavepoint [iceberg]

2024-05-29 Thread via GitHub
stevenzwu commented on code in PR #10393: URL: https://github.com/apache/iceberg/pull/10393#discussion_r1619233709 ## flink/v1.19/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceFailover.java: ## @@ -333,28 +336,28 @@ private static void restartTaskManager(R

Re: [PR] Flink 1.19: Fix flaky TestIcebergSourceFailover > testBoundedWithSavepoint [iceberg]

2024-05-29 Thread via GitHub
stevenzwu commented on code in PR #10393: URL: https://github.com/apache/iceberg/pull/10393#discussion_r1619233709 ## flink/v1.19/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceFailover.java: ## @@ -333,28 +336,28 @@ private static void restartTaskManager(R

Re: [PR] HADOOP-18679. Add API for bulk/paged object deletion: Iceberg PoC [iceberg]

2024-05-29 Thread via GitHub
steveloughran commented on PR #10233: URL: https://github.com/apache/iceberg/pull/10233#issuecomment-2137940386 This PR is in sync with https://github.com/apache/hadoop/pull/6686 ; the dynamic binding classes in here are from that PR (which also copies in the parquet DynMethods classes for

Re: [PR] Spark Action to Analyze table [iceberg]

2024-05-29 Thread via GitHub
amogh-jahagirdar commented on code in PR #10288: URL: https://github.com/apache/iceberg/pull/10288#discussion_r1619229462 ## api/src/main/java/org/apache/iceberg/actions/AnalyzeTable.java: ## @@ -0,0 +1,58 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + *

Re: [PR] Spark 3.5: Only traverse ancestors of current snapshot when building changelog scan [iceberg]

2024-05-29 Thread via GitHub
flyrain commented on PR #10252: URL: https://github.com/apache/iceberg/pull/10252#issuecomment-2138049218 Other PRs also hit the same issue, https://github.com/apache/iceberg/pull/10396 -- This is an automated message from the Apache Git Service. To respond to the message, please log on t

Re: [PR] Flink 1.19: Fix flaky TestIcebergSourceFailover > testBoundedWithSavepoint [iceberg]

2024-05-29 Thread via GitHub
pvary commented on code in PR #10393: URL: https://github.com/apache/iceberg/pull/10393#discussion_r1619334747 ## flink/v1.19/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceFailover.java: ## @@ -333,28 +336,28 @@ private static void restartTaskManager(Runna

Re: [PR] Flink 1.19: Fix flaky TestIcebergSourceFailover > testBoundedWithSavepoint [iceberg]

2024-05-29 Thread via GitHub
pvary commented on code in PR #10393: URL: https://github.com/apache/iceberg/pull/10393#discussion_r1619348996 ## flink/v1.19/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceFailover.java: ## @@ -333,28 +336,28 @@ private static void restartTaskManager(Runna

[PR] Remove tokens-endpoint from REST spec [iceberg]

2024-05-29 Thread via GitHub
snazy opened a new pull request, #10398: URL: https://github.com/apache/iceberg/pull/10398 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-ma

Re: [PR] Introduces the new IcebergSink based on the new V2 Flink Sink Abstraction [iceberg]

2024-05-29 Thread via GitHub
rodmeneses commented on code in PR #10179: URL: https://github.com/apache/iceberg/pull/10179#discussion_r1619378283 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/sink/IcebergSink.java: ## @@ -0,0 +1,780 @@ +/* + * Licensed to the Apache Software Foundation (ASF) un

Re: [PR] Introduces the new IcebergSink based on the new V2 Flink Sink Abstraction [iceberg]

2024-05-29 Thread via GitHub
stevenzwu commented on code in PR #10179: URL: https://github.com/apache/iceberg/pull/10179#discussion_r1619419797 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/sink/IcebergSink.java: ## @@ -0,0 +1,780 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

Re: [PR] Introduces the new IcebergSink based on the new V2 Flink Sink Abstraction [iceberg]

2024-05-29 Thread via GitHub
stevenzwu commented on code in PR #10179: URL: https://github.com/apache/iceberg/pull/10179#discussion_r1619421059 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/sink/IcebergSink.java: ## @@ -0,0 +1,780 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

Re: [PR] Introduces the new IcebergSink based on the new V2 Flink Sink Abstraction [iceberg]

2024-05-29 Thread via GitHub
rodmeneses commented on code in PR #10179: URL: https://github.com/apache/iceberg/pull/10179#discussion_r1619495787 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/sink/committer/SinkCommitter.java: ## @@ -0,0 +1,463 @@ +/* + * Licensed to the Apache Software Foundat

Re: [PR] Introduces the new IcebergSink based on the new V2 Flink Sink Abstraction [iceberg]

2024-05-29 Thread via GitHub
rodmeneses commented on code in PR #10179: URL: https://github.com/apache/iceberg/pull/10179#discussion_r1619505665 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/sink/committer/SinkCommitter.java: ## @@ -0,0 +1,463 @@ +/* + * Licensed to the Apache Software Foundat

[PR] Bump requests from 2.32.2 to 2.32.3 [iceberg-python]

2024-05-29 Thread via GitHub
dependabot[bot] opened a new pull request, #778: URL: https://github.com/apache/iceberg-python/pull/778 Bumps [requests](https://github.com/psf/requests) from 2.32.2 to 2.32.3. Release notes Sourced from https://github.com/psf/requests/releases";>requests's releases. v2.32.3

[PR] Bump getdaft from 0.2.24 to 0.2.25 [iceberg-python]

2024-05-29 Thread via GitHub
dependabot[bot] opened a new pull request, #779: URL: https://github.com/apache/iceberg-python/pull/779 Bumps [getdaft](https://github.com/Eventual-Inc/Daft) from 0.2.24 to 0.2.25. Release notes Sourced from https://github.com/Eventual-Inc/Daft/releases";>getdaft's releases.

[PR] Bump mypy-boto3-glue from 1.34.110 to 1.34.115 [iceberg-python]

2024-05-29 Thread via GitHub
dependabot[bot] opened a new pull request, #780: URL: https://github.com/apache/iceberg-python/pull/780 Bumps [mypy-boto3-glue](https://github.com/youtype/mypy_boto3_builder) from 1.34.110 to 1.34.115. Commits See full diff in https://github.com/youtype/mypy_boto3_builder/commi

Re: [PR] Flink 1.19: Fix flaky TestIcebergSourceFailover > testBoundedWithSavepoint [iceberg]

2024-05-29 Thread via GitHub
stevenzwu commented on code in PR #10393: URL: https://github.com/apache/iceberg/pull/10393#discussion_r1619547088 ## flink/v1.19/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceFailover.java: ## @@ -333,28 +336,28 @@ private static void restartTaskManager(R

Re: [PR] Flink 1.19: Fix flaky TestIcebergSourceFailover > testBoundedWithSavepoint [iceberg]

2024-05-29 Thread via GitHub
stevenzwu commented on code in PR #10393: URL: https://github.com/apache/iceberg/pull/10393#discussion_r1619548308 ## flink/v1.19/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceFailover.java: ## @@ -333,28 +336,28 @@ private static void restartTaskManager(R

Re: [I] BUG: Bug: partition name stored in partition data in data file contains special character [iceberg-python]

2024-05-29 Thread via GitHub
github-actions[bot] commented on issue #175: URL: https://github.com/apache/iceberg-python/issues/175#issuecomment-2138447601 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity oc

[PR] [draft] Parquet page skipping (vectorized and non-vectorized) [iceberg]

2024-05-29 Thread via GitHub
wypoon opened a new pull request, #10399: URL: https://github.com/apache/iceberg/pull/10399 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-m

Re: [PR] Core: Make RESTMetricsReporter public [iceberg]

2024-05-29 Thread via GitHub
manuzhang commented on PR #10397: URL: https://github.com/apache/iceberg/pull/10397#issuecomment-2138571831 cc @nastra @jbonofre -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

Re: [PR] Add Files metadata table [iceberg-python]

2024-05-29 Thread via GitHub
amogh-jahagirdar commented on code in PR #614: URL: https://github.com/apache/iceberg-python/pull/614#discussion_r1619789557 ## pyiceberg/table/__init__.py: ## @@ -3537,6 +3537,106 @@ def update_partitions_map( schema=table_schema, ) +def files(self,

Re: [PR] Bump requests from 2.32.2 to 2.32.3 [iceberg-python]

2024-05-29 Thread via GitHub
Fokko merged PR #778: URL: https://github.com/apache/iceberg-python/pull/778 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] Bump getdaft from 0.2.24 to 0.2.25 [iceberg-python]

2024-05-29 Thread via GitHub
Fokko merged PR #779: URL: https://github.com/apache/iceberg-python/pull/779 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] Test GitHub actions rate limit [iceberg]

2024-05-29 Thread via GitHub
manuzhang closed pull request #10400: Test GitHub actions rate limit URL: https://github.com/apache/iceberg/pull/10400 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubs

Re: [PR] Flink 1.19: Fix flaky TestIcebergSourceFailover > testBoundedWithSavepoint [iceberg]

2024-05-29 Thread via GitHub
pvary commented on code in PR #10393: URL: https://github.com/apache/iceberg/pull/10393#discussion_r1619876999 ## flink/v1.19/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceFailover.java: ## @@ -333,28 +336,28 @@ private static void restartTaskManager(Runna

Re: [PR] Flink 1.19: Fix flaky TestIcebergSourceFailover > testBoundedWithSavepoint [iceberg]

2024-05-29 Thread via GitHub
pvary commented on code in PR #10393: URL: https://github.com/apache/iceberg/pull/10393#discussion_r1619909229 ## flink/v1.19/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceFailover.java: ## @@ -333,28 +336,28 @@ private static void restartTaskManager(Runna

Re: [PR] Spark 3.5: Only traverse ancestors of current snapshot when building changelog scan [iceberg]

2024-05-29 Thread via GitHub
flyrain merged PR #10252: URL: https://github.com/apache/iceberg/pull/10252 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.a

Re: [I] Spark: CDC does not respect when the table is rolled back. [iceberg]

2024-05-29 Thread via GitHub
flyrain closed issue #10247: Spark: CDC does not respect when the table is rolled back. URL: https://github.com/apache/iceberg/issues/10247 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

Re: [PR] Spark 3.5: Only traverse ancestors of current snapshot when building changelog scan [iceberg]

2024-05-29 Thread via GitHub
flyrain commented on PR #10252: URL: https://github.com/apache/iceberg/pull/10252#issuecomment-2138767152 Thanks @manuzhang for working on it. We can also backport it to Spark 3.4. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

Re: [PR] Core: Make RESTMetricsReporter public [iceberg]

2024-05-29 Thread via GitHub
Fokko commented on PR #10397: URL: https://github.com/apache/iceberg/pull/10397#issuecomment-2138780007 @manuzhang How I read the doc is: - You either enable the `RESTMetricsReporter` through the property, no need to override the class. - You implement a custom reporter by overridi

Re: [PR] Core: Make RESTMetricsReporter public [iceberg]

2024-05-29 Thread via GitHub
Fokko commented on PR #10397: URL: https://github.com/apache/iceberg/pull/10397#issuecomment-2138788283 @manuzhang I see your comment in #10378. I don't think that the class being private is the underlying issue of the missing javadoc. For example, the RestCatalog doesn't have docs either:

Re: [PR] Core: Make RESTMetricsReporter public [iceberg]

2024-05-29 Thread via GitHub
Fokko commented on PR #10397: URL: https://github.com/apache/iceberg/pull/10397#issuecomment-2138792529 Hmm, the file is missing here: https://github.com/apache/iceberg/tree/javadoc/1.5.2/org/apache/iceberg/rest -- This is an automated message from the Apache Git Service. To respond to th

Re: [PR] Core: Make RESTMetricsReporter public [iceberg]

2024-05-29 Thread via GitHub
manuzhang commented on PR #10397: URL: https://github.com/apache/iceberg/pull/10397#issuecomment-2138795012 RESTCatalog has javadoc at https://github.com/apache/iceberg/blob/javadoc/1.5.2/org/apache/iceberg/rest/RESTCatalog.html. It looks another issue when deploying to website. -- This

Re: [PR] Core: Make RESTMetricsReporter public [iceberg]

2024-05-29 Thread via GitHub
manuzhang commented on PR #10397: URL: https://github.com/apache/iceberg/pull/10397#issuecomment-2138796365 Default level of generating javadoc is public and protected. `RESTMetricsReporter` is package private so its javadoc is not generated. -- This is an automated message from the Apach