Re: [PR] Core: Support commits with DVs [iceberg]

2024-11-09 Thread via GitHub
aokolnychyi commented on code in PR #11495: URL: https://github.com/apache/iceberg/pull/11495#discussion_r1835614940 ## core/src/main/java/org/apache/iceberg/BaseRowDelta.java: ## @@ -139,6 +139,10 @@ protected void validate(TableMetadata base, Snapshot parent) { if (val

Re: [PR] Core: Support commits with DVs [iceberg]

2024-11-09 Thread via GitHub
aokolnychyi commented on code in PR #11495: URL: https://github.com/apache/iceberg/pull/11495#discussion_r1835614576 ## core/src/main/java/org/apache/iceberg/MergingSnapshotProducer.java: ## @@ -769,6 +794,82 @@ protected void validateDataFilesExist( } } + // validate

Re: [PR] Core: Support commits with DVs [iceberg]

2024-11-09 Thread via GitHub
aokolnychyi commented on code in PR #11495: URL: https://github.com/apache/iceberg/pull/11495#discussion_r1835614371 ## core/src/main/java/org/apache/iceberg/MergingSnapshotProducer.java: ## @@ -769,6 +794,82 @@ protected void validateDataFilesExist( } } + // validate

Re: [PR] Core: Support commits with DVs [iceberg]

2024-11-09 Thread via GitHub
aokolnychyi commented on code in PR #11495: URL: https://github.com/apache/iceberg/pull/11495#discussion_r1835613879 ## core/src/main/java/org/apache/iceberg/MergingSnapshotProducer.java: ## @@ -245,13 +250,13 @@ private PartitionSpec spec(int specId) { /** Add a delete fil

Re: [PR] Core: Support commits with DVs [iceberg]

2024-11-09 Thread via GitHub
aokolnychyi commented on code in PR #11495: URL: https://github.com/apache/iceberg/pull/11495#discussion_r1835613811 ## core/src/main/java/org/apache/iceberg/MergingSnapshotProducer.java: ## @@ -271,6 +276,26 @@ private void add(PendingDeleteFile file) { } } + protect

[PR] Build: Bump mkdocs-material from 9.5.43 to 9.5.44 [iceberg]

2024-11-09 Thread via GitHub
dependabot[bot] opened a new pull request, #11510: URL: https://github.com/apache/iceberg/pull/11510 Bumps [mkdocs-material](https://github.com/squidfunk/mkdocs-material) from 9.5.43 to 9.5.44. Release notes Sourced from https://github.com/squidfunk/mkdocs-material/releases";>mkdoc

[PR] Build: Bump mkdocs-redirects from 1.2.1 to 1.2.2 [iceberg]

2024-11-09 Thread via GitHub
dependabot[bot] opened a new pull request, #11511: URL: https://github.com/apache/iceberg/pull/11511 Bumps [mkdocs-redirects](https://github.com/mkdocs/mkdocs-redirects) from 1.2.1 to 1.2.2. Release notes Sourced from https://github.com/mkdocs/mkdocs-redirects/releases";>mkdocs-red

[PR] Build: Bump software.amazon.awssdk:bom from 2.29.6 to 2.29.9 [iceberg]

2024-11-09 Thread via GitHub
dependabot[bot] opened a new pull request, #11509: URL: https://github.com/apache/iceberg/pull/11509 Bumps software.amazon.awssdk:bom from 2.29.6 to 2.29.9. [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=softwa

[PR] Build: Bump kafka from 3.8.1 to 3.9.0 [iceberg]

2024-11-09 Thread via GitHub
dependabot[bot] opened a new pull request, #11508: URL: https://github.com/apache/iceberg/pull/11508 Bumps `kafka` from 3.8.1 to 3.9.0. Updates `org.apache.kafka:kafka-clients` from 3.8.1 to 3.9.0 Updates `org.apache.kafka:connect-api` from 3.8.1 to 3.9.0 Updates `org.apache.

[PR] Build: Bump parquet from 1.13.1 to 1.14.3 [iceberg]

2024-11-09 Thread via GitHub
dependabot[bot] opened a new pull request, #11507: URL: https://github.com/apache/iceberg/pull/11507 Bumps `parquet` from 1.13.1 to 1.14.3. Updates `org.apache.parquet:parquet-avro` from 1.13.1 to 1.14.3 Release notes Sourced from https://github.com/apache/parquet-mr/releases";>o

Re: [PR] core: support support move a column with same name after rename column [iceberg]

2024-11-09 Thread via GitHub
FANNG1 commented on PR #10862: URL: https://github.com/apache/iceberg/pull/10862#issuecomment-2466567111 @RussellSpitzer @Fokko should we continue work on this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

Re: [PR] API: Define RepairManifests action interface [iceberg]

2024-11-09 Thread via GitHub
github-actions[bot] commented on PR #10784: URL: https://github.com/apache/iceberg/pull/10784#issuecomment-2466519748 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [I] Improvements to Iceberg Catalog Descriptions [iceberg]

2024-11-09 Thread via GitHub
github-actions[bot] commented on issue #10316: URL: https://github.com/apache/iceberg/issues/10316#issuecomment-2466519614 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [PR] Fence zombie coordinators [iceberg]

2024-11-09 Thread via GitHub
github-actions[bot] commented on PR #10812: URL: https://github.com/apache/iceberg/pull/10812#issuecomment-2466519764 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] Core: Upgrade Jetty to 12.0.12 [iceberg]

2024-11-09 Thread via GitHub
github-actions[bot] commented on PR #10837: URL: https://github.com/apache/iceberg/pull/10837#issuecomment-2466519784 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] Docs: Fix Typos and Markdown Formats in Documentation Files [iceberg]

2024-11-09 Thread via GitHub
github-actions[bot] commented on PR #10884: URL: https://github.com/apache/iceberg/pull/10884#issuecomment-2466519805 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] Docs: Clarify wording on releases [iceberg]

2024-11-09 Thread via GitHub
github-actions[bot] commented on PR #10806: URL: https://github.com/apache/iceberg/pull/10806#issuecomment-2466519755 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [I] Using subdirectory to dave data in ICEBERG. [iceberg]

2024-11-09 Thread via GitHub
github-actions[bot] commented on issue #10327: URL: https://github.com/apache/iceberg/issues/10327#issuecomment-2466519625 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [PR] Add row-level operation benchmarks [iceberg]

2024-11-09 Thread via GitHub
github-actions[bot] commented on PR #10687: URL: https://github.com/apache/iceberg/pull/10687#issuecomment-2466519696 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] Api, Core: Set upper boundary for the type parameter of `Scan` [iceberg]

2024-11-09 Thread via GitHub
github-actions[bot] commented on PR #10743: URL: https://github.com/apache/iceberg/pull/10743#issuecomment-2466519731 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] [Flink] the precondition judgment is incorrect in the FlinkSink class. [iceberg]

2024-11-09 Thread via GitHub
DaqianLiao commented on PR #10669: URL: https://github.com/apache/iceberg/pull/10669#issuecomment-2466519849 您好,您的邮件以收到,谢谢!未及时回复,请见谅。我会尽快处理。 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

Re: [PR] Hive: Pre-check namespace presence for listTables [iceberg]

2024-11-09 Thread via GitHub
github-actions[bot] commented on PR #10845: URL: https://github.com/apache/iceberg/pull/10845#issuecomment-2466519789 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] Docs: Add Databend docs url to sidebar [iceberg]

2024-11-09 Thread via GitHub
github-actions[bot] commented on PR #10834: URL: https://github.com/apache/iceberg/pull/10834#issuecomment-2466519776 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [I] read from Iceberg table throw java.lang.ArrayIndexOutOfBoundsException: 3 [iceberg]

2024-11-09 Thread via GitHub
github-actions[bot] commented on issue #10103: URL: https://github.com/apache/iceberg/issues/10103#issuecomment-2466519578 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache

Re: [PR] Add `IcebergAnalysisException` in iceberg-spark module [iceberg]

2024-11-09 Thread via GitHub
github-actions[bot] commented on PR #10766: URL: https://github.com/apache/iceberg/pull/10766#issuecomment-2466519741 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] core: support support move a column with same name after rename column [iceberg]

2024-11-09 Thread via GitHub
github-actions[bot] commented on PR #10862: URL: https://github.com/apache/iceberg/pull/10862#issuecomment-2466519793 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] Build: verify `gradle-wrapper jar` integrity [iceberg]

2024-11-09 Thread via GitHub
github-actions[bot] commented on PR #10719: URL: https://github.com/apache/iceberg/pull/10719#issuecomment-2466519703 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] Docs: Add a clarification in the spec regarding full name restrictions [iceberg]

2024-11-09 Thread via GitHub
github-actions[bot] commented on PR #10883: URL: https://github.com/apache/iceberg/pull/10883#issuecomment-2466519797 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] [Flink] the precondition judgment is incorrect in the FlinkSink class. [iceberg]

2024-11-09 Thread via GitHub
github-actions[bot] commented on PR #10669: URL: https://github.com/apache/iceberg/pull/10669#issuecomment-2466519689 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] API: Update Conversions.fromPartitionString API to handle timestamps [iceberg]

2024-11-09 Thread via GitHub
github-actions[bot] commented on PR #10820: URL: https://github.com/apache/iceberg/pull/10820#issuecomment-2466519773 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] add: test case for merge schema [iceberg]

2024-11-09 Thread via GitHub
github-actions[bot] commented on PR #10816: URL: https://github.com/apache/iceberg/pull/10816#issuecomment-2466519768 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [I] catalog issue [iceberg]

2024-11-09 Thread via GitHub
github-actions[bot] commented on issue #10324: URL: https://github.com/apache/iceberg/issues/10324#issuecomment-2466519619 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [PR] Add RocksDBStructLikeSet for storing equality deletes [iceberg]

2024-11-09 Thread via GitHub
github-actions[bot] commented on PR #10667: URL: https://github.com/apache/iceberg/pull/10667#issuecomment-2466519681 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] AddPartitionSpec: A new way to set new partition specs [iceberg]

2024-11-09 Thread via GitHub
github-actions[bot] commented on PR #10737: URL: https://github.com/apache/iceberg/pull/10737#issuecomment-2466519723 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] Core: Add DataFiles builder API to enable users to specify their own custom conversion logic for string partition values [iceberg]

2024-11-09 Thread via GitHub
github-actions[bot] commented on PR #10724: URL: https://github.com/apache/iceberg/pull/10724#issuecomment-2466519713 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [I] Add a table API to compute partition stats. [iceberg]

2024-11-09 Thread via GitHub
github-actions[bot] commented on issue #10105: URL: https://github.com/apache/iceberg/issues/10105#issuecomment-2466519589 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache

Re: [I] Add a table API to compute partition stats. [iceberg]

2024-11-09 Thread via GitHub
github-actions[bot] closed issue #10105: Add a table API to compute partition stats. URL: https://github.com/apache/iceberg/issues/10105 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [I] read from Iceberg table throw java.lang.ArrayIndexOutOfBoundsException: 3 [iceberg]

2024-11-09 Thread via GitHub
github-actions[bot] closed issue #10103: read from Iceberg table throw java.lang.ArrayIndexOutOfBoundsException: 3 URL: https://github.com/apache/iceberg/issues/10103 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

Re: [PR] Ignore schema merge updates from long -> int [iceberg]

2024-11-09 Thread via GitHub
rocco408 commented on PR #11419: URL: https://github.com/apache/iceberg/pull/11419#issuecomment-2466513581 This latest change to using TypeUtil.isPromotionAllowed() is failing a number of tests in TestSchemaUnionByFieldName. Further attempts to modify TypeUtil.isPromotionAllowed() and it's

Re: [PR] Boto Glue standard retry policy with configuration [iceberg-python]

2024-11-09 Thread via GitHub
mark-major commented on code in PR #1307: URL: https://github.com/apache/iceberg-python/pull/1307#discussion_r1835532735 ## pyiceberg/catalog/glue.py: ## @@ -305,7 +308,18 @@ def __init__(self, name: str, **properties: Any): aws_secret_access_key=get_first_property_

Re: [I] The source artifacts provided with GitHub releases are not buildable. [iceberg]

2024-11-09 Thread via GitHub
Fokko commented on issue #11505: URL: https://github.com/apache/iceberg/issues/11505#issuecomment-2466469594 Hey @AnatolyPopov Thanks for raising this 🙏 Where are you getting the release from? The official channel is through Apache: https://dlcdn.apache.org/iceberg/apache-iceberg-1.

Re: [I] Is `pyiceberg.Table` thread-safe? [iceberg-python]

2024-11-09 Thread via GitHub
kevinjqliu commented on issue #1305: URL: https://github.com/apache/iceberg-python/issues/1305#issuecomment-2466466411 > pyiceberg.exceptions.CommitFailedException This is not a thread issue. This is expected when you have concurrent writers. The writer should retry the commit. See #

Re: [PR] Test out Apache Parquet 1.14.4 RC2 [iceberg]

2024-11-09 Thread via GitHub
Fokko commented on code in PR #11502: URL: https://github.com/apache/iceberg/pull/11502#discussion_r1835516115 ## flink/v1.18/flink/src/test/java/org/apache/iceberg/flink/source/TestMetadataTableReadableMetrics.java: ## @@ -217,27 +217,27 @@ public void testPrimitiveColumns() th

Re: [PR] Test out Apache Parquet 1.14.4 RC2 [iceberg]

2024-11-09 Thread via GitHub
Fokko commented on code in PR #11502: URL: https://github.com/apache/iceberg/pull/11502#discussion_r1835516192 ## flink/v1.18/flink/src/test/java/org/apache/iceberg/flink/source/TestMetadataTableReadableMetrics.java: ## @@ -217,27 +217,27 @@ public void testPrimitiveColumns() th

Re: [PR] Boto Glue standard retry policy with configuration [iceberg-python]

2024-11-09 Thread via GitHub
kevinjqliu commented on code in PR #1307: URL: https://github.com/apache/iceberg-python/pull/1307#discussion_r1835489821 ## pyiceberg/catalog/__init__.py: ## @@ -94,6 +94,7 @@ URI = "uri" LOCATION = "location" EXTERNAL_TABLE = "EXTERNAL_TABLE" +MAX_RETRIES = 10 Review Commen

Re: [PR] Fix imports [iceberg-python]

2024-11-09 Thread via GitHub
vasco-lopo closed pull request #1309: Fix imports URL: https://github.com/apache/iceberg-python/pull/1309 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail

Re: [I] [bug] read from multiple s3 regions [iceberg-python]

2024-11-09 Thread via GitHub
kevinjqliu commented on issue #1279: URL: https://github.com/apache/iceberg-python/issues/1279#issuecomment-2466341561 @danhphan assigned to you! LMK if you have any questions -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

[I] Building from a release tag creates a "-SNAPSHOT" artifact with the version of next release. [iceberg]

2024-11-09 Thread via GitHub
AnatolyPopov opened a new issue, #11506: URL: https://github.com/apache/iceberg/issues/11506 ### Apache Iceberg version 1.7.0 (latest release) ### Query engine None ### Please describe the bug 🐞 When I'm trying to build the project from latest release tag I

[I] The artifacts provided with GitHub releases are not buildable. [iceberg]

2024-11-09 Thread via GitHub
AnatolyPopov opened a new issue, #11505: URL: https://github.com/apache/iceberg/issues/11505 ### Apache Iceberg version 1.7.0 (latest release) ### Query engine None ### Please describe the bug 🐞 The artifacts that are published with GitHub releases are not b

[PR] Support WASB scheme in ADLSFileIO [iceberg]

2024-11-09 Thread via GitHub
mrcnc opened a new pull request, #11504: URL: https://github.com/apache/iceberg/pull/11504 This is a second attempt to resolve https://github.com/apache/iceberg/issues/10127 but avoids using `java.net.URI` for parsing, which was found to be problematic after the first attempt was merged.

Re: [PR] Remove Hive 2 [iceberg]

2024-11-09 Thread via GitHub
manuzhang commented on PR #10996: URL: https://github.com/apache/iceberg/pull/10996#issuecomment-2466177889 @nastra This is ready for review now after 1.7.0 release. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[PR] Docs: Update multi-engine support after 1.7.0 release [iceberg]

2024-11-09 Thread via GitHub
manuzhang opened a new pull request, #11503: URL: https://github.com/apache/iceberg/pull/11503 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

Re: [PR] Core: Support commits with DVs [iceberg]

2024-11-09 Thread via GitHub
jbonofre commented on code in PR #11495: URL: https://github.com/apache/iceberg/pull/11495#discussion_r1835329015 ## core/src/main/java/org/apache/iceberg/BaseRowDelta.java: ## @@ -139,6 +139,10 @@ protected void validate(TableMetadata base, Snapshot parent) { if (valida

Re: [PR] Test out Apache Parquet 1.14.4 RC2 [iceberg]

2024-11-09 Thread via GitHub
jbonofre commented on code in PR #11502: URL: https://github.com/apache/iceberg/pull/11502#discussion_r1835328636 ## flink/v1.18/flink/src/test/java/org/apache/iceberg/flink/source/TestMetadataTableReadableMetrics.java: ## @@ -217,27 +217,27 @@ public void testPrimitiveColumns()