Re: [PR] Spark: Use Awaitility instead of Thread.sleep [iceberg]

nk1506 commented on PR #9224: URL: https://github.com/apache/iceberg/pull/9224#issuecomment-1844833641 > Hi, @nk1506 It seems that my code can't pass the unit tests. Is it because of the default waiting time is not enough? Do you have any ideas? Please refer [here](https://github.com

[I] When will the 0.6.0 version be released? [iceberg-python]

1taoze opened a new issue, #192: URL: https://github.com/apache/iceberg-python/issues/192 ### Feature Request / Improvement I am looking forward to the writing feature that will be released in version 0.6.0, and I would like to know approximately when it will be released. -- This i

Re: [PR] Hive: Add View support for HIVE catalog [iceberg]

nk1506 commented on code in PR #8907: URL: https://github.com/apache/iceberg/pull/8907#discussion_r1418491048 ## hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java: ## @@ -264,6 +269,162 @@ public void renameTable(TableIdentifier from, TableIdentifier origina

Re: [PR] Api: Track partition statistics via TableMetadata [iceberg]

aokolnychyi commented on code in PR #8502: URL: https://github.com/apache/iceberg/pull/8502#discussion_r1412672873 ## .palantir/revapi.yml: ## @@ -873,6 +873,10 @@ acceptedBreaks: new: "method void org.apache.iceberg.encryption.Ciphers::()" justification: "Static u

Re: [PR] Flink: switch to use SortKey for data statistics [iceberg]

pvary commented on code in PR #9212: URL: https://github.com/apache/iceberg/pull/9212#discussion_r1418460467 ## flink/v1.17/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceWithWatermarkExtractor.java: ## @@ -92,7 +93,10 @@ public class TestIcebergSourceWithW

Re: [PR] Docs: Document reading in Spark using branch and tag identifiers [iceberg]

wypoon commented on PR #9238: URL: https://github.com/apache/iceberg/pull/9238#issuecomment-1844516462 @rdblue @jackye1995 @amogh-jahagirdar what do you think? @nastra I added a note that the identifiers may not be combined with `VERSION AS OF` since you disallowed it in https://github.c

[PR] Docs: Document reading in Spark using branch and tag identifiers [iceberg]

wypoon opened a new pull request, #9238: URL: https://github.com/apache/iceberg/pull/9238 Writing to a branch or tag is supported in Spark using a table identifier with `branch_` or `tag_`. Reading from the branch or tag is supported using the same kind of identifier, in addition to the `VE

Re: [PR] Flink: Create JUnit5 version of TestFlinkScan [iceberg]

cgpoh commented on code in PR #9185: URL: https://github.com/apache/iceberg/pull/9185#discussion_r1418251955 ## flink/v1.17/flink/src/test/java/org/apache/iceberg/flink/TestHelpers.java: ## @@ -483,7 +473,7 @@ private static void assertArrayValues( private static void asser

Re: [PR] Flink: Create JUnit5 version of TestFlinkScan [iceberg]

cgpoh commented on code in PR #9185: URL: https://github.com/apache/iceberg/pull/9185#discussion_r1418251447 ## flink/v1.17/flink/src/test/java/org/apache/iceberg/flink/TestHelpers.java: ## @@ -523,89 +513,102 @@ public static void assertEquals(ManifestFile expected, ManifestFi

Re: [PR] Flink: Create JUnit5 version of TestFlinkScan [iceberg]

cgpoh commented on code in PR #9185: URL: https://github.com/apache/iceberg/pull/9185#discussion_r1418227381 ## data/src/test/java/org/apache/iceberg/data/GenAppenderHelper.java: ## @@ -0,0 +1,144 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more

Re: [PR] chore: Add cargo build and build guide [iceberg-rust]

manuzhang commented on PR #111: URL: https://github.com/apache/iceberg-rust/pull/111#issuecomment-1843994789 @Fokko any more comments? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] Core: Add PartitionMap [iceberg]

aokolnychyi commented on PR #9194: URL: https://github.com/apache/iceberg/pull/9194#issuecomment-1843990810 Thanks for reviewing, @qqqttt123 @jerqi @RussellSpitzer! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

Re: [PR] Core: Add PartitionMap [iceberg]

aokolnychyi merged PR #9194: URL: https://github.com/apache/iceberg/pull/9194 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [PR] Core: Add PartitionMap [iceberg]

aokolnychyi commented on code in PR #9194: URL: https://github.com/apache/iceberg/pull/9194#discussion_r1418166520 ## core/src/test/java/org/apache/iceberg/util/TestPartitionMap.java: ## @@ -0,0 +1,222 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or

Re: [PR] Core: Add PartitionMap [iceberg]

aokolnychyi commented on code in PR #9194: URL: https://github.com/apache/iceberg/pull/9194#discussion_r1418166372 ## core/src/test/java/org/apache/iceberg/util/TestPartitionMap.java: ## @@ -0,0 +1,222 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or

Re: [I] Flink DataStream Small file Issue And RewriteDataFiles Action [iceberg]

github-actions[bot] commented on issue #7568: URL: https://github.com/apache/iceberg/issues/7568#issuecomment-1843918076 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Flink DataStream Small file Issue And RewriteDataFiles Action [iceberg]

github-actions[bot] closed issue #7568: Flink DataStream Small file Issue And RewriteDataFiles Action URL: https://github.com/apache/iceberg/issues/7568 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] Update Slack Links [iceberg-docs]

RussellSpitzer commented on PR #295: URL: https://github.com/apache/iceberg-docs/pull/295#issuecomment-1843865471 Thanks @bitsondatadev ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

Re: [PR] Update Slack Links [iceberg-docs]

RussellSpitzer merged PR #295: URL: https://github.com/apache/iceberg-docs/pull/295 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@i

[PR] Rest Catalog: Add RESTful data operations [iceberg]

geruh opened a new pull request, #9237: URL: https://github.com/apache/iceberg/pull/9237 Hi All, I have a proposal that can be found [here](https://docs.google.com/document/d/1OG68EtPxLWvNBJACQwcMrRYuGJCnQas8_LSruTRcHG8). This proposal aims to allow for data commits to be a part of

Re: [PR] Flink: switch to use SortKey for data statistics [iceberg]

stevenzwu commented on code in PR #9212: URL: https://github.com/apache/iceberg/pull/9212#discussion_r1418105469 ## flink/v1.17/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceWithWatermarkExtractor.java: ## @@ -92,7 +93,10 @@ public class TestIcebergSourceW

Re: [PR] Flink: switch to use SortKey for data statistics [iceberg]

stevenzwu commented on code in PR #9212: URL: https://github.com/apache/iceberg/pull/9212#discussion_r1418105469 ## flink/v1.17/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceWithWatermarkExtractor.java: ## @@ -92,7 +93,10 @@ public class TestIcebergSourceW

[PR] Update Slack Links [iceberg-docs]

bitsondatadev opened a new pull request, #295: URL: https://github.com/apache/iceberg-docs/pull/295 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscr

[PR] Build: Bump actions/setup-python from 4 to 5 [iceberg-python]

dependabot[bot] opened a new pull request, #191: URL: https://github.com/apache/iceberg-python/pull/191 Bumps [actions/setup-python](https://github.com/actions/setup-python) from 4 to 5. Release notes Sourced from https://github.com/actions/setup-python/releases";>actions/setup-pyt

Re: [PR] API, Spark: Fix aggregation pushdown on struct fields [iceberg]

amogh-jahagirdar commented on code in PR #9176: URL: https://github.com/apache/iceberg/pull/9176#discussion_r1418053930 ## api/src/main/java/org/apache/iceberg/expressions/ValueAggregate.java: ## @@ -30,13 +30,16 @@ protected ValueAggregate(Operation op, BoundTerm term) { @

[PR] Build: Bump fastavro from 1.9.0 to 1.9.1 [iceberg-python]

dependabot[bot] opened a new pull request, #190: URL: https://github.com/apache/iceberg-python/pull/190 Bumps [fastavro](https://github.com/fastavro/fastavro) from 1.9.0 to 1.9.1. Changelog Sourced from https://github.com/fastavro/fastavro/blob/master/ChangeLog";>fastavro's changel

[PR] Build: Bump griffe from 0.38.0 to 0.38.1 [iceberg-python]

dependabot[bot] opened a new pull request, #189: URL: https://github.com/apache/iceberg-python/pull/189 Bumps [griffe](https://github.com/mkdocstrings/griffe) from 0.38.0 to 0.38.1. Release notes Sourced from https://github.com/mkdocstrings/griffe/releases";>griffe's releases.

Re: [I] Uploading Data to Iceberg [iceberg-python]

dbuades commented on issue #181: URL: https://github.com/apache/iceberg-python/issues/181#issuecomment-1843786012 I agree that this functionality will be very useful, thank you @Fokko for working on this ! Do you happen to have any ETA for that PR? -- This is an automated message f

Re: [PR] Core: Add PartitionMap [iceberg]

aokolnychyi commented on code in PR #9194: URL: https://github.com/apache/iceberg/pull/9194#discussion_r1418028589 ## core/src/main/java/org/apache/iceberg/util/PartitionSet.java: ## @@ -32,7 +33,7 @@ import org.apache.iceberg.relocated.com.google.common.collect.Maps; import o

Re: [PR] maint(transforms): replace `type()` calls with `isinstance()` [iceberg-python]

jayceslesar commented on PR #188: URL: https://github.com/apache/iceberg-python/pull/188#issuecomment-1843750742 https://github.com/apache/iceberg-python/blob/main/pyiceberg/expressions/visitors.py#L987 is probably fine to stay as adding `isinstance` would make it a little uglier to cast

Re: [PR] maint(transforms): replace `type()` calls with `isinstance()` [iceberg-python]

jayceslesar commented on PR #188: URL: https://github.com/apache/iceberg-python/pull/188#issuecomment-1843747021 Found what looks like a bug https://github.com/apache/iceberg-python/blob/main/pyiceberg/table/__init__.py#L173 when changing to `isinstance` -- This is an automated message f

Re: [PR] Core: Fix equality in StructLikeMap [iceberg]

aokolnychyi commented on PR #9236: URL: https://github.com/apache/iceberg/pull/9236#issuecomment-1843709180 Thanks, @RussellSpitzer! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [PR] Core: Fix equality in StructLikeMap [iceberg]

aokolnychyi merged PR #9236: URL: https://github.com/apache/iceberg/pull/9236 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [PR] fix(bucket transform): bad comparison for truthy value [iceberg-python]

Fokko merged PR #187: URL: https://github.com/apache/iceberg-python/pull/187 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [I] BUG: bucket transform on integer 0 return NAN [iceberg-python]

Fokko closed issue #173: BUG: bucket transform on integer 0 return NAN URL: https://github.com/apache/iceberg-python/issues/173 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [I] Catalog fails to load table using the table's identifier [iceberg-python]

pdames commented on issue #123: URL: https://github.com/apache/iceberg-python/issues/123#issuecomment-1843693977 Resolved by https://github.com/apache/iceberg-python/pull/150 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [I] Catalog fails to load table using the table's identifier [iceberg-python]

pdames closed issue #123: Catalog fails to load table using the table's identifier URL: https://github.com/apache/iceberg-python/issues/123 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

Re: [PR] fix(bucket transform): bad comparison for truthy value [iceberg-python]

jayceslesar commented on PR #187: URL: https://github.com/apache/iceberg-python/pull/187#issuecomment-1843687984 jeez also noticed that the time transforms could use some love. I did some very similar work in a project involving time series data so will take a stab at making that more maint

Re: [PR] Core: Fixed certain operations failing to add new data files during retries [iceberg]

jasonf20 commented on code in PR #9230: URL: https://github.com/apache/iceberg/pull/9230#discussion_r1417962641 ## core/src/test/java/org/apache/iceberg/TestFastAppend.java: ## @@ -313,6 +313,37 @@ public void testRecoveryWithoutManifestList() { metadata.currentSnapsho

Re: [PR] Core: Fixed certain operations failing to add new data files during retries [iceberg]

jasonf20 commented on code in PR #9230: URL: https://github.com/apache/iceberg/pull/9230#discussion_r1417962641 ## core/src/test/java/org/apache/iceberg/TestFastAppend.java: ## @@ -313,6 +313,37 @@ public void testRecoveryWithoutManifestList() { metadata.currentSnapsho

Re: [PR] Core: Fixed certain operations failing to add new data files during retries [iceberg]

jasonf20 commented on code in PR #9230: URL: https://github.com/apache/iceberg/pull/9230#discussion_r1417960440 ## core/src/main/java/org/apache/iceberg/FastAppend.java: ## @@ -187,8 +187,7 @@ protected void cleanUncommitted(Set committed) { deleteFile(manifest.path(

Re: [PR] Core: Fix equality in StructLikeMap [iceberg]

aokolnychyi commented on code in PR #9236: URL: https://github.com/apache/iceberg/pull/9236#discussion_r1417915340 ## core/src/test/java/org/apache/iceberg/util/TestStructLikeMap.java: ## @@ -147,4 +149,32 @@ public void testKeysWithNulls() { assertThat(map.remove(record3

Re: [PR] Core: Fix equality in StructLikeMap [iceberg]

RussellSpitzer commented on code in PR #9236: URL: https://github.com/apache/iceberg/pull/9236#discussion_r1417914889 ## core/src/test/java/org/apache/iceberg/util/TestStructLikeMap.java: ## @@ -147,4 +149,32 @@ public void testKeysWithNulls() { assertThat(map.remove(reco

Re: [PR] Core: Fix equality in StructLikeMap [iceberg]

aokolnychyi commented on code in PR #9236: URL: https://github.com/apache/iceberg/pull/9236#discussion_r1417902414 ## core/src/main/java/org/apache/iceberg/util/StructLikeMap.java: ## @@ -146,25 +145,19 @@ public R getValue() { @Override public int hashCode() { -

Re: [PR] Core: Fix equality in StructLikeMap [iceberg]

aokolnychyi commented on code in PR #9236: URL: https://github.com/apache/iceberg/pull/9236#discussion_r1417901732 ## core/src/main/java/org/apache/iceberg/util/StructLikeMap.java: ## @@ -146,25 +145,19 @@ public R getValue() { @Override public int hashCode() { -

Re: [PR] Core: Fix equality in StructLikeMap [iceberg]

aokolnychyi commented on code in PR #9236: URL: https://github.com/apache/iceberg/pull/9236#discussion_r1417900027 ## core/src/main/java/org/apache/iceberg/util/StructLikeMap.java: ## @@ -146,25 +145,19 @@ public R getValue() { @Override public int hashCode() { -

[PR] Core: Fix equality in StructLikeMap [iceberg]

aokolnychyi opened a new pull request, #9236: URL: https://github.com/apache/iceberg/pull/9236 This PR fixes equality `StructLikeMap`. See tests for use cases that previously failed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to G

Re: [PR] Core: Introduce AssertViewUUID [iceberg]

rdblue merged PR #8831: URL: https://github.com/apache/iceberg/pull/8831 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

Re: [PR] Core: Fixed certain operations failing to add new data files during retries [iceberg]

rdblue commented on code in PR #9230: URL: https://github.com/apache/iceberg/pull/9230#discussion_r1417822086 ## core/src/main/java/org/apache/iceberg/FastAppend.java: ## @@ -187,8 +187,7 @@ protected void cleanUncommitted(Set committed) { deleteFile(manifest.path())

Re: [PR] Core: Fixed certain operations failing to add new data files during retries [iceberg]

rdblue commented on code in PR #9230: URL: https://github.com/apache/iceberg/pull/9230#discussion_r1417820651 ## core/src/test/java/org/apache/iceberg/TestFastAppend.java: ## @@ -313,6 +313,37 @@ public void testRecoveryWithoutManifestList() { metadata.currentSnapshot(

Re: [PR] Core: Fixed certain operations failing to add new data files during retries [iceberg]

rdblue commented on code in PR #9230: URL: https://github.com/apache/iceberg/pull/9230#discussion_r1417819364 ## core/src/test/java/org/apache/iceberg/TestFastAppend.java: ## @@ -313,6 +313,37 @@ public void testRecoveryWithoutManifestList() { metadata.currentSnapshot(

Re: [PR] Core: Introduce AssertViewUUID [iceberg]

rdblue commented on PR #8831: URL: https://github.com/apache/iceberg/pull/8831#issuecomment-1843399469 +1 when tests are passing. Thanks for making the refactor to move methods into the base class. -- This is an automated message from the Apache Git Service. To respond to the message, ple

Re: [PR] Core: Introduce AssertViewUUID [iceberg]

rdblue commented on code in PR #8831: URL: https://github.com/apache/iceberg/pull/8831#discussion_r1417763740 ## open-api/rest-catalog-open-api.yaml: ## @@ -2510,6 +2510,32 @@ components: default-sort-order-id: type: integer +ViewRequirement: Review Co

Re: [PR] Core: Introduce AssertViewUUID [iceberg]

nastra commented on code in PR #8831: URL: https://github.com/apache/iceberg/pull/8831#discussion_r1417738523 ## core/src/main/java/org/apache/iceberg/UpdateRequirement.java: ## @@ -62,12 +62,31 @@ public void validate(TableMetadata base) { "Requirement failed: UUID

Re: [PR] Core: Introduce AssertViewUUID [iceberg]

nastra commented on code in PR #8831: URL: https://github.com/apache/iceberg/pull/8831#discussion_r1417735452 ## core/src/main/java/org/apache/iceberg/UpdateRequirement.java: ## @@ -62,12 +62,31 @@ public void validate(TableMetadata base) { "Requirement failed: UUID

Re: [PR] Core: Introduce AssertViewUUID [iceberg]

rdblue commented on code in PR #8831: URL: https://github.com/apache/iceberg/pull/8831#discussion_r1417671572 ## open-api/rest-catalog-open-api.yaml: ## @@ -2510,6 +2510,32 @@ components: default-sort-order-id: type: integer +ViewRequirement: Review Co

Re: [PR] Core: Introduce AssertViewUUID [iceberg]

rdblue commented on code in PR #8831: URL: https://github.com/apache/iceberg/pull/8831#discussion_r1417668998 ## core/src/main/java/org/apache/iceberg/UpdateRequirement.java: ## @@ -62,12 +62,31 @@ public void validate(TableMetadata base) { "Requirement failed: UUID

Re: [PR] Spark: Use Awaitility instead of Thread.sleep [iceberg]

yyy1000 commented on PR #9224: URL: https://github.com/apache/iceberg/pull/9224#issuecomment-1843230107 Hi, @nk1506 It seems that my code can't pass the unit tests. Is it because of the default waiting time is not enough? Do you have any ideas? -- This is an automated message from t

Re: [I] Apache Flink not committing new snapshots to Iceberg Table [iceberg]

pvary commented on issue #9089: URL: https://github.com/apache/iceberg/issues/9089#issuecomment-1843199768 Flink Writer operators write the data files to their final place - to the data directory of the table. Then sends the file metadata (file name, statistics, etc) to the Flink Committer

Re: [I] BUG: bucket transform on integer 0 return NAN [iceberg-python]

jayceslesar commented on issue #173: URL: https://github.com/apache/iceberg-python/issues/173#issuecomment-1843190418 Looks like the error is just a bad truthy comparison on v, which evaluates False if it is 0, resulting in the None return? https://github.com/apache/iceberg-python/blob/main

[I] StreamingMonitorFunction ignores the "streaming-strategy" and the "start-snapshot-timestamp" when set [iceberg]

asdaraujo opened a new issue, #9235: URL: https://github.com/apache/iceberg/issues/9235 ### Apache Iceberg version 1.3.0 ### Query engine Flink ### Please describe the bug 🐞 The [StreamingMonitorFunction](https://github.com/apache/iceberg/blob/main/flink/v1

Re: [PR] Spark: Bump Spark minor versions for 3.3 and 3.4 [iceberg]

RussellSpitzer commented on PR #9187: URL: https://github.com/apache/iceberg/pull/9187#issuecomment-1843129829 Thanks @ajantha-bhat for the PR and @Fokko for review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

Re: [PR] Spark: Bump Spark minor versions for 3.3 and 3.4 [iceberg]

RussellSpitzer merged PR #9187: URL: https://github.com/apache/iceberg/pull/9187 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceb

Re: [PR] Spark: Bump Spark minor versions for 3.3 and 3.4 [iceberg]

RussellSpitzer commented on code in PR #9187: URL: https://github.com/apache/iceberg/pull/9187#discussion_r1417524087 ## spark/v3.4/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestAddFilesProcedure.java: ## @@ -77,7 +77,7 @@ public void setupTempDirs() {

Re: [PR] Nessie: Support views for NessieCatalog [iceberg]

dimas-b commented on code in PR #8909: URL: https://github.com/apache/iceberg/pull/8909#discussion_r1417478873 ## core/src/test/java/org/apache/iceberg/view/ViewCatalogTests.java: ## @@ -465,8 +472,15 @@ public void replaceViewThatAlreadyExistsAsTable() { .w

Re: [PR] Nessie: reimplement namespace operations [iceberg]

nastra commented on code in PR #8857: URL: https://github.com/apache/iceberg/pull/8857#discussion_r1417446745 ## nessie/src/test/java/org/apache/iceberg/nessie/TestNessieIcebergClient.java: ## @@ -91,15 +113,444 @@ public void testWithReferenceAfterRecreatingBranch() Assert

Re: [PR] Nessie: reimplement namespace operations [iceberg]

nastra commented on code in PR #8857: URL: https://github.com/apache/iceberg/pull/8857#discussion_r1417126334 ## nessie/src/test/java/org/apache/iceberg/nessie/TestMultipleClients.java: ## @@ -67,33 +71,84 @@ public void afterEach() throws Exception { } @Test - public v

Re: [PR] Spark: Use Awaitility instead of Thread.sleep [iceberg]

yyy1000 commented on PR #9224: URL: https://github.com/apache/iceberg/pull/9224#issuecomment-1842988192 > Please change for other spark version too like 3.2,3.3,3.4. Sure, I just change a single version for review to verify I'm doing right. :) -- This is an automated message from th

Re: [I] Apache Flink not committing new snapshots to Iceberg Table [iceberg]

FranMorilloAWS commented on issue #9089: URL: https://github.com/apache/iceberg/issues/9089#issuecomment-1842974757 I upgraded to Iceberg 1.14 and Flink 1.15 and still am seeing the issue if the table has more than a 1 snapshots. Could somebody explain how writing, and comitting actual

Re: [PR] Flink: switch to use SortKey for data statistics [iceberg]

pvary commented on code in PR #9212: URL: https://github.com/apache/iceberg/pull/9212#discussion_r1417296998 ## flink/v1.17/flink/src/test/java/org/apache/iceberg/flink/sink/shuffle/TestDataStatisticsOperator.java: ## @@ -50,33 +49,36 @@ import org.apache.flink.table.data.Gener

Re: [PR] Flink: switch to use SortKey for data statistics [iceberg]

pvary commented on code in PR #9212: URL: https://github.com/apache/iceberg/pull/9212#discussion_r1417290847 ## flink/v1.17/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceWithWatermarkExtractor.java: ## @@ -92,7 +93,10 @@ public class TestIcebergSourceWithW

Re: [I] Max number of columns [iceberg]

ajantha-bhat commented on issue #9220: URL: https://github.com/apache/iceberg/issues/9220#issuecomment-1842692615 I don't remember any hard coded limittaion. But Field ID of partition columns in schema starts from 10K. So if you have 300K columns it will conflict with column ids. So,

Re: [PR] Spark: Use Awaitility instead of Thread.sleep [iceberg]

nk1506 commented on code in PR #9224: URL: https://github.com/apache/iceberg/pull/9224#discussion_r1417129615 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestCopyOnWriteDelete.java: ## @@ -113,9 +116,10 @@ public synchronized void testDelete

Re: [PR] Docs: Add GC.ENABLED property [iceberg]

ajantha-bhat commented on PR #9231: URL: https://github.com/apache/iceberg/pull/9231#issuecomment-1842664896 PR title can be "gc.enabled" instead of "GC.ENABLED" as properties are case sensitive. -- This is an automated message from the Apache Git Service. To respond to the message, plea

Re: [PR] Docs: Add GC.ENABLED property [iceberg]

ajantha-bhat commented on code in PR #9231: URL: https://github.com/apache/iceberg/pull/9231#discussion_r1417117291 ## docs/configuration.md: ## @@ -111,6 +111,7 @@ Iceberg tables support table properties to configure table behavior, like the de | history.expire.max-snapshot-a

[PR] Docs: Add GC.ENABLED property [iceberg]

nk1506 opened a new pull request, #9231: URL: https://github.com/apache/iceberg/pull/9231 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mai

[PR] Fixed certain operations failing to add new data files during retries [iceberg]

jasonf20 opened a new pull request, #9230: URL: https://github.com/apache/iceberg/pull/9230 Since the following PR: https://github.com/apache/iceberg/pull/6335 FastAppend and subclasses of MergingSnapshotProducer will skip newly added data files during retries. This happens because t

Re: [PR] Core: Implement equals/hashCode method for RESTResponse [iceberg]

liurenjie1024 commented on code in PR #9049: URL: https://github.com/apache/iceberg/pull/9049#discussion_r1416984941 ## core/src/main/java/org/apache/iceberg/TableMetadata.java: ## @@ -846,6 +846,92 @@ private static Map validateRefs( return inputRefs; } + @Override +

Re: [PR] Core: Implement equals/hashCode method for RESTResponse [iceberg]

liurenjie1024 commented on code in PR #9049: URL: https://github.com/apache/iceberg/pull/9049#discussion_r1416984941 ## core/src/main/java/org/apache/iceberg/TableMetadata.java: ## @@ -846,6 +846,92 @@ private static Map validateRefs( return inputRefs; } + @Override +

[PR] Automatically create the tables for the `SqlCatalog` [iceberg-python]

cosmastech opened a new pull request, #186: URL: https://github.com/apache/iceberg-python/pull/186 In order to avoid having to manually call `SqlCatalog.create_tables()`, we will check that both tables exist on initialization -- This is an automated message from the Apache Git Service. To

Re: [PR] Fixed certain operations failing to add new data files during retries [iceberg]

jasonf20 commented on code in PR #9222: URL: https://github.com/apache/iceberg/pull/9222#discussion_r1416968091 ## core/src/main/java/org/apache/iceberg/FastAppend.java: ## @@ -188,7 +188,7 @@ protected void cleanUncommitted(Set committed) { } } - this.ne

Re: [PR] Fixed certain operations failing to add new data files during retries [iceberg]

jasonf20 commented on code in PR #9222: URL: https://github.com/apache/iceberg/pull/9222#discussion_r1416968091 ## core/src/main/java/org/apache/iceberg/FastAppend.java: ## @@ -188,7 +188,7 @@ protected void cleanUncommitted(Set committed) { } } - this.ne

Re: [PR] Fixed certain operations failing to add new data files during retries [iceberg]

jasonf20 commented on code in PR #9222: URL: https://github.com/apache/iceberg/pull/9222#discussion_r1416968091 ## core/src/main/java/org/apache/iceberg/FastAppend.java: ## @@ -188,7 +188,7 @@ protected void cleanUncommitted(Set committed) { } } - this.ne

Re: [PR] Fixed certain operations failing to add new data files during retries [iceberg]

jasonf20 commented on code in PR #9222: URL: https://github.com/apache/iceberg/pull/9222#discussion_r1416968091 ## core/src/main/java/org/apache/iceberg/FastAppend.java: ## @@ -188,7 +188,7 @@ protected void cleanUncommitted(Set committed) { } } - this.ne

Re: [I] Avoid dictionary (de)serialization for model modification [iceberg-python]

Fokko closed issue #179: Avoid dictionary (de)serialization for model modification URL: https://github.com/apache/iceberg-python/issues/179 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

Re: [PR] Use Pydantic's `model_copy` for model modification when updating table metadata [iceberg-python]

Fokko merged PR #182: URL: https://github.com/apache/iceberg-python/pull/182 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [I] The "Status" paragraph in the readme seems very outdated [iceberg]

Fokko commented on issue #9127: URL: https://github.com/apache/iceberg/issues/9127#issuecomment-1842441255 Hey @ronkorving Thanks for raising this. That's very true, the status page seems to be very outdated. The V1, and V2 specs are out there, and work is being done on the V3 spec (

Re: [PR] Core: Implement equals/hashCode method for RESTResponse [iceberg]

nastra commented on code in PR #9049: URL: https://github.com/apache/iceberg/pull/9049#discussion_r1416913245 ## api/src/main/java/org/apache/iceberg/Schema.java: ## @@ -507,4 +508,25 @@ public String toString() { .map(this::identifierFieldToString)

Re: [I] Question about "Column ID" in Parquet Sepc [iceberg]

Fokko closed issue #9099: Question about "Column ID" in Parquet Sepc URL: https://github.com/apache/iceberg/issues/9099 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsub

Re: [PR] Spec: Clarify how column IDs are required for parquet files [iceberg]

Fokko merged PR #9162: URL: https://github.com/apache/iceberg/pull/9162 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apach

Re: [PR] Spec: Clarify how column IDs are required for parquet files [iceberg]

Fokko commented on PR #9162: URL: https://github.com/apache/iceberg/pull/9162#issuecomment-1842370347 @emkornfield Yes, I have. Just wanted to see if anyone else had some addition to this, but this looks great. Thanks for clarifying this. -- This is an automated message from the Apache Gi

Re: [PR] Spec: Clarify how column IDs are required for parquet files [iceberg]

emkornfield commented on PR #9162: URL: https://github.com/apache/iceberg/pull/9162#issuecomment-1842351441 @Fokko do you have the ability to merge this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] docs: Clarify some view issues [iceberg]

emkornfield commented on PR #7504: URL: https://github.com/apache/iceberg/pull/7504#issuecomment-1842345834 I think this has probably gone stale, I'm going to close this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] docs: Clarify some view issues [iceberg]

emkornfield closed pull request #7504: docs: Clarify some view issues URL: https://github.com/apache/iceberg/pull/7504 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubs