Re: [I] Adopt `Catalog` API to include references to the `TableMetadata` and the `metadata_location` in the `TableCommit` payload for the `update_table` method [iceberg-rust]

2023-10-05 Thread via GitHub
JanKaul commented on issue #75: URL: https://github.com/apache/iceberg-rust/issues/75#issuecomment-1750077811 A typical usage would look something like the following: 1. Get `Table` with `load_table` operation. (This already fetches the `metadata_location` and `TableMetadata`) 2. C

Re: [I] Support Hudi `DeltaStreamer` compatible feature [iceberg]

2023-10-05 Thread via GitHub
amogh-jahagirdar commented on issue #8724: URL: https://github.com/apache/iceberg/issues/8724#issuecomment-1750065550 I'm not very familiar with Hudi Deltastreamer but for Kafka ingest into Iceberg tables, @bryanck is working on contributing a Kafka sink connector into the project https://g

Re: [I] write.parquet.page-size-bytes isn't respected when writing data [iceberg]

2023-10-05 Thread via GitHub
paulpaul1076 commented on issue #8729: URL: https://github.com/apache/iceberg/issues/8729#issuecomment-1750061470 Wrong name in the title, sorry, copied it and didn't look. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

Re: [PR] Thread.sleep() method is replaced with Awaitility [iceberg]

2023-10-05 Thread via GitHub
shreyanshR7 commented on PR #8725: URL: https://github.com/apache/iceberg/pull/8725#issuecomment-1750055473 I've made the required changes :) @nk1506 @nastra -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

Re: [PR] Common docs for 1.4.0 [iceberg-docs]

2023-10-05 Thread via GitHub
nastra merged PR #276: URL: https://github.com/apache/iceberg-docs/pull/276 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.a

Re: [PR] Versioned docs for 1.4.0 [iceberg-docs]

2023-10-05 Thread via GitHub
nastra merged PR #277: URL: https://github.com/apache/iceberg-docs/pull/277 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.a

Re: [I] write.parquet.page-size-bytes isn't respected when writing data [iceberg]

2023-10-05 Thread via GitHub
amogh-jahagirdar commented on issue #8729: URL: https://github.com/apache/iceberg/issues/8729#issuecomment-1750041439 The issue title and parts of the description refer to `write.parquet.page-size-bytes` but what you are describing is the file size and in your code also you refer to file si

Re: [PR] Build: Add 1.4.0 to issue template [iceberg]

2023-10-05 Thread via GitHub
nastra merged PR #8728: URL: https://github.com/apache/iceberg/pull/8728 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

Re: [PR] Build: Let revapi compare against 1.4.0 [iceberg]

2023-10-05 Thread via GitHub
nastra merged PR #8727: URL: https://github.com/apache/iceberg/pull/8727 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

Re: [PR] Thread.sleep() method is replaced with Awaitility [iceberg]

2023-10-05 Thread via GitHub
nk1506 commented on code in PR #8725: URL: https://github.com/apache/iceberg/pull/8725#discussion_r1348266266 ## flink/v1.15/flink/src/test/java/org/apache/iceberg/flink/source/TestStreamingMonitorFunction.java: ## @@ -148,9 +150,10 @@ public void testConsumeFromStartSnapshotId(

[I] write.parquet.page-size-bytes isn't respected when writing data [iceberg]

2023-10-05 Thread via GitHub
paulpaul1076 opened a new issue, #8729: URL: https://github.com/apache/iceberg/issues/8729 ### Apache Iceberg version None ### Query engine None ### Please describe the bug 🐞 I have a job that reads orc files and writes them to a Spark table, the files that

Re: [I] Manifest List Writer Design [iceberg-rust]

2023-10-05 Thread via GitHub
ZENOTME commented on issue #72: URL: https://github.com/apache/iceberg-rust/issues/72#issuecomment-1749958215 cc @liurenjie1024 @Xuanwo @JanKaul @Fokko -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [I] Manifest List Writer Design [iceberg-rust]

2023-10-05 Thread via GitHub
ZENOTME commented on issue #72: URL: https://github.com/apache/iceberg-rust/issues/72#issuecomment-1749957317 We can integrate V1 and V2 into a writer ``` struck ManifestListWriter { ... } impl ManifestListWriter { fn write_v1(&mut self, manifest_list: ManifestL

Re: [PR] Versioned docs for 1.4.0 [iceberg-docs]

2023-10-05 Thread via GitHub
aokolnychyi commented on PR #277: URL: https://github.com/apache/iceberg-docs/pull/277#issuecomment-1749945932 @amogh-jahagirdar @Fokko @nastra @rdblue @danielcweeks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[PR] Versioned docs for 1.4.0 [iceberg-docs]

2023-10-05 Thread via GitHub
aokolnychyi opened a new pull request, #277: URL: https://github.com/apache/iceberg-docs/pull/277 This PR adds versioned docs for the 1.4.0 release. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

Re: [I] Spark fails to write into an iceberg table after updating its schema [iceberg]

2023-10-05 Thread via GitHub
paulpaul1076 commented on issue #8721: URL: https://github.com/apache/iceberg/issues/8721#issuecomment-1749943303 Calling this before my second insert of df2 solves the problem: spark.catalog.refreshTable("db_nessie.schema_evolution15") -- This is an automated message from the Apach

Re: [PR] Common docs for 1.4.0 [iceberg-docs]

2023-10-05 Thread via GitHub
aokolnychyi commented on PR #276: URL: https://github.com/apache/iceberg-docs/pull/276#issuecomment-1749941117 @amogh-jahagirdar @Fokko @nastra @rdblue @danielcweeks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[PR] Common docs for 1.4.0 [iceberg-docs]

2023-10-05 Thread via GitHub
aokolnychyi opened a new pull request, #276: URL: https://github.com/apache/iceberg-docs/pull/276 This PR is the first step to update our docs as described [here](https://iceberg.apache.org/how-to-release/#common-documentation-update). -- This is an automated message from the Apache Git S

Re: [PR] Build: Let revapi compare against 1.4.0 [iceberg]

2023-10-05 Thread via GitHub
aokolnychyi commented on PR #8727: URL: https://github.com/apache/iceberg/pull/8727#issuecomment-1749928311 @Fokko @nastra @rdblue @RussellSpitzer -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Build: Add 1.4.0 to issue template [iceberg]

2023-10-05 Thread via GitHub
aokolnychyi commented on PR #8728: URL: https://github.com/apache/iceberg/pull/8728#issuecomment-1749928447 @Fokko @nastra @rdblue @RussellSpitzer -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[PR] Build: Add 1.4.0 to issue template [iceberg]

2023-10-05 Thread via GitHub
aokolnychyi opened a new pull request, #8728: URL: https://github.com/apache/iceberg/pull/8728 This PR adds 1.4.0 to our issue template. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

Re: [PR] Phase 1 - New Docs Deployment [iceberg]

2023-10-05 Thread via GitHub
bitsondatadev commented on code in PR #8659: URL: https://github.com/apache/iceberg/pull/8659#discussion_r1348198680 ## docs-new/mkdocs.yml: ## @@ -0,0 +1,96 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTI

Re: [I] Support get partition table with filter [iceberg-python]

2023-10-05 Thread via GitHub
puchengy commented on issue #24: URL: https://github.com/apache/iceberg-python/issues/24#issuecomment-1749897984 @Fokko Yes, I can help. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

Re: [PR] Python: Remove python directory and references [iceberg]

2023-10-05 Thread via GitHub
ajantha-bhat commented on PR #8695: URL: https://github.com/apache/iceberg/pull/8695#issuecomment-1749882115 Shall we merge this if no comments? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

Re: [PR] Phase 1 - New Docs Deployment [iceberg]

2023-10-05 Thread via GitHub
bitsondatadev commented on code in PR #8659: URL: https://github.com/apache/iceberg/pull/8659#discussion_r1348126895 ## docs-new/.github/bin/deploy_docs.sh: ## @@ -0,0 +1,39 @@ +#!/bin/bash + +while [[ "$#" -gt 0 ]]; do +case $1 in +-v|--version) ICEBERG_VERSION="$2"

Re: [PR] Kafka Connect: Initial project setup and event data structures [iceberg]

2023-10-05 Thread via GitHub
rdblue commented on code in PR #8701: URL: https://github.com/apache/iceberg/pull/8701#discussion_r1348115184 ## kafka-connect/kafka-connect-events/src/main/java/org/apache/iceberg/connect/events/Element.java: ## @@ -0,0 +1,37 @@ +/* + * Licensed to the Apache Software Foundatio

Re: [PR] Kafka Connect: Initial project setup and event data structures [iceberg]

2023-10-05 Thread via GitHub
rdblue commented on code in PR #8701: URL: https://github.com/apache/iceberg/pull/8701#discussion_r1348114157 ## kafka-connect/kafka-connect-events/src/main/java/org/apache/iceberg/connect/events/CommitCompletePayload.java: ## @@ -0,0 +1,97 @@ +/* + * Licensed to the Apache Soft

Re: [PR] Spec: add types timstamp_ns and timestamptz_ns [iceberg]

2023-10-05 Thread via GitHub
jacobmarble commented on PR #8683: URL: https://github.com/apache/iceberg/pull/8683#issuecomment-1749799494 > What do you think? If you agree, then we should probably add `ms` types at the same time. Sounds good to me. I'll add milliseconds to this PR. -- This is an automated messa

Re: [PR] Spec: add types timstamp_ns and timestamptz_ns [iceberg]

2023-10-05 Thread via GitHub
rdblue commented on code in PR #8683: URL: https://github.com/apache/iceberg/pull/8683#discussion_r1348103716 ## format/spec.md: ## @@ -167,30 +167,34 @@ A **`map`** is a collection of key-value pairs with a key type and a value type. Primitive Types -| Primitive type

Re: [PR] Spec: add types timstamp_ns and timestamptz_ns [iceberg]

2023-10-05 Thread via GitHub
rdblue commented on PR #8683: URL: https://github.com/apache/iceberg/pull/8683#issuecomment-1749792874 @jacobmarble, I was talking with @danielcweeks a couple days ago about v3 additions and we came up with a good reason to add `timestamp_ms` and `timestamptz_ms`. Currently, type promotion

Re: [PR] Core: Mark `503: added_snapshot_id` as required [iceberg]

2023-10-05 Thread via GitHub
rdblue commented on code in PR #8673: URL: https://github.com/apache/iceberg/pull/8673#discussion_r1348100841 ## core/src/main/java/org/apache/iceberg/GenericManifestFile.java: ## @@ -84,15 +84,24 @@ public GenericManifestFile(Schema avroSchema) { } } + /** + * @dep

Re: [PR] Core: Mark `503: added_snapshot_id` as required [iceberg]

2023-10-05 Thread via GitHub
rdblue commented on code in PR #8673: URL: https://github.com/apache/iceberg/pull/8673#discussion_r1348099841 ## core/src/main/java/org/apache/iceberg/V2Metadata.java: ## @@ -39,7 +39,7 @@ private V2Metadata() {} ManifestFile.MANIFEST_CONTENT.asRequired(),

Re: [PR] add support of before and after actions in preorderschema traversal [iceberg-python]

2023-10-05 Thread via GitHub
rdblue commented on PR #42: URL: https://github.com/apache/iceberg-python/pull/42#issuecomment-1749783634 I don't think that this is necessary and I think it also breaks future uses of the visitor. Despite the name, the visitor this updates doesn't actually perform pre-order traversa

Re: [PR] Phase 1 - New Docs Deployment [iceberg]

2023-10-05 Thread via GitHub
rdblue commented on code in PR #8659: URL: https://github.com/apache/iceberg/pull/8659#discussion_r1348089146 ## docs-new/README.md: ## @@ -0,0 +1,175 @@ + + +# Iceberg site and documentation + +This holds the static files that define and build the documentation site for Apache

Re: [PR] Phase 1 - New Docs Deployment [iceberg]

2023-10-05 Thread via GitHub
rdblue commented on code in PR #8659: URL: https://github.com/apache/iceberg/pull/8659#discussion_r1348087991 ## docs-new/README.md: ## @@ -0,0 +1,123 @@ + + +# making-iceberg-docs + +Testbed for the Iceberg docs. + +## Requirements + +* Python >=3.9 +* pip + + +## Build proces

Re: [PR] Phase 1 - New Docs Deployment [iceberg]

2023-10-05 Thread via GitHub
rdblue commented on code in PR #8659: URL: https://github.com/apache/iceberg/pull/8659#discussion_r1348087050 ## docs-new/.github/bin/deploy_docs.sh: ## @@ -0,0 +1,39 @@ +#!/bin/bash + +while [[ "$#" -gt 0 ]]; do +case $1 in +-v|--version) ICEBERG_VERSION="$2"; shift

Re: [PR] Phase 1 - New Docs Deployment [iceberg]

2023-10-05 Thread via GitHub
rdblue commented on code in PR #8659: URL: https://github.com/apache/iceberg/pull/8659#discussion_r1348077823 ## docs-new/mkdocs.yml: ## @@ -0,0 +1,96 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file

Re: [PR] Phase 1 - New Docs Deployment [iceberg]

2023-10-05 Thread via GitHub
rdblue commented on code in PR #8659: URL: https://github.com/apache/iceberg/pull/8659#discussion_r1348077823 ## docs-new/mkdocs.yml: ## @@ -0,0 +1,96 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file

Re: [PR] Phase 1 - New Docs Deployment [iceberg]

2023-10-05 Thread via GitHub
rdblue commented on code in PR #8659: URL: https://github.com/apache/iceberg/pull/8659#discussion_r1348076686 ## docs-new/mkdocs.yml: ## @@ -0,0 +1,96 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file

Re: [PR] Phase 1 - New Docs Deployment [iceberg]

2023-10-05 Thread via GitHub
rdblue commented on code in PR #8659: URL: https://github.com/apache/iceberg/pull/8659#discussion_r1348075495 ## docs-new/mkdocs.yml: ## @@ -0,0 +1,96 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file

[PR] add support of before and after in preorderschema visitor for traversal [iceberg-python]

2023-10-05 Thread via GitHub
MehulBatra opened a new pull request, #42: URL: https://github.com/apache/iceberg-python/pull/42 Add support to perform an action before and after field traversal in preorder fashion for schema for object: structtype, maptype & listtype -- This is an automated message from the Apache

Re: [PR] Phase 1 - New Docs Deployment [iceberg]

2023-10-05 Thread via GitHub
bitsondatadev commented on PR #8659: URL: https://github.com/apache/iceberg/pull/8659#issuecomment-1749722304 I've addressed the comments I intend to hit this review round, but have some open questions: - https://github.com/apache/iceberg/pull/8659#discussion_r1343890396 - https://gith

Re: [PR] Phase 1 - New Docs Deployment [iceberg]

2023-10-05 Thread via GitHub
bitsondatadev commented on code in PR #8659: URL: https://github.com/apache/iceberg/pull/8659#discussion_r1348043125 ## docs-new/mkdocs.yml: ## @@ -0,0 +1,96 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTI

Re: [PR] Core: Mark `503: added_snapshot_id` as required [iceberg]

2023-10-05 Thread via GitHub
Fokko commented on code in PR #8673: URL: https://github.com/apache/iceberg/pull/8673#discussion_r1347949961 ## api/src/main/java/org/apache/iceberg/ManifestFile.java: ## @@ -49,7 +49,7 @@ public interface ManifestFile { Types.LongType.get(), "Lowest sequen

Re: [PR] Revert "Spec: Mark added_snapshot_id as optional (#8600)" [iceberg]

2023-10-05 Thread via GitHub
rdblue merged PR #8726: URL: https://github.com/apache/iceberg/pull/8726 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

Re: [PR] Spec: Inconsistency around files_count [iceberg]

2023-10-05 Thread via GitHub
Fokko merged PR #5338: URL: https://github.com/apache/iceberg/pull/5338 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apach

Re: [I] Spec disagrees with implementation on field names [iceberg]

2023-10-05 Thread via GitHub
Fokko closed issue #8684: Spec disagrees with implementation on field names URL: https://github.com/apache/iceberg/issues/8684 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

Re: [PR] Spec: Inconsistency around files_count [iceberg]

2023-10-05 Thread via GitHub
Fokko commented on PR #5338: URL: https://github.com/apache/iceberg/pull/5338#issuecomment-1749550427 Thanks for the review @rdblue 🙌 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] Revert "Spec: Mark added_snapshot_id as optional (#8600)" [iceberg]

2023-10-05 Thread via GitHub
rdblue commented on PR #8726: URL: https://github.com/apache/iceberg/pull/8726#issuecomment-1749494924 Thank you! Good to get this fixed quickly. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[PR] Revert "Spec: Mark added_snapshot_id as optional (#8600)" [iceberg]

2023-10-05 Thread via GitHub
Fokko opened a new pull request, #8726: URL: https://github.com/apache/iceberg/pull/8726 This reverts commit f7a7eb2c10cb4a9b6b3ea5bfdfc5d085be8b9c31. This should stay required. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] Spec: Clarify missing fields when writing [iceberg]

2023-10-05 Thread via GitHub
rdblue commented on code in PR #8672: URL: https://github.com/apache/iceberg/pull/8672#discussion_r1347854144 ## format/spec.md: ## @@ -128,13 +128,13 @@ Tables do not require rename, except for tables that use atomic rename to implem Writer requirements -Some tables

Re: [PR] Spec: Clarify missing fields when writing [iceberg]

2023-10-05 Thread via GitHub
rdblue commented on code in PR #8672: URL: https://github.com/apache/iceberg/pull/8672#discussion_r1347851474 ## format/spec.md: ## @@ -128,13 +128,13 @@ Tables do not require rename, except for tables that use atomic rename to implem Writer requirements -Some tables

Re: [PR] Spec: Clarify missing fields when writing [iceberg]

2023-10-05 Thread via GitHub
rdblue commented on code in PR #8672: URL: https://github.com/apache/iceberg/pull/8672#discussion_r1347851058 ## format/spec.md: ## @@ -128,13 +128,13 @@ Tables do not require rename, except for tables that use atomic rename to implem Writer requirements -Some tables

Re: [PR] Rename master branch to main [iceberg]

2023-10-05 Thread via GitHub
jbonofre commented on PR #8722: URL: https://github.com/apache/iceberg/pull/8722#issuecomment-1749444235 @rdblue no impact on the existing PRs: they will be "attached" to main instead of master. The only impact is for the remotes for our users. -- This is an automated message from the Ap

Re: [PR] Hive: Push filtering for Iceberg table type to Hive MetaStore when listing tables [iceberg]

2023-10-05 Thread via GitHub
hankfanchiu commented on PR #2722: URL: https://github.com/apache/iceberg/pull/2722#issuecomment-1749443748 @mderoy, thanks for the interest! I don't plan on following through with apache/hive#2484, #2751, and this PR. Please feel free to take over. -- This is an automated message from th

Re: [I] Support get partition table with filter [iceberg-python]

2023-10-05 Thread via GitHub
Fokko commented on issue #24: URL: https://github.com/apache/iceberg-python/issues/24#issuecomment-1749431518 @puchengy Yes, certainly. Would this be something that you're interested in working on? From the snapshot, we can load the manifest list, and from there the manifests themselves, wh

Re: [PR] Rename master branch to main [iceberg]

2023-10-05 Thread via GitHub
rdblue commented on PR #8722: URL: https://github.com/apache/iceberg/pull/8722#issuecomment-1749425255 @jbonofre, how will this affect existing PRs? Do all of them need to be reopened against main instead of master? -- This is an automated message from the Apache Git Service. To respond t

Re: [PR] Rename master branch to main [iceberg]

2023-10-05 Thread via GitHub
rdblue commented on code in PR #8722: URL: https://github.com/apache/iceberg/pull/8722#discussion_r1347814726 ## .github/workflows/python-release.yml: ## @@ -51,7 +51,7 @@ jobs: - name: Set version run: python -m poetry version "${{ inputs.version }}" wo

Re: [PR] Core: Mark `503: added_snapshot_id` as required [iceberg]

2023-10-05 Thread via GitHub
Fokko commented on code in PR #8673: URL: https://github.com/apache/iceberg/pull/8673#discussion_r1347814559 ## core/src/main/java/org/apache/iceberg/GenericManifestFile.java: ## @@ -92,7 +92,7 @@ public GenericManifestFile(Schema avroSchema) { this.specId = specId; th

Re: [I] Write data to iceberg with using java api multi thread, cause data lost [iceberg]

2023-10-05 Thread via GitHub
rdblue commented on issue #8610: URL: https://github.com/apache/iceberg/issues/8610#issuecomment-1749415259 Yes, I agree. The safety of the HadoopCatalog depends on the file system, which is one reason why we don't recommend using that catalog. -- This is an automated message from the Apa

Re: [I] Write data to iceberg with using java api multi thread, cause data lost [iceberg]

2023-10-05 Thread via GitHub
rdblue closed issue #8610: Write data to iceberg with using java api multi thread, cause data lost URL: https://github.com/apache/iceberg/issues/8610 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [I] Support get partition table with filter [iceberg-python]

2023-10-05 Thread via GitHub
puchengy commented on issue #24: URL: https://github.com/apache/iceberg-python/issues/24#issuecomment-1749412122 @Fokko gentle ping, thanks ^ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

Re: [PR] Thread.sleep() method is replaced with Awaitility [iceberg]

2023-10-05 Thread via GitHub
shreyanshR7 commented on PR #8715: URL: https://github.com/apache/iceberg/pull/8715#issuecomment-1749406399 @nastra I made another pull request addressing the issue with some more clean changes, please consider that. -- This is an automated message from the Apache Git Service. To respond

Re: [PR] Thread.sleep() method is replaced with Awaitility [iceberg]

2023-10-05 Thread via GitHub
shreyanshR7 commented on PR #8715: URL: https://github.com/apache/iceberg/pull/8715#issuecomment-1749205309 Let me check, I will try my best to undo these unrelated changes, these changes may be occurred due to adding awaitility method. sorry for inconvenience @nastra -- This is an auto

Re: [PR] feat: add builder to TableMetadata interface [iceberg-rust]

2023-10-05 Thread via GitHub
y0psolo commented on PR #62: URL: https://github.com/apache/iceberg-rust/pull/62#issuecomment-1749197704 Ok i added some validation steps. Tell me if its ok i don't know if we need to have all fields visible on the builder. Tell me. We can choose to open and restrict later or do the rever

Re: [PR] Thread.sleep() method is replaced with Awaitility [iceberg]

2023-10-05 Thread via GitHub
nastra commented on PR #8715: URL: https://github.com/apache/iceberg/pull/8715#issuecomment-1749152592 @shreyanshR7 there are still many unrelated changes to files. Can you please fix all of those? -- This is an automated message from the Apache Git Service. To respond to the message, ple

Re: [PR] Thread.sleep() method is replaced with Awaitility [iceberg]

2023-10-05 Thread via GitHub
shreyanshR7 commented on PR #8715: URL: https://github.com/apache/iceberg/pull/8715#issuecomment-1749145628 I tried to fix the issues :) @nastra -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

Re: [PR] Core: Add View support for REST catalog [iceberg]

2023-10-05 Thread via GitHub
amogh-jahagirdar commented on code in PR #7913: URL: https://github.com/apache/iceberg/pull/7913#discussion_r1347539389 ## core/src/main/java/org/apache/iceberg/rest/CatalogHandlers.java: ## @@ -374,4 +385,107 @@ static TableMetadata commit(TableOperations ops, UpdateTableReque

Re: [PR] Core: Mark `503: added_snapshot_id` as required [iceberg]

2023-10-05 Thread via GitHub
Fokko commented on code in PR #8673: URL: https://github.com/apache/iceberg/pull/8673#discussion_r1347466602 ## core/src/main/java/org/apache/iceberg/GenericManifestFile.java: ## @@ -92,7 +92,7 @@ public GenericManifestFile(Schema avroSchema) { this.specId = specId; th

Re: [PR] Core: Mark `503: added_snapshot_id` as required [iceberg]

2023-10-05 Thread via GitHub
Fokko commented on code in PR #8673: URL: https://github.com/apache/iceberg/pull/8673#discussion_r1347466602 ## core/src/main/java/org/apache/iceberg/GenericManifestFile.java: ## @@ -92,7 +92,7 @@ public GenericManifestFile(Schema avroSchema) { this.specId = specId; th

Re: [PR] Core: Mark `503: added_snapshot_id` as required [iceberg]

2023-10-05 Thread via GitHub
Fokko commented on code in PR #8673: URL: https://github.com/apache/iceberg/pull/8673#discussion_r1347459324 ## core/src/main/java/org/apache/iceberg/GenericManifestFile.java: ## @@ -92,7 +92,7 @@ public GenericManifestFile(Schema avroSchema) { this.specId = specId; th

Re: [PR] Core: Mark `503: added_snapshot_id` as required [iceberg]

2023-10-05 Thread via GitHub
amogh-jahagirdar commented on code in PR #8673: URL: https://github.com/apache/iceberg/pull/8673#discussion_r1347419881 ## core/src/main/java/org/apache/iceberg/GenericManifestFile.java: ## @@ -92,7 +92,7 @@ public GenericManifestFile(Schema avroSchema) { this.specId = spec

Re: [PR] Build: Upgrade to gradle 8.4 [iceberg]

2023-10-05 Thread via GitHub
jbonofre commented on PR #8486: URL: https://github.com/apache/iceberg/pull/8486#issuecomment-1748930989 Re-testing `revapi` on Gradle 8.4 (it should be the same as with Gradle 8.3). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Gi

Re: [PR] Construct a writer tree [iceberg-python]

2023-10-05 Thread via GitHub
Fokko commented on code in PR #40: URL: https://github.com/apache/iceberg-python/pull/40#discussion_r1345991299 ## pyiceberg/manifest.py: ## @@ -262,15 +346,13 @@ class DataFile(Record): "split_offsets", "equality_ids", "sort_order_id", -"spec_

Re: [PR] Core: Make view metadata properties optional in JSON parser [iceberg]

2023-10-05 Thread via GitHub
amogh-jahagirdar commented on code in PR #8723: URL: https://github.com/apache/iceberg/pull/8723#discussion_r1347436052 ## core/src/main/java/org/apache/iceberg/view/ViewMetadataParser.java: ## @@ -66,7 +67,9 @@ static void toJson(ViewMetadata metadata, JsonGenerator gen) throw

Re: [I] Upgrade to gradle 8.4 [iceberg]

2023-10-05 Thread via GitHub
jbonofre commented on issue #8485: URL: https://github.com/apache/iceberg/issues/8485#issuecomment-1748905073 As gradle 8.4 is now available, I'm updating the PR (I'm also re-testing `revapi` but I'm pretty sure it will be the same as in Gradle 8.4 :) ). -- This is an automated message fr

Re: [PR] Core: Make view metadata properties optional in JSON parser [iceberg]

2023-10-05 Thread via GitHub
amogh-jahagirdar commented on code in PR #8723: URL: https://github.com/apache/iceberg/pull/8723#discussion_r1347411761 ## core/src/main/java/org/apache/iceberg/view/ViewMetadataParser.java: ## @@ -66,7 +67,9 @@ static void toJson(ViewMetadata metadata, JsonGenerator gen) throw

Re: [PR] Core: Make view metadata properties optional in JSON parser [iceberg]

2023-10-05 Thread via GitHub
amogh-jahagirdar commented on code in PR #8723: URL: https://github.com/apache/iceberg/pull/8723#discussion_r1347411761 ## core/src/main/java/org/apache/iceberg/view/ViewMetadataParser.java: ## @@ -66,7 +67,9 @@ static void toJson(ViewMetadata metadata, JsonGenerator gen) throw

Re: [PR] Core: Make view metadata properties optional in JSON parser [iceberg]

2023-10-05 Thread via GitHub
amogh-jahagirdar commented on code in PR #8723: URL: https://github.com/apache/iceberg/pull/8723#discussion_r1347395043 ## core/src/main/java/org/apache/iceberg/view/ViewMetadataParser.java: ## @@ -66,7 +67,9 @@ static void toJson(ViewMetadata metadata, JsonGenerator gen) throw

Re: [PR] Core: Make view metadata properties optional in JSON parser [iceberg]

2023-10-05 Thread via GitHub
amogh-jahagirdar commented on code in PR #8723: URL: https://github.com/apache/iceberg/pull/8723#discussion_r1347395043 ## core/src/main/java/org/apache/iceberg/view/ViewMetadataParser.java: ## @@ -66,7 +67,9 @@ static void toJson(ViewMetadata metadata, JsonGenerator gen) throw

Re: [PR] Core: Make view metadata properties optional in JSON parser [iceberg]

2023-10-05 Thread via GitHub
amogh-jahagirdar commented on code in PR #8723: URL: https://github.com/apache/iceberg/pull/8723#discussion_r1347395043 ## core/src/main/java/org/apache/iceberg/view/ViewMetadataParser.java: ## @@ -66,7 +67,9 @@ static void toJson(ViewMetadata metadata, JsonGenerator gen) throw

Re: [PR] Core: Make view metadata properties optional in JSON parser [iceberg]

2023-10-05 Thread via GitHub
amogh-jahagirdar commented on code in PR #8723: URL: https://github.com/apache/iceberg/pull/8723#discussion_r1347395043 ## core/src/main/java/org/apache/iceberg/view/ViewMetadataParser.java: ## @@ -66,7 +67,9 @@ static void toJson(ViewMetadata metadata, JsonGenerator gen) throw

Re: [PR] Core: Make view metadata properties optional in JSON parser [iceberg]

2023-10-05 Thread via GitHub
amogh-jahagirdar commented on code in PR #8723: URL: https://github.com/apache/iceberg/pull/8723#discussion_r1347395043 ## core/src/main/java/org/apache/iceberg/view/ViewMetadataParser.java: ## @@ -66,7 +67,9 @@ static void toJson(ViewMetadata metadata, JsonGenerator gen) throw

Re: [PR] Core: Make view metadata properties optional in JSON parser [iceberg]

2023-10-05 Thread via GitHub
nastra commented on code in PR #8723: URL: https://github.com/apache/iceberg/pull/8723#discussion_r1347392925 ## core/src/main/java/org/apache/iceberg/view/ViewMetadataParser.java: ## @@ -66,7 +67,9 @@ static void toJson(ViewMetadata metadata, JsonGenerator gen) throws IOExcept

Re: [I] Adopt `Catalog` API to include references to the `TableMetadata` and the `metadata_location` in the `TableCommit` payload for the `update_table` method [iceberg-rust]

2023-10-05 Thread via GitHub
Xuanwo commented on issue #75: URL: https://github.com/apache/iceberg-rust/issues/75#issuecomment-1748854108 > It would therefore be helpful to include references to the `metadata_location` and the `TableMetadata` in the [`TableCommit`](https://github.com/apache/iceberg-rust/blob/main/crate

Re: [PR] Core: Make view metadata properties optional in JSON parser [iceberg]

2023-10-05 Thread via GitHub
ajantha-bhat commented on code in PR #8723: URL: https://github.com/apache/iceberg/pull/8723#discussion_r1347371858 ## core/src/main/java/org/apache/iceberg/view/ViewMetadataParser.java: ## @@ -66,7 +67,9 @@ static void toJson(ViewMetadata metadata, JsonGenerator gen) throws IO

Re: [PR] Thread.sleep() method is replaced with Awaitility [iceberg]

2023-10-05 Thread via GitHub
shreyanshR7 commented on PR #8715: URL: https://github.com/apache/iceberg/pull/8715#issuecomment-1748822301 wait let me check -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] Core: Make view metadata properties optional in JSON parser [iceberg]

2023-10-05 Thread via GitHub
amogh-jahagirdar commented on code in PR #8723: URL: https://github.com/apache/iceberg/pull/8723#discussion_r1347356619 ## core/src/main/java/org/apache/iceberg/view/ViewMetadataParser.java: ## @@ -66,7 +67,9 @@ static void toJson(ViewMetadata metadata, JsonGenerator gen) throw

Re: [PR] Thread.sleep() method is replaced with Awaitility [iceberg]

2023-10-05 Thread via GitHub
nastra commented on code in PR #8715: URL: https://github.com/apache/iceberg/pull/8715#discussion_r1347352377 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestIcebergSourceTablesBase.java: ## @@ -1955,7 +1958,6 @@ public void testRemoveOrphanFilesActionSuppo

Re: [PR] Thread.sleep() method is replaced with Awaitility [iceberg]

2023-10-05 Thread via GitHub
nastra commented on code in PR #8715: URL: https://github.com/apache/iceberg/pull/8715#discussion_r1347351835 ## flink/v1.16/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceFailover.java: ## @@ -208,7 +210,7 @@ private void testContinuousIcebergSource(Failov

Re: [PR] Thread.sleep() method is replaced with Awaitility [iceberg]

2023-10-05 Thread via GitHub
nastra commented on code in PR #8715: URL: https://github.com/apache/iceberg/pull/8715#discussion_r1347351094 ## core/src/test/java/org/apache/iceberg/util/TestInMemoryLockManager.java: ## @@ -112,7 +114,7 @@ public void testAcquireSingleProcess() throws Exception { Complet

Re: [PR] Thread.sleep() method is replaced with Awaitility [iceberg]

2023-10-05 Thread via GitHub
nastra commented on code in PR #8715: URL: https://github.com/apache/iceberg/pull/8715#discussion_r1347350422 ## api/src/test/java/org/apache/iceberg/metrics/TestDefaultTimer.java: ## @@ -104,7 +104,7 @@ public void measureRunnable() { Runnable runnable = () -> {

Re: [PR] Thread.sleep() method is replaced with Awaitility [iceberg]

2023-10-05 Thread via GitHub
shreyanshR7 commented on PR #8715: URL: https://github.com/apache/iceberg/pull/8715#issuecomment-1748692640 I've made the changes :)@nastra -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

Re: [PR] Rename master branch to main [iceberg]

2023-10-05 Thread via GitHub
jbonofre commented on code in PR #8722: URL: https://github.com/apache/iceberg/pull/8722#discussion_r1347247075 ## docs/flink-writes.md: ## @@ -270,4 +270,4 @@ INSERT INTO tableName /*+ OPTIONS('upsert-enabled'='true') */ ... ``` -Check out all the options here: [write-opt

Re: [PR] Core: Make view metadata properties optional in JSON parser [iceberg]

2023-10-05 Thread via GitHub
ajantha-bhat commented on code in PR #8723: URL: https://github.com/apache/iceberg/pull/8723#discussion_r1347242330 ## core/src/main/java/org/apache/iceberg/view/ViewMetadataParser.java: ## @@ -66,7 +67,9 @@ static void toJson(ViewMetadata metadata, JsonGenerator gen) throws IO

Re: [PR] Phase 1 - New Docs Deployment [iceberg]

2023-10-05 Thread via GitHub
bitsondatadev commented on code in PR #8659: URL: https://github.com/apache/iceberg/pull/8659#discussion_r1347234000 ## docs-new/site/releases.md: ## @@ -0,0 +1,777 @@ +--- +title: "Releases" +--- + + +## Downloads + +The latest version of Iceberg is [{{ icebergVersion }}](http

Re: [PR] Rename master branch to main [iceberg]

2023-10-05 Thread via GitHub
ajantha-bhat commented on code in PR #8722: URL: https://github.com/apache/iceberg/pull/8722#discussion_r1347234482 ## docs/flink-writes.md: ## @@ -270,4 +270,4 @@ INSERT INTO tableName /*+ OPTIONS('upsert-enabled'='true') */ ... ``` -Check out all the options here: [write

Re: [PR] Phase 1 - New Docs Deployment [iceberg]

2023-10-05 Thread via GitHub
bitsondatadev commented on code in PR #8659: URL: https://github.com/apache/iceberg/pull/8659#discussion_r1347234000 ## docs-new/site/releases.md: ## @@ -0,0 +1,777 @@ +--- +title: "Releases" +--- + + +## Downloads + +The latest version of Iceberg is [{{ icebergVersion }}](http

Re: [PR] Phase 1 - New Docs Deployment [iceberg]

2023-10-05 Thread via GitHub
bitsondatadev commented on code in PR #8659: URL: https://github.com/apache/iceberg/pull/8659#discussion_r1347232147 ## docs-new/site/releases.md: ## @@ -0,0 +1,777 @@ +--- +title: "Releases" +--- + + +## Downloads + +The latest version of Iceberg is [{{ icebergVersion }}](http

[PR] Core: Make view metadata properties optional in JSON parser [iceberg]

2023-10-05 Thread via GitHub
nastra opened a new pull request, #8723: URL: https://github.com/apache/iceberg/pull/8723 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mai

  1   2   >