Re: [PR] Spark 3.5: Fix flaky TestSparkExecutorCache [iceberg]

2024-01-30 Thread via GitHub
aokolnychyi closed pull request #9583: Spark 3.5: Fix flaky TestSparkExecutorCache URL: https://github.com/apache/iceberg/pull/9583 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

Re: [PR] Partition Evolution [iceberg-python]

2024-01-30 Thread via GitHub
HonahX commented on code in PR #245: URL: https://github.com/apache/iceberg-python/pull/245#discussion_r1469034571 ## pyiceberg/table/__init__.py: ## @@ -533,6 +551,39 @@ def _(update: SetCurrentSchemaUpdate, base_metadata: TableMetadata, context: _Ta return base_metadata.

Re: [PR] Flink: Implement enumerator metrics for pending splits, pending recor… [iceberg]

2024-01-30 Thread via GitHub
pvary merged PR #9524: URL: https://github.com/apache/iceberg/pull/9524 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apach

Re: [PR] Flink: Implement enumerator metrics for pending splits, pending recor… [iceberg]

2024-01-30 Thread via GitHub
pvary commented on PR #9524: URL: https://github.com/apache/iceberg/pull/9524#issuecomment-1916303375 @mas-chen: Please backport this to 1.17, 1.18 and continue with the Junit5 migration PR. Thanks @mas-chen for the PR and @stevenzwu for the review! -- This is an automated message

[PR] Docs: Enhance Java quickstart example [iceberg]

2024-01-30 Thread via GitHub
manuzhang opened a new pull request, #9585: URL: https://github.com/apache/iceberg/pull/9585 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-

[I] Cannot drop table with AWS Glue Catalog [iceberg]

2024-01-30 Thread via GitHub
duwanqiebi opened a new issue, #9586: URL: https://github.com/apache/iceberg/issues/9586 ### Apache Iceberg version None ### Query engine None ### Please describe the bug 🐞 - Conf: - "spark.sql.catalog.spark_catalog": "org.apache.iceberg.spark.SparkS

Re: [PR] Spark 3.5: Add an option not to delete files in ExpireSnapshots [iceberg]

2024-01-30 Thread via GitHub
manuzhang closed pull request #9584: Spark 3.5: Add an option not to delete files in ExpireSnapshots URL: https://github.com/apache/iceberg/pull/9584 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[PR] feat: Add user guide for website. [iceberg-rust]

2024-01-30 Thread via GitHub
liurenjie1024 opened a new pull request, #178: URL: https://github.com/apache/iceberg-rust/pull/178 This pr add user guide for website. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

Re: [PR] feat: Add user guide for website. [iceberg-rust]

2024-01-30 Thread via GitHub
liurenjie1024 commented on PR #178: URL: https://github.com/apache/iceberg-rust/pull/178#issuecomment-1916332649 cc @Xuanwo @ZENOTME @Fokko PTAL -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

Re: [PR] feat: add parquet writer [iceberg-rust]

2024-01-30 Thread via GitHub
liurenjie1024 commented on code in PR #176: URL: https://github.com/apache/iceberg-rust/pull/176#discussion_r1470792480 ## crates/iceberg/src/spec/manifest.rs: ## @@ -924,7 +924,7 @@ impl TryFrom for ManifestStatus { } /// Data file carries data file path, partition tuple, m

Re: [I] Cannot drop table with AWS Glue Catalog [iceberg]

2024-01-30 Thread via GitHub
nastra commented on issue #9586: URL: https://github.com/apache/iceberg/issues/9586#issuecomment-1916390985 Might be similar to https://github.com/apache/iceberg/issues/5565, I wonder if you're missing the catalog name in the table identifier -- This is an automated message from the Apach

Re: [I] Remove `unwrap()` in `ManifestListWriter.close()` [iceberg-rust]

2024-01-30 Thread via GitHub
liurenjie1024 commented on issue #177: URL: https://github.com/apache/iceberg-rust/issues/177#issuecomment-1916433902 Thanks for reporting! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

Re: [PR] feat: add support for catalogs with glue implementation to start [iceberg-go]

2024-01-30 Thread via GitHub
wolfeidau commented on PR #51: URL: https://github.com/apache/iceberg-go/pull/51#issuecomment-1916574894 @zeroshade done -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] Python: Introduce setters [iceberg-python]

2024-01-30 Thread via GitHub
Fokko closed pull request #304: Python: Introduce setters URL: https://github.com/apache/iceberg-python/pull/304 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

Re: [PR] Spark: Rewrite identifier when using Subquery expressions in View [iceberg]

2024-01-30 Thread via GitHub
nastra commented on code in PR #9587: URL: https://github.com/apache/iceberg/pull/9587#discussion_r1471120727 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestViews.java: ## @@ -877,6 +877,9 @@ public void dropV1View() { v1Session

[PR] Use `.read_bytes()` instead [iceberg-python]

2024-01-30 Thread via GitHub
Fokko opened a new pull request, #325: URL: https://github.com/apache/iceberg-python/pull/325 Noticed this while profiling. This simplifies the code a bit. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[I] Performance optimization [iceberg]

2024-01-30 Thread via GitHub
Fokko opened a new issue, #9588: URL: https://github.com/apache/iceberg/issues/9588 ### Apache Iceberg version None ### Query engine None ### Please describe the bug 🐞 Seeing if there is anything we can do to improve performance: ![profile](https

Re: [I] Performance optimization [iceberg]

2024-01-30 Thread via GitHub
liurenjie1024 commented on issue #9588: URL: https://github.com/apache/iceberg/issues/9588#issuecomment-1916860122 Seems a lot of time are spent on decompression. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [I] Performance optimization [iceberg]

2024-01-30 Thread via GitHub
Fokko commented on issue #9588: URL: https://github.com/apache/iceberg/issues/9588#issuecomment-1916881774 s3fs: ![output](https://github.com/apache/iceberg/assets/1134248/4332cd61-d280-48b5-88de-30574fd728fc) PyArrow: ![output](https://github.com/apache/iceberg/assets/1134248/

Re: [PR] InMemory Catalog Implementation [iceberg-python]

2024-01-30 Thread via GitHub
syun64 commented on code in PR #289: URL: https://github.com/apache/iceberg-python/pull/289#discussion_r1471348272 ## tests/catalog/test_base.py: ## Review Comment: This is a _super-nit_ (please feel free to take it or leave it): should we rename this test module to **test

Re: [I] Support deletion in Apache Flink [iceberg]

2024-01-30 Thread via GitHub
VidakM commented on issue #8718: URL: https://github.com/apache/iceberg/issues/8718#issuecomment-1917071229 Is `DELTE FROM` still on the roadmap for Flink or could we help out in some way? -- This is an automated message from the Apache Git Service. To respond to the message, please log o

Re: [PR] Spark 3.5: Add an option not to delete files in ExpireSnapshots [iceberg]

2024-01-30 Thread via GitHub
nastra commented on code in PR #9584: URL: https://github.com/apache/iceberg/pull/9584#discussion_r1471390452 ## api/src/main/java/org/apache/iceberg/ExpireSnapshots.java: ## @@ -118,4 +118,9 @@ public interface ExpireSnapshots extends PendingUpdate> { * @return this for me

Re: [PR] Spark 3.5: Add an option not to delete files in ExpireSnapshots [iceberg]

2024-01-30 Thread via GitHub
nastra commented on code in PR #9584: URL: https://github.com/apache/iceberg/pull/9584#discussion_r1471391266 ## api/src/main/java/org/apache/iceberg/actions/ExpireSnapshots.java: ## @@ -99,6 +99,11 @@ public interface ExpireSnapshots extends Action

Re: [PR] Spark 3.5: Add an option not to delete files in ExpireSnapshots [iceberg]

2024-01-30 Thread via GitHub
nastra commented on code in PR #9584: URL: https://github.com/apache/iceberg/pull/9584#discussion_r1471397337 ## core/src/main/java/org/apache/iceberg/RemoveSnapshots.java: ## @@ -85,6 +85,8 @@ public void accept(String file) { private ExecutorService planExecutorService = Th

Re: [PR] Spark 3.5: Add an option not to delete files in ExpireSnapshots [iceberg]

2024-01-30 Thread via GitHub
nastra commented on code in PR #9584: URL: https://github.com/apache/iceberg/pull/9584#discussion_r1471399965 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestExpireSnapshotsProcedure.java: ## @@ -541,6 +541,29 @@ public void testExpireSnapsh

Re: [PR] Spark 3.5: Add an option not to delete files in ExpireSnapshots [iceberg]

2024-01-30 Thread via GitHub
nastra commented on code in PR #9584: URL: https://github.com/apache/iceberg/pull/9584#discussion_r1471405013 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/ExpireSnapshotsSparkAction.java: ## @@ -67,6 +67,8 @@ public class ExpireSnapshotsSparkAction extends

Re: [PR] Partition Evolution [iceberg-python]

2024-01-30 Thread via GitHub
amogh-jahagirdar commented on code in PR #245: URL: https://github.com/apache/iceberg-python/pull/245#discussion_r1471453813 ## pyiceberg/table/__init__.py: ## @@ -2271,3 +2325,243 @@ def commit(self) -> Snapshot: ) return snapshot + + +class UpdateSpec:

Re: [PR] add github add to check md link [iceberg-python]

2024-01-30 Thread via GitHub
kevinjqliu commented on PR #324: URL: https://github.com/apache/iceberg-python/pull/324#issuecomment-1917449075 Here's the result from the GitHub Action run from my fork https://github.com/kevinjqliu/iceberg-python/actions/runs/7707078539/job/21003669945 -- This is an automated message

Re: [PR] InMemory Catalog Implementation [iceberg-python]

2024-01-30 Thread via GitHub
kevinjqliu commented on code in PR #289: URL: https://github.com/apache/iceberg-python/pull/289#discussion_r1471604886 ## tests/catalog/test_base.py: ## Review Comment: thanks for the review @syun64. Since the In-Memory catalog is not a "production" catalog, I'm inclined t

Re: [PR] InMemory Catalog Implementation [iceberg-python]

2024-01-30 Thread via GitHub
syun64 commented on code in PR #289: URL: https://github.com/apache/iceberg-python/pull/289#discussion_r1471609730 ## tests/catalog/test_base.py: ## Review Comment: Makes sense @kevinjqliu💯 -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] Move nightly versioned docs to top-level docs directory [iceberg]

2024-01-30 Thread via GitHub
Fokko merged PR #9578: URL: https://github.com/apache/iceberg/pull/9578 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apach

Re: [PR] Spark 3.5: Fix flaky TestSparkExecutorCache [iceberg]

2024-01-30 Thread via GitHub
aokolnychyi closed pull request #9583: Spark 3.5: Fix flaky TestSparkExecutorCache URL: https://github.com/apache/iceberg/pull/9583 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

Re: [PR] Spark 3.4: Fix writing of default values in CoW for rows with NULL columns which are unmatched [iceberg]

2024-01-30 Thread via GitHub
rdblue merged PR #9556: URL: https://github.com/apache/iceberg/pull/9556 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

Re: [PR] Spark 3.4: Fix writing of default values in CoW for rows with NULL columns which are unmatched [iceberg]

2024-01-30 Thread via GitHub
rdblue commented on PR #9556: URL: https://github.com/apache/iceberg/pull/9556#issuecomment-1917527435 Merging this. Thanks @amogh-jahagirdar! > It is still not clear how a wrong nullability info in MergeRows led to inserting default values. @aokolnychyi, I think what's happeni

Re: [PR] Spark: Add support for describing/showing views [iceberg]

2024-01-30 Thread via GitHub
rdblue commented on code in PR #9513: URL: https://github.com/apache/iceberg/pull/9513#discussion_r1471656403 ## spark/v3.5/spark-extensions/src/main/scala/org/apache/spark/sql/catalyst/analysis/RewriteViewCommands.scala: ## @@ -60,6 +63,13 @@ case class RewriteViewCommands(spar

Re: [PR] Spark: Add support for describing/showing views [iceberg]

2024-01-30 Thread via GitHub
rdblue commented on code in PR #9513: URL: https://github.com/apache/iceberg/pull/9513#discussion_r1471663971 ## spark/v3.5/spark-extensions/src/main/scala/org/apache/spark/sql/execution/datasources/v2/ExtendedDataSourceV2Strategy.scala: ## @@ -123,6 +128,18 @@ case class Extend

Re: [PR] Spark: Add support for describing/showing views [iceberg]

2024-01-30 Thread via GitHub
rdblue commented on code in PR #9513: URL: https://github.com/apache/iceberg/pull/9513#discussion_r1471663971 ## spark/v3.5/spark-extensions/src/main/scala/org/apache/spark/sql/execution/datasources/v2/ExtendedDataSourceV2Strategy.scala: ## @@ -123,6 +128,18 @@ case class Extend

Re: [PR] Spark: Add support for describing/showing views [iceberg]

2024-01-30 Thread via GitHub
rdblue commented on code in PR #9513: URL: https://github.com/apache/iceberg/pull/9513#discussion_r1471666384 ## spark/v3.5/spark-extensions/src/main/scala/org/apache/spark/sql/execution/datasources/v2/ExtendedDataSourceV2Strategy.scala: ## @@ -123,6 +128,18 @@ case class Extend

Re: [PR] Spark: Add support for describing/showing views [iceberg]

2024-01-30 Thread via GitHub
rdblue commented on code in PR #9513: URL: https://github.com/apache/iceberg/pull/9513#discussion_r1471669400 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestViews.java: ## @@ -1149,6 +1149,159 @@ public void createViewWithSubqueryExpression

Re: [PR] Spark: Add support for describing/showing views [iceberg]

2024-01-30 Thread via GitHub
rdblue commented on code in PR #9513: URL: https://github.com/apache/iceberg/pull/9513#discussion_r1471671329 ## spark/v3.5/spark-extensions/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DescribeV2ViewExec.scala: ## @@ -0,0 +1,78 @@ +/* + * Licensed to the Apache

Re: [PR] Spark: Add support for describing/showing views [iceberg]

2024-01-30 Thread via GitHub
rdblue commented on code in PR #9513: URL: https://github.com/apache/iceberg/pull/9513#discussion_r1471676481 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestViews.java: ## @@ -1149,6 +1149,159 @@ public void createViewWithSubqueryExpression

Re: [PR] Spark: Add support for describing/showing views [iceberg]

2024-01-30 Thread via GitHub
rdblue commented on code in PR #9513: URL: https://github.com/apache/iceberg/pull/9513#discussion_r1471676989 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestViews.java: ## @@ -1149,6 +1149,159 @@ public void createViewWithSubqueryExpression

Re: [PR] Spark: Add support for describing/showing views [iceberg]

2024-01-30 Thread via GitHub
rdblue commented on code in PR #9513: URL: https://github.com/apache/iceberg/pull/9513#discussion_r1471677391 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestViews.java: ## @@ -1149,6 +1149,159 @@ public void createViewWithSubqueryExpression

Re: [I] Support compaction [iceberg-python]

2024-01-30 Thread via GitHub
HonahX commented on issue #270: URL: https://github.com/apache/iceberg-python/issues/270#issuecomment-1917560747 I am interested in taking this if no one has started working on it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

Re: [PR] Spark: Add support for describing/showing views [iceberg]

2024-01-30 Thread via GitHub
rdblue commented on code in PR #9513: URL: https://github.com/apache/iceberg/pull/9513#discussion_r1471678490 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestViews.java: ## @@ -1149,6 +1149,159 @@ public void createViewWithSubqueryExpression

Re: [PR] Spark: Add support for describing/showing views [iceberg]

2024-01-30 Thread via GitHub
rdblue commented on code in PR #9513: URL: https://github.com/apache/iceberg/pull/9513#discussion_r1471683079 ## spark/v3.5/spark-extensions/src/main/scala/org/apache/spark/sql/execution/datasources/v2/ShowCreateV2ViewExec.scala: ## @@ -0,0 +1,79 @@ +/* + * Licensed to the Apach

Re: [PR] Spark: Add support for describing/showing views [iceberg]

2024-01-30 Thread via GitHub
rdblue commented on PR #9513: URL: https://github.com/apache/iceberg/pull/9513#issuecomment-1917565679 I pointed out a few things that I think I would change, but there is nothing blocking this. +1 overall. Please fix what you agree with (including the `SHOW VIEWS` behavior with temp views)

Re: [PR] Spark: Support altering view properties [iceberg]

2024-01-30 Thread via GitHub
rdblue commented on code in PR #9582: URL: https://github.com/apache/iceberg/pull/9582#discussion_r1471690098 ## spark/v3.5/spark-extensions/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckViews.scala: ## @@ -36,6 +38,13 @@ object CheckViews extends (LogicalPlan => Un

Re: [PR] Spark: Support altering view properties [iceberg]

2024-01-30 Thread via GitHub
rdblue commented on code in PR #9582: URL: https://github.com/apache/iceberg/pull/9582#discussion_r1471691985 ## spark/v3.5/spark-extensions/src/main/scala/org/apache/spark/sql/catalyst/analysis/RewriteViewCommands.scala: ## @@ -21,19 +21,21 @@ package org.apache.spark.sql.catal

[I] Add example of using PyIceberg with minimal external dependencies [iceberg-python]

2024-01-30 Thread via GitHub
kevinjqliu opened a new issue, #326: URL: https://github.com/apache/iceberg-python/issues/326 ### Feature Request / Improvement With #299 resolved, let's add an example to show working with Iceberg using minimal external dependencies. Get started with Iceberg using the local f

Re: [I] Add partition evolution [iceberg-python]

2024-01-30 Thread via GitHub
syun64 commented on issue #193: URL: https://github.com/apache/iceberg-python/issues/193#issuecomment-1917576022 @Fokko Should we move this to 0.7.0 release Milestone? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

Re: [PR] Spark: Support altering view properties [iceberg]

2024-01-30 Thread via GitHub
rdblue commented on code in PR #9582: URL: https://github.com/apache/iceberg/pull/9582#discussion_r1471693576 ## spark/v3.5/spark-extensions/src/main/scala/org/apache/spark/sql/catalyst/analysis/RewriteViewCommands.scala: ## @@ -60,17 +62,18 @@ case class RewriteViewCommands(spa

Re: [PR] Use `.read_bytes()` instead [iceberg-python]

2024-01-30 Thread via GitHub
HonahX merged PR #325: URL: https://github.com/apache/iceberg-python/pull/325 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [PR] Spark: Throw exception on `ALTER VIEW AS ` [iceberg]

2024-01-30 Thread via GitHub
nastra commented on code in PR #9510: URL: https://github.com/apache/iceberg/pull/9510#discussion_r1471696587 ## spark/v3.5/spark-extensions/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckViews.scala: ## @@ -36,6 +38,9 @@ object CheckViews extends (LogicalPlan => Uni

Re: [PR] Spark: Support altering view properties [iceberg]

2024-01-30 Thread via GitHub
rdblue commented on code in PR #9582: URL: https://github.com/apache/iceberg/pull/9582#discussion_r1471696609 ## spark/v3.5/spark-extensions/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckViews.scala: ## @@ -36,6 +38,13 @@ object CheckViews extends (LogicalPlan => Un

Re: [PR] Spark: Support altering view properties [iceberg]

2024-01-30 Thread via GitHub
rdblue commented on code in PR #9582: URL: https://github.com/apache/iceberg/pull/9582#discussion_r1471699613 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/SparkCatalog.java: ## @@ -606,6 +607,29 @@ public View createView( @Override public View alterView(Ident

Re: [PR] Spark: Support altering view properties [iceberg]

2024-01-30 Thread via GitHub
nastra commented on code in PR #9582: URL: https://github.com/apache/iceberg/pull/9582#discussion_r1471703651 ## spark/v3.5/spark-extensions/src/main/scala/org/apache/spark/sql/catalyst/analysis/RewriteViewCommands.scala: ## @@ -60,17 +62,18 @@ case class RewriteViewCommands(spa

Re: [PR] Spark: Rewrite identifier when using Subquery expressions in View [iceberg]

2024-01-30 Thread via GitHub
rdblue commented on PR #9587: URL: https://github.com/apache/iceberg/pull/9587#issuecomment-1917587797 Thanks, @nastra! Good to have this fixed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

Re: [PR] Spark: Support altering view properties [iceberg]

2024-01-30 Thread via GitHub
nastra commented on code in PR #9582: URL: https://github.com/apache/iceberg/pull/9582#discussion_r1471703651 ## spark/v3.5/spark-extensions/src/main/scala/org/apache/spark/sql/catalyst/analysis/RewriteViewCommands.scala: ## @@ -60,17 +62,18 @@ case class RewriteViewCommands(spa

Re: [PR] Spark: Rewrite identifier when using Subquery expressions in View [iceberg]

2024-01-30 Thread via GitHub
rdblue merged PR #9587: URL: https://github.com/apache/iceberg/pull/9587 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

Re: [PR] Build: Bump software.amazon.s3.accessgrants:aws-s3-accessgrants-java-plugin from 1.0.1 to 2.0.0 [iceberg]

2024-01-30 Thread via GitHub
jackye1995 merged PR #9575: URL: https://github.com/apache/iceberg/pull/9575 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] Spark 3.5: Fix flaky TestSparkExecutorCache [iceberg]

2024-01-30 Thread via GitHub
aokolnychyi commented on PR #9583: URL: https://github.com/apache/iceberg/pull/9583#issuecomment-1917593034 @amogh-jahagirdar @szehon-ho @nastra, could you check this one? I had 5 successful CI runs. -- This is an automated message from the Apache Git Service. To respond to the message, p

Re: [PR] Build: Bump software.amazon.awssdk:bom from 2.23.2 to 2.23.12 [iceberg]

2024-01-30 Thread via GitHub
jackye1995 merged PR #9573: URL: https://github.com/apache/iceberg/pull/9573 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [I] [Proposal] Iceberg Materialized View Spec [iceberg]

2024-01-30 Thread via GitHub
szehon-ho commented on issue #6420: URL: https://github.com/apache/iceberg/issues/6420#issuecomment-1917629464 Great, this all makes sense to me except following points: > 3. Question: Most people seem to be for Option 1 As we clarified in the google doc, it seems at least local

Re: [I] Create Iceberg Table from pyarrow Schema with no IDs [iceberg-python]

2024-01-30 Thread via GitHub
syun64 closed issue #278: Create Iceberg Table from pyarrow Schema with no IDs URL: https://github.com/apache/iceberg-python/issues/278 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

Re: [PR] Spark: Throw exception on `ALTER VIEW AS ` [iceberg]

2024-01-30 Thread via GitHub
rdblue commented on code in PR #9510: URL: https://github.com/apache/iceberg/pull/9510#discussion_r1471786657 ## spark/v3.5/spark-extensions/src/main/scala/org/apache/spark/sql/catalyst/analysis/RewriteViewCommands.scala: ## @@ -60,6 +62,9 @@ case class RewriteViewCommands(spark

[PR] REST: Set fresh IDs on create-table [iceberg-python]

2024-01-30 Thread via GitHub
Fokko opened a new pull request, #327: URL: https://github.com/apache/iceberg-python/pull/327 For the other implementation we do this when writing the metadata, but for the REST catalog we just post the schema itself. The REST spec does not mention anything about setting fresh IDs,

Re: [PR] Remove ASF publish property [iceberg-docs]

2024-01-30 Thread via GitHub
rdblue merged PR #308: URL: https://github.com/apache/iceberg-docs/pull/308 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.a

Re: [PR] Move nightly versioned docs to top-level docs directory [iceberg]

2024-01-30 Thread via GitHub
rdblue commented on PR #9578: URL: https://github.com/apache/iceberg/pull/9578#issuecomment-1917693542 Thanks for reverting this, @Fokko! I think we need to make sure that the docs don't move so that we have history and we can validate the changes from merging nightly into these docs. --

Re: [PR] New docs switch [iceberg]

2024-01-30 Thread via GitHub
rdblue merged PR #9520: URL: https://github.com/apache/iceberg/pull/9520 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

Re: [I] Problems in release doc about versioned documentation [iceberg]

2024-01-30 Thread via GitHub
rdblue closed issue #8151: Problems in release doc about versioned documentation URL: https://github.com/apache/iceberg/issues/8151 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

Re: [PR] REST: Set fresh IDs on create-table [iceberg-python]

2024-01-30 Thread via GitHub
rdblue commented on code in PR #327: URL: https://github.com/apache/iceberg-python/pull/327#discussion_r1471795199 ## pyiceberg/catalog/rest.py: ## @@ -447,13 +447,14 @@ def create_table( sort_order: SortOrder = UNSORTED_SORT_ORDER, properties: Properties = EMP