[GitHub] [iceberg] rdblue commented on a diff in pull request #6139: Python: Remove dataclasses

2022-11-16 Thread GitBox
rdblue commented on code in PR #6139: URL: https://github.com/apache/iceberg/pull/6139#discussion_r1024482375 ## python/pyiceberg/expressions/__init__.py: ## @@ -281,249 +361,313 @@ def __invert__(self) -> BoundIsNull: return BoundIsNull(self.term) -@dataclass(froze

[GitHub] [iceberg] rdblue commented on a diff in pull request #6139: Python: Remove dataclasses

2022-11-16 Thread GitBox
rdblue commented on code in PR #6139: URL: https://github.com/apache/iceberg/pull/6139#discussion_r1024482797 ## python/pyiceberg/expressions/__init__.py: ## @@ -281,249 +361,313 @@ def __invert__(self) -> BoundIsNull: return BoundIsNull(self.term) -@dataclass(froze

[GitHub] [iceberg] rdblue commented on a diff in pull request #6139: Python: Remove dataclasses

2022-11-16 Thread GitBox
rdblue commented on code in PR #6139: URL: https://github.com/apache/iceberg/pull/6139#discussion_r1024483160 ## python/pyiceberg/expressions/__init__.py: ## @@ -281,249 +361,313 @@ def __invert__(self) -> BoundIsNull: return BoundIsNull(self.term) -@dataclass(froze

[GitHub] [iceberg] rdblue commented on a diff in pull request #6139: Python: Remove dataclasses

2022-11-16 Thread GitBox
rdblue commented on code in PR #6139: URL: https://github.com/apache/iceberg/pull/6139#discussion_r1024484533 ## python/pyiceberg/expressions/__init__.py: ## @@ -18,18 +18,37 @@ from __future__ import annotations from abc import ABC, abstractmethod -from dataclasses import d

[GitHub] [iceberg] Fokko commented on a diff in pull request #6197: Python: Fix rough edges around literals

2022-11-16 Thread GitBox
Fokko commented on code in PR #6197: URL: https://github.com/apache/iceberg/pull/6197#discussion_r1024484631 ## python/pyiceberg/utils/datetime.py: ## @@ -81,13 +81,19 @@ def timestamp_to_micros(timestamp_str: str) -> int: """Converts an ISO-9601 formatted timestamp without

[GitHub] [iceberg] rdblue commented on a diff in pull request #6139: Python: Remove dataclasses

2022-11-16 Thread GitBox
rdblue commented on code in PR #6139: URL: https://github.com/apache/iceberg/pull/6139#discussion_r1024485376 ## python/pyiceberg/expressions/visitors.py: ## @@ -60,125 +60,127 @@ PrimitiveType, ) +R = TypeVar("R") Review Comment: Since `T` is no longer imported, thi

[GitHub] [iceberg] rdblue commented on a diff in pull request #6139: Python: Remove dataclasses

2022-11-16 Thread GitBox
rdblue commented on code in PR #6139: URL: https://github.com/apache/iceberg/pull/6139#discussion_r1024486151 ## python/pyiceberg/expressions/visitors.py: ## @@ -60,125 +60,127 @@ PrimitiveType, ) +R = TypeVar("R") -class BooleanExpressionVisitor(Generic[T], ABC): + +c

[GitHub] [iceberg] rdblue commented on a diff in pull request #6139: Python: Remove dataclasses

2022-11-16 Thread GitBox
rdblue commented on code in PR #6139: URL: https://github.com/apache/iceberg/pull/6139#discussion_r1024487574 ## python/tests/expressions/test_literals.py: ## @@ -398,8 +397,8 @@ def test_fixed_literal(): def test_binary_literal(): -bin_lit012 = literal(bytearray([0x00,

[GitHub] [iceberg] rdblue commented on a diff in pull request #6139: Python: Remove dataclasses

2022-11-16 Thread GitBox
rdblue commented on code in PR #6139: URL: https://github.com/apache/iceberg/pull/6139#discussion_r1024487825 ## python/pyproject.toml: ## @@ -55,6 +55,7 @@ pyyaml = "6.0.0" pydantic = "1.10.2" fsspec = "2022.10.0" + Review Comment: Nit: unnecessary whitespace change.

[GitHub] [iceberg] rdblue commented on a diff in pull request #6139: Python: Remove dataclasses

2022-11-16 Thread GitBox
rdblue commented on code in PR #6139: URL: https://github.com/apache/iceberg/pull/6139#discussion_r1024488952 ## python/tests/expressions/test_expressions.py: ## @@ -202,48 +178,36 @@ def test_notnan_bind_nonfloat(): assert NotNaN(Reference("i")).bind(schema) == AlwaysTrue(

[GitHub] [iceberg] rdblue commented on a diff in pull request #6139: Python: Remove dataclasses

2022-11-16 Thread GitBox
rdblue commented on code in PR #6139: URL: https://github.com/apache/iceberg/pull/6139#discussion_r1024489449 ## python/tests/expressions/test_expressions.py: ## @@ -254,15 +218,15 @@ def test_empty_bind_not_in(table_schema_simple: Schema): def test_bind_not_in_equal_term(tab

[GitHub] [iceberg] Fokko merged pull request #6197: Python: Fix rough edges around literals

2022-11-16 Thread GitBox
Fokko merged PR #6197: URL: https://github.com/apache/iceberg/pull/6197 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apach

[GitHub] [iceberg] Fokko commented on a diff in pull request #6139: Python: Remove dataclasses

2022-11-16 Thread GitBox
Fokko commented on code in PR #6139: URL: https://github.com/apache/iceberg/pull/6139#discussion_r1024505483 ## python/pyiceberg/expressions/__init__.py: ## @@ -194,85 +251,108 @@ def __new__(cls, child: BooleanExpression): return AlwaysTrue() elif isinstan

[GitHub] [iceberg] Fokko commented on a diff in pull request #6139: Python: Remove dataclasses

2022-11-16 Thread GitBox
Fokko commented on code in PR #6139: URL: https://github.com/apache/iceberg/pull/6139#discussion_r1024505659 ## python/pyiceberg/expressions/__init__.py: ## @@ -281,249 +361,313 @@ def __invert__(self) -> BoundIsNull: return BoundIsNull(self.term) -@dataclass(frozen

[GitHub] [iceberg] joao-parana closed issue #6164: The Literals class does not handle literals of type LocalDateTime. This causes errors in expressions involving Timestamp.

2022-11-16 Thread GitBox
joao-parana closed issue #6164: The Literals class does not handle literals of type LocalDateTime. This causes errors in expressions involving Timestamp. URL: https://github.com/apache/iceberg/issues/6164 -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [iceberg] joao-parana commented on issue #6164: The Literals class does not handle literals of type LocalDateTime. This causes errors in expressions involving Timestamp.

2022-11-16 Thread GitBox
joao-parana commented on issue #6164: URL: https://github.com/apache/iceberg/issues/6164#issuecomment-1317670657 Thanks @RussellSpitzer, I made the changes below and it worked. I'm closing the issue. ```java Long beginInterval = 1368023502001001L); Long endInterval= 17311714

[GitHub] [iceberg] Fokko commented on a diff in pull request #6139: Python: Remove dataclasses

2022-11-16 Thread GitBox
Fokko commented on code in PR #6139: URL: https://github.com/apache/iceberg/pull/6139#discussion_r1024521025 ## python/pyiceberg/expressions/__init__.py: ## @@ -281,249 +361,313 @@ def __invert__(self) -> BoundIsNull: return BoundIsNull(self.term) -@dataclass(frozen

[GitHub] [iceberg] Fokko commented on a diff in pull request #6139: Python: Remove dataclasses

2022-11-16 Thread GitBox
Fokko commented on code in PR #6139: URL: https://github.com/apache/iceberg/pull/6139#discussion_r1024521225 ## python/pyiceberg/expressions/__init__.py: ## @@ -281,249 +361,313 @@ def __invert__(self) -> BoundIsNull: return BoundIsNull(self.term) -@dataclass(frozen

[GitHub] [iceberg] Fokko commented on a diff in pull request #6139: Python: Remove dataclasses

2022-11-16 Thread GitBox
Fokko commented on code in PR #6139: URL: https://github.com/apache/iceberg/pull/6139#discussion_r1024522089 ## python/pyiceberg/expressions/__init__.py: ## @@ -281,249 +361,313 @@ def __invert__(self) -> BoundIsNull: return BoundIsNull(self.term) -@dataclass(frozen

[GitHub] [iceberg] Fokko commented on a diff in pull request #6139: Python: Remove dataclasses

2022-11-16 Thread GitBox
Fokko commented on code in PR #6139: URL: https://github.com/apache/iceberg/pull/6139#discussion_r1024523643 ## python/pyiceberg/expressions/__init__.py: ## @@ -18,18 +18,37 @@ from __future__ import annotations from abc import ABC, abstractmethod -from dataclasses import da

[GitHub] [iceberg] Fokko commented on a diff in pull request #6139: Python: Remove dataclasses

2022-11-16 Thread GitBox
Fokko commented on code in PR #6139: URL: https://github.com/apache/iceberg/pull/6139#discussion_r1024524079 ## python/pyiceberg/expressions/visitors.py: ## @@ -60,125 +60,127 @@ PrimitiveType, ) +R = TypeVar("R") Review Comment: Yes, that's true, I've reverted the c

[GitHub] [iceberg] Fokko commented on a diff in pull request #6139: Python: Remove dataclasses

2022-11-16 Thread GitBox
Fokko commented on code in PR #6139: URL: https://github.com/apache/iceberg/pull/6139#discussion_r1024524963 ## python/tests/expressions/test_literals.py: ## @@ -398,8 +397,8 @@ def test_fixed_literal(): def test_binary_literal(): -bin_lit012 = literal(bytearray([0x00,

[GitHub] [iceberg] Fokko commented on a diff in pull request #6139: Python: Remove dataclasses

2022-11-16 Thread GitBox
Fokko commented on code in PR #6139: URL: https://github.com/apache/iceberg/pull/6139#discussion_r1024528687 ## python/tests/expressions/test_expressions.py: ## @@ -254,15 +218,15 @@ def test_empty_bind_not_in(table_schema_simple: Schema): def test_bind_not_in_equal_term(tabl

[GitHub] [iceberg] Fokko commented on a diff in pull request #6139: Python: Remove dataclasses

2022-11-16 Thread GitBox
Fokko commented on code in PR #6139: URL: https://github.com/apache/iceberg/pull/6139#discussion_r1024530887 ## python/tests/expressions/test_expressions.py: ## @@ -202,48 +178,36 @@ def test_notnan_bind_nonfloat(): assert NotNaN(Reference("i")).bind(schema) == AlwaysTrue()

[GitHub] [iceberg-docs] Fokko merged pull request #177: Docs: Add 5 new blogs from Dremio

2022-11-16 Thread GitBox
Fokko merged PR #177: URL: https://github.com/apache/iceberg-docs/pull/177 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

[GitHub] [iceberg] Fokko commented on a diff in pull request #6139: Python: Remove dataclasses

2022-11-16 Thread GitBox
Fokko commented on code in PR #6139: URL: https://github.com/apache/iceberg/pull/6139#discussion_r1024536334 ## python/pyproject.toml: ## @@ -55,6 +55,8 @@ pyyaml = "6.0.0" pydantic = "1.10.2" fsspec = "2022.10.0" +typing-extensions = 'typing-extensions==4.4.0' Review Comme

[GitHub] [iceberg] Fokko commented on a diff in pull request #6139: Python: Remove dataclasses

2022-11-16 Thread GitBox
Fokko commented on code in PR #6139: URL: https://github.com/apache/iceberg/pull/6139#discussion_r1024536655 ## python/pyiceberg/expressions/literals.py: ## @@ -53,24 +49,27 @@ UUIDType, ) from pyiceberg.utils.datetime import ( -date_str_to_days, +date_str_to_date

[GitHub] [iceberg] RussellSpitzer commented on a diff in pull request #5376: Core: Add readable metrics columns to files metadata tables

2022-11-16 Thread GitBox
RussellSpitzer commented on code in PR #5376: URL: https://github.com/apache/iceberg/pull/5376#discussion_r1024565829 ## api/src/main/java/org/apache/iceberg/Schema.java: ## @@ -233,6 +233,16 @@ public Map getAliases() { return aliasToId; } + /** + * Returns a map f

[GitHub] [iceberg] islamismailov opened a new issue, #6204: Allow dropping a column used by old SortOrders

2022-11-16 Thread GitBox
islamismailov opened a new issue, #6204: URL: https://github.com/apache/iceberg/issues/6204 ### Apache Iceberg version 1.0.0 (latest release) ### Query engine Spark ### Please describe the bug 🐞 Here's my first go at a [PR for this](https://gist.github.com/

[GitHub] [iceberg] aokolnychyi commented on a diff in pull request #6199: API, Core: Move micros and days conversions to DateTimeUtil

2022-11-16 Thread GitBox
aokolnychyi commented on code in PR #6199: URL: https://github.com/apache/iceberg/pull/6199#discussion_r1024595729 ## api/src/main/java/org/apache/iceberg/util/DateTimeUtil.java: ## @@ -133,4 +134,56 @@ public static long isoTimestampToMicros(String timestampString) { retu

[GitHub] [iceberg] aokolnychyi commented on a diff in pull request #6199: API, Core: Move micros and days conversions to DateTimeUtil

2022-11-16 Thread GitBox
aokolnychyi commented on code in PR #6199: URL: https://github.com/apache/iceberg/pull/6199#discussion_r1024596919 ## api/src/main/java/org/apache/iceberg/util/DateTimeUtil.java: ## @@ -133,4 +134,56 @@ public static long isoTimestampToMicros(String timestampString) { retu

[GitHub] [iceberg] github-actions[bot] commented on issue #4363: DataFrameWriterV2 doesn't create partitions when using partitionedBy

2022-11-16 Thread GitBox
github-actions[bot] commented on issue #4363: URL: https://github.com/apache/iceberg/issues/4363#issuecomment-1317856331 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

[GitHub] [iceberg] github-actions[bot] commented on issue #4317: Flink - Investigate migrating deprecated usage of `TableSchema` to new `Schema` and `ResolvedSchema` classes

2022-11-16 Thread GitBox
github-actions[bot] commented on issue #4317: URL: https://github.com/apache/iceberg/issues/4317#issuecomment-1317856357 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

[GitHub] [iceberg] github-actions[bot] closed issue #4363: DataFrameWriterV2 doesn't create partitions when using partitionedBy

2022-11-16 Thread GitBox
github-actions[bot] closed issue #4363: DataFrameWriterV2 doesn't create partitions when using partitionedBy URL: https://github.com/apache/iceberg/issues/4363 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [iceberg] linfey90 commented on a diff in pull request #5824: Spark: support hilbert curve when rewrite

2022-11-16 Thread GitBox
linfey90 commented on code in PR #5824: URL: https://github.com/apache/iceberg/pull/5824#discussion_r1024671032 ## spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/actions/SparkSpaceCurveUDF.java: ## @@ -0,0 +1,45 @@ +/* + * Licensed to the Apache Software Foundation (ASF

[GitHub] [iceberg] eemario opened a new issue, #6205: Using iceberg-parquet and iceberg-spark-runtime throws ClassCastException for MessageType

2022-11-16 Thread GitBox
eemario opened a new issue, #6205: URL: https://github.com/apache/iceberg/issues/6205 ### Query engine Spark ### Question When I am using iceberg-parquet to perform write to iceberg table in my source code and using iceberg-spark-runtime to validate table content in unit

[GitHub] [iceberg] stevenzwu opened a new pull request, #6206: Extract Flink package version programmatically for EnvironmentContext…

2022-11-16 Thread GitBox
stevenzwu opened a new pull request, #6206: URL: https://github.com/apache/iceberg/pull/6206 … engine-version This is to address hard-coded Flink version from PR #6184. Hard-code value can be very tricky to detect and fix when we copy the 1.16 module to 1.17 module in the future.

[GitHub] [iceberg] ConeyLiu commented on a diff in pull request #4627: Parquet: Fixes get null values for the nested field partition column

2022-11-16 Thread GitBox
ConeyLiu commented on code in PR #4627: URL: https://github.com/apache/iceberg/pull/4627#discussion_r1024727941 ## parquet/src/main/java/org/apache/iceberg/data/parquet/BaseParquetReaders.java: ## @@ -149,11 +153,14 @@ public ParquetValueReader struct(Types.StructType expected,

[GitHub] [iceberg] ConeyLiu commented on a diff in pull request #4627: Parquet: Fixes get null values for the nested field partition column

2022-11-16 Thread GitBox
ConeyLiu commented on code in PR #4627: URL: https://github.com/apache/iceberg/pull/4627#discussion_r1024728230 ## parquet/src/main/java/org/apache/iceberg/parquet/ParquetValueReaders.java: ## @@ -116,9 +120,45 @@ public void setPageSource(PageReadStore pageStore, long rowPosit

[GitHub] [iceberg] ConeyLiu commented on pull request #4627: Parquet: Fixes get null values for the nested field partition column

2022-11-16 Thread GitBox
ConeyLiu commented on PR #4627: URL: https://github.com/apache/iceberg/pull/4627#issuecomment-1318032973 Thanks @szehon-ho @pvary for the review. > May I ask you to put the fix in the main branches for Spark (3.3) and Flink (1.16) first, and then with another PR we can backport to all

[GitHub] [iceberg] tbinoy commented on issue #6186: Iceberg table creation fails in AWS Glue

2022-11-16 Thread GitBox
tbinoy commented on issue #6186: URL: https://github.com/apache/iceberg/issues/6186#issuecomment-1318117329 I am using Presto 0.277 to connect create/ insert iceberg tables in AWS Glue. I am able to run select query for iceberg tables in glue but not able to do insert or table creation.

[GitHub] [iceberg] aokolnychyi commented on a diff in pull request #6199: API, Core: Move micros and days conversions to DateTimeUtil

2022-11-16 Thread GitBox
aokolnychyi commented on code in PR #6199: URL: https://github.com/apache/iceberg/pull/6199#discussion_r1024786018 ## api/src/main/java/org/apache/iceberg/util/DateTimeUtil.java: ## @@ -133,4 +134,56 @@ public static long isoTimestampToMicros(String timestampString) { retu

[GitHub] [iceberg] flyrain commented on a diff in pull request #6012: Spark 3.3: Add a procedure to generate table changes

2022-11-16 Thread GitBox
flyrain commented on code in PR #6012: URL: https://github.com/apache/iceberg/pull/6012#discussion_r1024835530 ## spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/procedures/GenerateChangesProcedure.java: ## @@ -0,0 +1,210 @@ +/* + * Licensed to the Apache Software Founda

[GitHub] [iceberg] Fokko commented on issue #6204: Allow dropping a column used by old SortOrders

2022-11-16 Thread GitBox
Fokko commented on issue #6204: URL: https://github.com/apache/iceberg/issues/6204#issuecomment-1318209724 Thanks for opening this PR @islamismailov. We had something similar for the partition-spec: https://github.com/apache/iceberg/issues/5676 What we did there was only look up the columns

[GitHub] [iceberg] aokolnychyi merged pull request #6199: API, Core: Move micros and days conversions to DateTimeUtil

2022-11-16 Thread GitBox
aokolnychyi merged PR #6199: URL: https://github.com/apache/iceberg/pull/6199 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

[GitHub] [iceberg] aokolnychyi commented on a diff in pull request #6199: API, Core: Move micros and days conversions to DateTimeUtil

2022-11-16 Thread GitBox
aokolnychyi commented on code in PR #6199: URL: https://github.com/apache/iceberg/pull/6199#discussion_r1024845392 ## api/src/main/java/org/apache/iceberg/util/DateTimeUtil.java: ## @@ -133,4 +134,56 @@ public static long isoTimestampToMicros(String timestampString) { retu

[GitHub] [iceberg] aokolnychyi commented on pull request #6199: API, Core: Move micros and days conversions to DateTimeUtil

2022-11-16 Thread GitBox
aokolnychyi commented on PR #6199: URL: https://github.com/apache/iceberg/pull/6199#issuecomment-1318216016 Thanks for reviewing, @nastra @szehon-ho @gaborkaszab! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [iceberg] aokolnychyi opened a new pull request, #6207: Spark 3.3: Add years transform function

2022-11-16 Thread GitBox
aokolnychyi opened a new pull request, #6207: URL: https://github.com/apache/iceberg/pull/6207 This PR adds `years` transform function to Spark 3.3. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [iceberg] aokolnychyi commented on pull request #6207: Spark 3.3: Add years transform function

2022-11-16 Thread GitBox
aokolnychyi commented on PR #6207: URL: https://github.com/apache/iceberg/pull/6207#issuecomment-1318219571 cc @kbendick @rdblue @RussellSpitzer @szehon-ho @flyrain @gaborkaszab @nastra -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [iceberg] nastra commented on issue #6205: Using iceberg-parquet and iceberg-spark-runtime throws ClassCastException for MessageType

2022-11-16 Thread GitBox
nastra commented on issue #6205: URL: https://github.com/apache/iceberg/issues/6205#issuecomment-1318227196 There have been some fixes in the past around Spark + Parquet (https://github.com/apache/iceberg/pull/4667), so would be interesting to know which Iceberg + Spark version you're using

[GitHub] [iceberg] Fokko commented on issue #6194: Spark DDL: upgrade format-version from v1 to v2 is not working

2022-11-16 Thread GitBox
Fokko commented on issue #6194: URL: https://github.com/apache/iceberg/issues/6194#issuecomment-1318232667 It works fine with Spark: ![image](https://user-images.githubusercontent.com/1134248/202388833-2935c614-634a-4389-9265-4232022f1f46.png) In Hive, could it be that you need

[GitHub] [iceberg] fortarch commented on issue #5993: Quick start docker-compose demo doesn't work

2022-11-17 Thread GitBox
fortarch commented on issue #5993: URL: https://github.com/apache/iceberg/issues/5993#issuecomment-1318234553 get exception, please resolve that. ` Caused by: java.net.UnknownHostException: rest: Name or service not known ` -- This is an automated message from the Apache Git Servi

[GitHub] [iceberg] 0xffmeta commented on issue #6194: Spark DDL: upgrade format-version from v1 to v2 is not working

2022-11-17 Thread GitBox
0xffmeta commented on issue #6194: URL: https://github.com/apache/iceberg/issues/6194#issuecomment-1318240764 Seems I need to use `alter table tmp.table SET TBLPROPERTIES('option.format-version'='2')` to update the format-version. But I'm not very sure why it required prefix `option.`. --

[GitHub] [iceberg] nastra commented on issue #5993: Quick start docker-compose demo doesn't work

2022-11-17 Thread GitBox
nastra commented on issue #5993: URL: https://github.com/apache/iceberg/issues/5993#issuecomment-1318267420 This should be fixed by https://github.com/apache/iceberg-docs/pull/178. Until the docs have been updated on the [website](https://iceberg.apache.org/spark-quickstart/), you could use

[GitHub] [iceberg] gaborkaszab commented on pull request #6045: [iceberg-hive-metastore] Support setting individual and group ownership for Namespace

2022-11-17 Thread GitBox
gaborkaszab commented on PR #6045: URL: https://github.com/apache/iceberg/pull/6045#issuecomment-1318275597 > @gaborkaszab sorry i know we are super close, but do you think we could change both to use UserGroupInformation quickly before cutting final 1.1 RC? That way no need to change behav

[GitHub] [iceberg] szehon-ho commented on a diff in pull request #5376: Core: Add readable metrics columns to files metadata tables

2022-11-17 Thread GitBox
szehon-ho commented on code in PR #5376: URL: https://github.com/apache/iceberg/pull/5376#discussion_r1024931871 ## api/src/main/java/org/apache/iceberg/Schema.java: ## @@ -233,6 +233,16 @@ public Map getAliases() { return aliasToId; } + /** + * Returns a map for th

[GitHub] [iceberg] szehon-ho commented on pull request #6045: [iceberg-hive-metastore] Support setting individual and group ownership for Namespace

2022-11-17 Thread GitBox
szehon-ho commented on PR #6045: URL: https://github.com/apache/iceberg/pull/6045#issuecomment-1318326348 Thought so, no worries :), thought it was worth a try -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

[GitHub] [iceberg] itaise commented on issue #4510: Spark overwrite issue when partition changes

2022-11-17 Thread GitBox
itaise commented on issue #4510: URL: https://github.com/apache/iceberg/issues/4510#issuecomment-1318450967 Hi :) any update regarding that? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

[GitHub] [iceberg] Gajai opened a new issue, #6208: Flink version Support Issue wrt IceBerg

2022-11-17 Thread GitBox
Gajai opened a new issue, #6208: URL: https://github.com/apache/iceberg/issues/6208 ### Query engine _No response_ ### Question Latest Flink Version Supported is is 1.11.x only? Do we have any upcoming possibilities for flink 1.16.x version anytime soon? -- This is a

[GitHub] [iceberg] szehon-ho closed pull request #5376: Core: Add readable metrics columns to files metadata tables

2022-11-17 Thread GitBox
szehon-ho closed pull request #5376: Core: Add readable metrics columns to files metadata tables URL: https://github.com/apache/iceberg/pull/5376 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [iceberg] nastra commented on issue #6208: Flink version Support Issue wrt IceBerg

2022-11-17 Thread GitBox
nastra commented on issue #6208: URL: https://github.com/apache/iceberg/issues/6208#issuecomment-1318491097 Note that Iceberg 1.0.0 supports Flink 1.15.0. The project is about to ship Iceberg 1.1.0 in the next few days, which will include Flink 1.16.0 support. -- This is an automated mes

[GitHub] [iceberg] luoyuxia commented on issue #6208: Flink version Support Issue wrt IceBerg

2022-11-17 Thread GitBox
luoyuxia commented on issue #6208: URL: https://github.com/apache/iceberg/issues/6208#issuecomment-1318490882 The comming iceberg 1.1.0 which is in preparing release will support flink 1.16.x. -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [iceberg] nastra closed issue #6208: Flink version Support Issue wrt IceBerg

2022-11-17 Thread GitBox
nastra closed issue #6208: Flink version Support Issue wrt IceBerg URL: https://github.com/apache/iceberg/issues/6208 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubsc

[GitHub] [iceberg] nastra commented on issue #6208: Flink version Support Issue wrt IceBerg

2022-11-17 Thread GitBox
nastra commented on issue #6208: URL: https://github.com/apache/iceberg/issues/6208#issuecomment-1318500305 @Gajai I hope that answered your question. Please re-open the issue if not. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to G

[GitHub] [iceberg] szehon-ho commented on a diff in pull request #5376: Core: Add readable metrics columns to files metadata tables

2022-11-17 Thread GitBox
szehon-ho commented on code in PR #5376: URL: https://github.com/apache/iceberg/pull/5376#discussion_r1024931871 ## api/src/main/java/org/apache/iceberg/Schema.java: ## @@ -233,6 +233,16 @@ public Map getAliases() { return aliasToId; } + /** + * Returns a map for th

[GitHub] [iceberg] Fokko commented on a diff in pull request #5338: Spec: Inconsistency around files_count

2022-11-17 Thread GitBox
Fokko commented on code in PR #5338: URL: https://github.com/apache/iceberg/pull/5338#discussion_r1025186972 ## api/src/main/java/org/apache/iceberg/ManifestFile.java: ## @@ -52,11 +52,11 @@ public interface ManifestFile { optional( 503, "added_snapshot_id", Ty

[GitHub] [iceberg] Fokko closed pull request #5338: Spec: Inconsistency around files_count

2022-11-17 Thread GitBox
Fokko closed pull request #5338: Spec: Inconsistency around files_count URL: https://github.com/apache/iceberg/pull/5338 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsu

[GitHub] [iceberg] Fokko opened a new pull request, #5338: Spec: Inconsistency around files_count

2022-11-17 Thread GitBox
Fokko opened a new pull request, #5338: URL: https://github.com/apache/iceberg/pull/5338 When building the Manifest mappers for Python, @rdblue noticed that the `added_data_files_count` should be `added_files_count` according to the spec. However, this field is written in Java as `a

[GitHub] [iceberg] Fokko commented on issue #6204: Allow dropping a column used by old SortOrders

2022-11-17 Thread GitBox
Fokko commented on issue #6204: URL: https://github.com/apache/iceberg/issues/6204#issuecomment-1318656478 @islamismailov I'm just seeing your gist. I think it looks good, except one copy-paste: ``` + if (defaultSortOrderId == null) { +throw new IllegalArgumentException

[GitHub] [iceberg] pvary commented on pull request #6175: Hive: Add UGI to the key in CachedClientPool

2022-11-17 Thread GitBox
pvary commented on PR #6175: URL: https://github.com/apache/iceberg/pull/6175#issuecomment-1318658845 @lirui-apache: I am still a bit concerned that in Hive or Impala this would cause issues by not reusing the client pools. @gaborkaszab: Could you please check from Hive/Impala side? Th

[GitHub] [iceberg] pvary commented on pull request #6175: Hive: Add UGI to the key in CachedClientPool

2022-11-17 Thread GitBox
pvary commented on PR #6175: URL: https://github.com/apache/iceberg/pull/6175#issuecomment-1318660031 @lirui-apache: Ohh, and big thanks for the detailed check! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[GitHub] [iceberg] ELHoussineT commented on issue #6105: Add tblproperties using alter table with spark

2022-11-17 Thread GitBox
ELHoussineT commented on issue #6105: URL: https://github.com/apache/iceberg/issues/6105#issuecomment-1318661985 @AlvaroNavarroMora Did it work for you? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [iceberg] ajantha-bhat commented on a diff in pull request #5338: Spec: Inconsistency around files_count

2022-11-17 Thread GitBox
ajantha-bhat commented on code in PR #5338: URL: https://github.com/apache/iceberg/pull/5338#discussion_r1025216240 ## api/src/main/java/org/apache/iceberg/ManifestFile.java: ## @@ -52,11 +52,11 @@ public interface ManifestFile { optional( 503, "added_snapshot_

[GitHub] [iceberg] Fokko merged pull request #6178: Core: Remove redundant initialization

2022-11-17 Thread GitBox
Fokko merged PR #6178: URL: https://github.com/apache/iceberg/pull/6178 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apach

[GitHub] [iceberg] pvary merged pull request #6206: Extract Flink package version programmatically for EnvironmentContext…

2022-11-17 Thread GitBox
pvary merged PR #6206: URL: https://github.com/apache/iceberg/pull/6206 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apach

[GitHub] [iceberg] ELHoussineT commented on pull request #6034: Python: GlueCatalog Full Implementation

2022-11-17 Thread GitBox
ELHoussineT commented on PR #6034: URL: https://github.com/apache/iceberg/pull/6034#issuecomment-1318711725 Any estimated time when this will be merged? Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [iceberg] szehon-ho commented on a diff in pull request #4627: Parquet: Fixes get null values for the nested field partition column

2022-11-17 Thread GitBox
szehon-ho commented on code in PR #4627: URL: https://github.com/apache/iceberg/pull/4627#discussion_r1025283707 ## parquet/src/main/java/org/apache/iceberg/data/parquet/BaseParquetReaders.java: ## @@ -149,11 +153,14 @@ public ParquetValueReader struct(Types.StructType expected

[GitHub] [iceberg] szehon-ho commented on a diff in pull request #4627: Parquet: Fixes get null values for the nested field partition column

2022-11-17 Thread GitBox
szehon-ho commented on code in PR #4627: URL: https://github.com/apache/iceberg/pull/4627#discussion_r1025283707 ## parquet/src/main/java/org/apache/iceberg/data/parquet/BaseParquetReaders.java: ## @@ -149,11 +153,14 @@ public ParquetValueReader struct(Types.StructType expected

[GitHub] [iceberg] szehon-ho commented on a diff in pull request #6045: [iceberg-hive-metastore] Support setting individual and group ownership for Namespace

2022-11-17 Thread GitBox
szehon-ho commented on code in PR #6045: URL: https://github.com/apache/iceberg/pull/6045#discussion_r1025370462 ## hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java: ## @@ -365,6 +374,13 @@ public boolean dropNamespace(Namespace namespace) { @Override

[GitHub] [iceberg] szehon-ho commented on a diff in pull request #6045: [iceberg-hive-metastore] Support setting individual and group ownership for Namespace

2022-11-17 Thread GitBox
szehon-ho commented on code in PR #6045: URL: https://github.com/apache/iceberg/pull/6045#discussion_r1025379199 ## hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java: ## @@ -518,11 +541,26 @@ private Map convertToMetadata(Database database) { if (databa

[GitHub] [iceberg] szehon-ho commented on a diff in pull request #6045: [iceberg-hive-metastore] Support setting individual and group ownership for Namespace

2022-11-17 Thread GitBox
szehon-ho commented on code in PR #6045: URL: https://github.com/apache/iceberg/pull/6045#discussion_r1025379199 ## hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java: ## @@ -518,11 +541,26 @@ private Map convertToMetadata(Database database) { if (databa

[GitHub] [iceberg] szehon-ho commented on a diff in pull request #6045: [iceberg-hive-metastore] Support setting individual and group ownership for Namespace

2022-11-17 Thread GitBox
szehon-ho commented on code in PR #6045: URL: https://github.com/apache/iceberg/pull/6045#discussion_r1025379199 ## hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java: ## @@ -518,11 +541,26 @@ private Map convertToMetadata(Database database) { if (databa

[GitHub] [iceberg] szehon-ho commented on a diff in pull request #6045: [iceberg-hive-metastore] Support setting individual and group ownership for Namespace

2022-11-17 Thread GitBox
szehon-ho commented on code in PR #6045: URL: https://github.com/apache/iceberg/pull/6045#discussion_r1025379199 ## hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java: ## @@ -518,11 +541,26 @@ private Map convertToMetadata(Database database) { if (databa

[GitHub] [iceberg] szehon-ho commented on a diff in pull request #6045: [iceberg-hive-metastore] Support setting individual and group ownership for Namespace

2022-11-17 Thread GitBox
szehon-ho commented on code in PR #6045: URL: https://github.com/apache/iceberg/pull/6045#discussion_r1025379199 ## hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java: ## @@ -518,11 +541,26 @@ private Map convertToMetadata(Database database) { if (databa

[GitHub] [iceberg] stevenzwu commented on pull request #6206: Extract Flink package version programmatically for EnvironmentContext…

2022-11-17 Thread GitBox
stevenzwu commented on PR #6206: URL: https://github.com/apache/iceberg/pull/6206#issuecomment-1318867484 @Fokko Yes, I was thinking about adding a unit test with assertion. I had it locally but was on the fence. - pro: it can ensure the correctness and address @nastra 's concern. Re

[GitHub] [iceberg] Fokko commented on issue #5993: Quick start docker-compose demo doesn't work

2022-11-17 Thread GitBox
Fokko commented on issue #5993: URL: https://github.com/apache/iceberg/issues/5993#issuecomment-1318904274 The latest version of the image has been pushed. Please run `docker-compose pull` to fetch the latest images. Let me know if this works 👍🏻 -- This is an automated message from the A

[GitHub] [iceberg] szehon-ho opened a new issue, #6209: testDeleteWithSnapshotIsolation is flaky

2022-11-17 Thread GitBox
szehon-ho opened a new issue, #6209: URL: https://github.com/apache/iceberg/issues/6209 ### Apache Iceberg version main (development) ### Query engine _No response_ ### Please describe the bug 🐞 ``` org.apache.iceberg.spark.extensions.TestCopyOnWriteDelet

[GitHub] [iceberg] RussellSpitzer commented on a diff in pull request #6207: Spark 3.3: Add years transform function

2022-11-17 Thread GitBox
RussellSpitzer commented on code in PR #6207: URL: https://github.com/apache/iceberg/pull/6207#discussion_r1025443474 ## spark/v3.3/spark/src/test/java/org/apache/iceberg/spark/sql/TestSparkYearsFunction.java: ## @@ -0,0 +1,116 @@ +/* + * Licensed to the Apache Software Foundati

[GitHub] [iceberg] nastra commented on pull request #6206: Extract Flink package version programmatically for EnvironmentContext…

2022-11-17 Thread GitBox
nastra commented on PR #6206: URL: https://github.com/apache/iceberg/pull/6206#issuecomment-1318927977 I'm in favor of adding the unit test. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

[GitHub] [iceberg] RussellSpitzer commented on a diff in pull request #6207: Spark 3.3: Add years transform function

2022-11-17 Thread GitBox
RussellSpitzer commented on code in PR #6207: URL: https://github.com/apache/iceberg/pull/6207#discussion_r1025452084 ## spark/v3.3/spark/src/test/java/org/apache/iceberg/spark/sql/TestSparkYearsFunction.java: ## @@ -0,0 +1,116 @@ +/* + * Licensed to the Apache Software Foundati

[GitHub] [iceberg] RussellSpitzer commented on a diff in pull request #6207: Spark 3.3: Add years transform function

2022-11-17 Thread GitBox
RussellSpitzer commented on code in PR #6207: URL: https://github.com/apache/iceberg/pull/6207#discussion_r1025453770 ## spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/functions/YearsFunction.java: ## @@ -0,0 +1,129 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [iceberg] ajantha-bhat commented on issue #6209: Spark: TestDeleteWithSnapshotIsolation fails randomly

2022-11-17 Thread GitBox
ajantha-bhat commented on issue #6209: URL: https://github.com/apache/iceberg/issues/6209#issuecomment-1318931068 Duplicate of https://github.com/apache/iceberg/issues/6060 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[GitHub] [iceberg] RussellSpitzer commented on a diff in pull request #6207: Spark 3.3: Add years transform function

2022-11-17 Thread GitBox
RussellSpitzer commented on code in PR #6207: URL: https://github.com/apache/iceberg/pull/6207#discussion_r1025458116 ## spark/v3.3/spark/src/test/java/org/apache/iceberg/spark/sql/TestSparkYearsFunction.java: ## @@ -0,0 +1,116 @@ +/* + * Licensed to the Apache Software Foundati

[GitHub] [iceberg] islamismailov commented on issue #6204: Allow dropping a column used by old SortOrders

2022-11-17 Thread GitBox
islamismailov commented on issue #6204: URL: https://github.com/apache/iceberg/issues/6204#issuecomment-1318941894 Ok thank you, yes I noticed that problem just before sending it out. Can I send a PR? Or do I need to get on an approved list of committers first? -- This is an automated mes

[GitHub] [iceberg] RussellSpitzer commented on a diff in pull request #5376: Core: Add readable metrics columns to files metadata tables

2022-11-17 Thread GitBox
RussellSpitzer commented on code in PR #5376: URL: https://github.com/apache/iceberg/pull/5376#discussion_r1025472556 ## api/src/main/java/org/apache/iceberg/Schema.java: ## @@ -233,6 +233,16 @@ public Map getAliases() { return aliasToId; } + /** + * Returns a map f

[GitHub] [iceberg] Fokko commented on issue #6204: Allow dropping a column used by old SortOrders

2022-11-17 Thread GitBox
Fokko commented on issue #6204: URL: https://github.com/apache/iceberg/issues/6204#issuecomment-1318969998 Feel free to open up a PR 👍🏻 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

[GitHub] [iceberg] haizhou-zhao commented on a diff in pull request #6045: [iceberg-hive-metastore] Support setting individual and group ownership for Namespace

2022-11-17 Thread GitBox
haizhou-zhao commented on code in PR #6045: URL: https://github.com/apache/iceberg/pull/6045#discussion_r1025502497 ## hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java: ## @@ -365,6 +374,13 @@ public boolean dropNamespace(Namespace namespace) { @Overrid

[GitHub] [iceberg-docs] rdblue merged pull request #178: Update Spark+Iceberg quickstart docker-compose example

2022-11-17 Thread GitBox
rdblue merged PR #178: URL: https://github.com/apache/iceberg-docs/pull/178 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.a

[GitHub] [iceberg-docs] rdblue commented on pull request #175: Docs: Update spark-3.0 removal

2022-11-17 Thread GitBox
rdblue commented on PR #175: URL: https://github.com/apache/iceberg-docs/pull/175#issuecomment-1319018934 @ajantha-bhat can you fix the conflict? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [iceberg-docs] rdblue merged pull request #174: Fix spark runtime jar link

2022-11-17 Thread GitBox
rdblue merged PR #174: URL: https://github.com/apache/iceberg-docs/pull/174 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.a

<    17   18   19   20   21   22   23   24   25   26   >