[GitHub] [iceberg] pvary commented on issue #6067: exec insert into (hive on spark),no erro log,but table no data

pvary commented on issue #6067: URL: https://github.com/apache/iceberg/issues/6067#issuecomment-1305645778 Hive on Spark is not supported/tested Also Hive 2.1.1 is also not supported/tested Could you use newer CDH/CDP version? -- This is an automated message from the Apache Git Servi

[GitHub] [iceberg] findepi commented on pull request #6090: Core: Handle statistics file clean up from expireSnapshots

findepi commented on PR #6090: URL: https://github.com/apache/iceberg/pull/6090#issuecomment-1305652072 I think we should change the label of the `snapshot-id` entry in https://iceberg.apache.org/spec/#table-statistics (to level, not blob level) -- This is an automated message from the Ap

[GitHub] [iceberg] Fokko merged pull request #6135: Python: Use pythonic `len()` built-in instead of `length` property

Fokko merged PR #6135: URL: https://github.com/apache/iceberg/pull/6135 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apach

[GitHub] [iceberg] pvary commented on issue #6071: Should ClientPool consider UGI when reusing a connection?

pvary commented on issue #6071: URL: https://github.com/apache/iceberg/issues/6071#issuecomment-1305705704 Sorry for the delay - I was OOO. Since we are reusing the HMSClient objects, we can face authorization issues if we are using Catalogs with multi-tenant scenarios. We were briefl

[GitHub] [iceberg] Fokko commented on a diff in pull request #6127: Python: Add expression evaluator

Fokko commented on code in PR #6127: URL: https://github.com/apache/iceberg/pull/6127#discussion_r1015073328 ## python/pyiceberg/expressions/visitors.py: ## @@ -417,6 +422,75 @@ def visit_bound_predicate(self, predicate) -> BooleanExpression: return predicate +def

[GitHub] [iceberg] Fokko commented on a diff in pull request #6127: Python: Add expression evaluator

Fokko commented on code in PR #6127: URL: https://github.com/apache/iceberg/pull/6127#discussion_r1015499369 ## python/pyiceberg/expressions/__init__.py: ## @@ -281,6 +289,30 @@ def __invert__(self) -> BoundIsNull: return BoundIsNull(self.term) +def coerce_unary_arg

[GitHub] [iceberg] Fokko commented on a diff in pull request #6139: Python: Remove dataclass

Fokko commented on code in PR #6139: URL: https://github.com/apache/iceberg/pull/6139#discussion_r1015502404 ## python/tests/expressions/test_visitors.py: ## @@ -1647,73 +1641,6 @@ def test_manifest_evaluator_or(): assert _create_manifest_evaluator(expr).eval(manifest)

[GitHub] [iceberg] Fokko commented on a diff in pull request #6139: Python: Remove dataclass

Fokko commented on code in PR #6139: URL: https://github.com/apache/iceberg/pull/6139#discussion_r1015503114 ## python/tests/expressions/test_expressions.py: ## @@ -73,27 +65,10 @@ NestedField, StringType, ) +from tests.conftest import FooStruct from tests.expression

[GitHub] [iceberg] Fokko commented on a diff in pull request #6139: Python: Remove dataclass

Fokko commented on code in PR #6139: URL: https://github.com/apache/iceberg/pull/6139#discussion_r1015503864 ## python/pyiceberg/expressions/literals.py: ## @@ -439,6 +439,9 @@ def _(self, type_var: DecimalType) -> Optional[Literal[Decimal]]: else: return

[GitHub] [iceberg] dimas-b commented on a diff in pull request #6134: Spec: Add query-column-names to SQL view representation in view spec

dimas-b commented on code in PR #6134: URL: https://github.com/apache/iceberg/pull/6134#discussion_r1015497935 ## format/view-spec.md: ## @@ -116,11 +116,19 @@ This type of representation stores the original view definition in SQL and its S | Optional | schema-id | ID of the v

[GitHub] [iceberg] haizhou-zhao commented on a diff in pull request #6045: [iceberg-hive-metastore] Support setting individual and group ownership for Namespace

haizhou-zhao commented on code in PR #6045: URL: https://github.com/apache/iceberg/pull/6045#discussion_r1015549849 ## core/src/main/java/org/apache/iceberg/TableProperties.java: ## @@ -360,5 +360,7 @@ private TableProperties() {} public static final String UPSERT_ENABLED = "

[GitHub] [iceberg] aokolnychyi commented on a diff in pull request #2276: Core: Add option to combine tasks by partition

aokolnychyi commented on code in PR #2276: URL: https://github.com/apache/iceberg/pull/2276#discussion_r1015562517 ## core/src/main/java/org/apache/iceberg/util/TableScanUtil.java: ## @@ -71,6 +78,57 @@ public static CloseableIterable splitFiles( return CloseableIterable.co

[GitHub] [iceberg] dhruv-pratap commented on a diff in pull request #6069: Python: TableScan Plan files API implementation without residual evaluation

dhruv-pratap commented on code in PR #6069: URL: https://github.com/apache/iceberg/pull/6069#discussion_r1015576539 ## python/pyiceberg/table/scan.py: ## @@ -0,0 +1,103 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. S

[GitHub] [iceberg] dhruv-pratap commented on a diff in pull request #6069: Python: TableScan Plan files API implementation without residual evaluation

dhruv-pratap commented on code in PR #6069: URL: https://github.com/apache/iceberg/pull/6069#discussion_r1015579816 ## python/tests/table/test_scan.py: ## @@ -0,0 +1,177 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements.

[GitHub] [iceberg] RussellSpitzer commented on a diff in pull request #5376: Core: Add readable metrics columns to files metadata tables

RussellSpitzer commented on code in PR #5376: URL: https://github.com/apache/iceberg/pull/5376#discussion_r1015582931 ## api/src/main/java/org/apache/iceberg/DataFile.java: ## @@ -99,10 +99,24 @@ public interface DataFile extends ContentFile { optional(140, "sort_order_id

[GitHub] [iceberg] RussellSpitzer commented on a diff in pull request #5376: Core: Add readable metrics columns to files metadata tables

RussellSpitzer commented on code in PR #5376: URL: https://github.com/apache/iceberg/pull/5376#discussion_r1015583682 ## api/src/main/java/org/apache/iceberg/DataFile.java: ## @@ -99,10 +99,24 @@ public interface DataFile extends ContentFile { optional(140, "sort_order_id

[GitHub] [iceberg] szehon-ho commented on a diff in pull request #5376: Core: Add readable metrics columns to files metadata tables

szehon-ho commented on code in PR #5376: URL: https://github.com/apache/iceberg/pull/5376#discussion_r1015598536 ## api/src/main/java/org/apache/iceberg/DataFile.java: ## @@ -99,10 +99,24 @@ public interface DataFile extends ContentFile { optional(140, "sort_order_id", In

[GitHub] [iceberg] szehon-ho commented on a diff in pull request #5376: Core: Add readable metrics columns to files metadata tables

szehon-ho commented on code in PR #5376: URL: https://github.com/apache/iceberg/pull/5376#discussion_r1015598536 ## api/src/main/java/org/apache/iceberg/DataFile.java: ## @@ -99,10 +99,24 @@ public interface DataFile extends ContentFile { optional(140, "sort_order_id", In

[GitHub] [iceberg] aokolnychyi commented on a diff in pull request #2276: Core: Add option to combine tasks by partition

aokolnychyi commented on code in PR #2276: URL: https://github.com/apache/iceberg/pull/2276#discussion_r1015562517 ## core/src/main/java/org/apache/iceberg/util/TableScanUtil.java: ## @@ -71,6 +78,57 @@ public static CloseableIterable splitFiles( return CloseableIterable.co

[GitHub] [iceberg] szehon-ho commented on a diff in pull request #5376: Core: Add readable metrics columns to files metadata tables

szehon-ho commented on code in PR #5376: URL: https://github.com/apache/iceberg/pull/5376#discussion_r1015603424 ## api/src/main/java/org/apache/iceberg/DataFile.java: ## @@ -99,10 +99,24 @@ public interface DataFile extends ContentFile { optional(140, "sort_order_id", In

[GitHub] [iceberg] szehon-ho commented on a diff in pull request #5376: Core: Add readable metrics columns to files metadata tables

szehon-ho commented on code in PR #5376: URL: https://github.com/apache/iceberg/pull/5376#discussion_r1015603424 ## api/src/main/java/org/apache/iceberg/DataFile.java: ## @@ -99,10 +99,24 @@ public interface DataFile extends ContentFile { optional(140, "sort_order_id", In

[GitHub] [iceberg] szehon-ho commented on a diff in pull request #5376: Core: Add readable metrics columns to files metadata tables

szehon-ho commented on code in PR #5376: URL: https://github.com/apache/iceberg/pull/5376#discussion_r1015605206 ## spark/v3.3/spark/src/test/java/org/apache/iceberg/spark/source/TestMetadataTableReadableMetrics.java: ## @@ -0,0 +1,498 @@ +/* + * Licensed to the Apache Software

[GitHub] [iceberg] szehon-ho commented on a diff in pull request #5376: Core: Add readable metrics columns to files metadata tables

szehon-ho commented on code in PR #5376: URL: https://github.com/apache/iceberg/pull/5376#discussion_r1015605561 ## spark/v3.3/spark/src/test/java/org/apache/iceberg/spark/source/TestMetadataTableReadableMetrics.java: ## @@ -0,0 +1,498 @@ +/* + * Licensed to the Apache Software

[GitHub] [iceberg] szehon-ho commented on a diff in pull request #5376: Core: Add readable metrics columns to files metadata tables

szehon-ho commented on code in PR #5376: URL: https://github.com/apache/iceberg/pull/5376#discussion_r1015603424 ## api/src/main/java/org/apache/iceberg/DataFile.java: ## @@ -99,10 +99,24 @@ public interface DataFile extends ContentFile { optional(140, "sort_order_id", In

[GitHub] [iceberg] szehon-ho commented on a diff in pull request #5376: Core: Add readable metrics columns to files metadata tables

szehon-ho commented on code in PR #5376: URL: https://github.com/apache/iceberg/pull/5376#discussion_r1015604784 ## spark/v3.3/spark/src/test/java/org/apache/iceberg/spark/source/TestMetadataTableReadableMetrics.java: ## @@ -0,0 +1,498 @@ +/* + * Licensed to the Apache Software

[GitHub] [iceberg] haizhou-zhao commented on a diff in pull request #6045: [iceberg-hive-metastore] Support setting individual and group ownership for Namespace

haizhou-zhao commented on code in PR #6045: URL: https://github.com/apache/iceberg/pull/6045#discussion_r1013067243 ## hive-metastore/src/test/java/org/apache/iceberg/hive/TestHiveCatalog.java: ## @@ -448,6 +540,36 @@ public void testRemoveNamespaceProperties() throws TExceptio

[GitHub] [iceberg] ahshahid commented on issue #6039: Spark : Perf enhancement by leveraging Dynamic Partition Pruning rule of spark for non partition columns used as join condition

ahshahid commented on issue #6039: URL: https://github.com/apache/iceberg/issues/6039#issuecomment-1305869163 @rdblue @aokolnychyi is there any particular reason why colStats in TableContext is by default false ? With this flag false, for non partition cols the bounds are not being written

[GitHub] [iceberg] ahshahid commented on issue #6039: Spark : Perf enhancement by leveraging Dynamic Partition Pruning rule of spark for non partition columns used as join condition

ahshahid commented on issue #6039: URL: https://github.com/apache/iceberg/issues/6039#issuecomment-1305877706 > Some update: For tpcds query with limited data and enabling stats at manifest level for non partition cols, still does not improve perf.. the cost of dpp query is pretty high, esp

[GitHub] [iceberg] rdblue commented on a diff in pull request #2276: Core: Add option to combine tasks by partition

rdblue commented on code in PR #2276: URL: https://github.com/apache/iceberg/pull/2276#discussion_r1015644370 ## core/src/main/java/org/apache/iceberg/util/TableScanUtil.java: ## @@ -71,6 +78,57 @@ public static CloseableIterable splitFiles( return CloseableIterable.combine

[GitHub] [iceberg] rdblue commented on a diff in pull request #4925: API: Add view interfaces

rdblue commented on code in PR #4925: URL: https://github.com/apache/iceberg/pull/4925#discussion_r1015647765 ## api/src/main/java/org/apache/iceberg/view/SQLViewRepresentation.java: ## @@ -0,0 +1,53 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or mo

[GitHub] [iceberg] rdblue commented on a diff in pull request #4925: API: Add view interfaces

rdblue commented on code in PR #4925: URL: https://github.com/apache/iceberg/pull/4925#discussion_r1015648978 ## api/src/main/java/org/apache/iceberg/view/SQLViewRepresentation.java: ## @@ -0,0 +1,53 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or mo

[GitHub] [iceberg] rdblue commented on a diff in pull request #4925: API: Add view interfaces

rdblue commented on code in PR #4925: URL: https://github.com/apache/iceberg/pull/4925#discussion_r1015649887 ## api/src/main/java/org/apache/iceberg/view/SQLViewRepresentation.java: ## @@ -0,0 +1,53 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or mo

[GitHub] [iceberg] dhruv-pratap commented on a diff in pull request #6069: Python: TableScan Plan files API implementation without residual evaluation

dhruv-pratap commented on code in PR #6069: URL: https://github.com/apache/iceberg/pull/6069#discussion_r1015651935 ## python/pyiceberg/table/scan.py: ## @@ -0,0 +1,103 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. S

[GitHub] [iceberg] rdblue commented on a diff in pull request #4925: API: Add view interfaces

rdblue commented on code in PR #4925: URL: https://github.com/apache/iceberg/pull/4925#discussion_r1015652744 ## api/src/main/java/org/apache/iceberg/view/ViewBuilder.java: ## @@ -0,0 +1,151 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contri

[GitHub] [iceberg] rdblue commented on a diff in pull request #4925: API: Add view interfaces

rdblue commented on code in PR #4925: URL: https://github.com/apache/iceberg/pull/4925#discussion_r1015653216 ## api/src/main/java/org/apache/iceberg/view/ViewBuilder.java: ## @@ -0,0 +1,151 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contri

[GitHub] [iceberg] nastra commented on pull request #6053: Build: Let revapi compare API compatibility against apache-iceberg-1.0.0

nastra commented on PR #6053: URL: https://github.com/apache/iceberg/pull/6053#issuecomment-1305895789 @ajantha-bhat this should work: ``` - org.apache.iceberg:iceberg-api:apache-iceberg-0.14.0: "0.14.0" + org.apache.iceberg:iceberg-api:1.0.0: "1.0.0" ``` -- This is an au

[GitHub] [iceberg] rdblue commented on a diff in pull request #4925: API: Add view interfaces

rdblue commented on code in PR #4925: URL: https://github.com/apache/iceberg/pull/4925#discussion_r1015658211 ## api/src/main/java/org/apache/iceberg/view/ViewVersion.java: ## @@ -0,0 +1,64 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contrib

[GitHub] [iceberg] rdblue commented on a diff in pull request #4925: API: Add view interfaces

rdblue commented on code in PR #4925: URL: https://github.com/apache/iceberg/pull/4925#discussion_r1015658509 ## api/src/main/java/org/apache/iceberg/view/ViewVersion.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contrib

[GitHub] [iceberg] rdblue commented on a diff in pull request #4925: API: Add view interfaces

rdblue commented on code in PR #4925: URL: https://github.com/apache/iceberg/pull/4925#discussion_r1015658926 ## api/src/main/java/org/apache/iceberg/view/ViewVersion.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contrib

[GitHub] [iceberg] rdblue commented on a diff in pull request #4925: API: Add view interfaces

rdblue commented on code in PR #4925: URL: https://github.com/apache/iceberg/pull/4925#discussion_r1015661549 ## api/src/main/java/org/apache/iceberg/view/SQLViewRepresentation.java: ## @@ -0,0 +1,53 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or mo

[GitHub] [iceberg] wmoustafa commented on a diff in pull request #4925: API: Add view interfaces

wmoustafa commented on code in PR #4925: URL: https://github.com/apache/iceberg/pull/4925#discussion_r1015680437 ## api/src/main/java/org/apache/iceberg/view/SQLViewRepresentation.java: ## @@ -0,0 +1,53 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or

[GitHub] [iceberg] wmoustafa commented on a diff in pull request #4925: API: Add view interfaces

wmoustafa commented on code in PR #4925: URL: https://github.com/apache/iceberg/pull/4925#discussion_r1015685367 ## api/src/main/java/org/apache/iceberg/view/SQLViewRepresentation.java: ## @@ -0,0 +1,53 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or

[GitHub] [iceberg] ddrinka commented on pull request #6131: Python: Add initial TableScan implementation

ddrinka commented on PR #6131: URL: https://github.com/apache/iceberg/pull/6131#issuecomment-1305936075 I'm just an outside observer here, but isn't there already a Python implementation that followed the Java API, but folks thought it would be good to do all this work to rewrite it to be m

[GitHub] [iceberg] dhruv-pratap commented on pull request #6069: Python: TableScan Plan files API implementation without residual evaluation

dhruv-pratap commented on PR #6069: URL: https://github.com/apache/iceberg/pull/6069#issuecomment-1305942424 In retrospect, I think this is becoming too large of a PR and would benefit from breaking down into smaller tasks. I'm going to go ahead and close this PR and if you guys are onboard

[GitHub] [iceberg] sunchao commented on a diff in pull request #2276: Core: Add option to combine tasks by partition

sunchao commented on code in PR #2276: URL: https://github.com/apache/iceberg/pull/2276#discussion_r1015704077 ## core/src/main/java/org/apache/iceberg/util/TableScanUtil.java: ## @@ -71,6 +78,57 @@ public static CloseableIterable splitFiles( return CloseableIterable.combin

[GitHub] [iceberg] sunchao commented on a diff in pull request #2276: Core: Add option to combine tasks by partition

sunchao commented on code in PR #2276: URL: https://github.com/apache/iceberg/pull/2276#discussion_r1015704882 ## core/src/main/java/org/apache/iceberg/util/TableScanUtil.java: ## @@ -71,6 +78,57 @@ public static CloseableIterable splitFiles( return CloseableIterable.combin

[GitHub] [iceberg] sunchao commented on a diff in pull request #2276: Core: Add option to combine tasks by partition

sunchao commented on code in PR #2276: URL: https://github.com/apache/iceberg/pull/2276#discussion_r1015705315 ## core/src/main/java/org/apache/iceberg/util/TableScanUtil.java: ## @@ -71,6 +78,57 @@ public static CloseableIterable splitFiles( return CloseableIterable.combin

[GitHub] [iceberg] jzhuge commented on a diff in pull request #4925: API: Add view interfaces

jzhuge commented on code in PR #4925: URL: https://github.com/apache/iceberg/pull/4925#discussion_r1015709176 ## api/src/main/java/org/apache/iceberg/view/SQLViewRepresentation.java: ## @@ -0,0 +1,53 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or mo

[GitHub] [iceberg] Fokko commented on a diff in pull request #6131: Python: Add initial TableScan implementation

Fokko commented on code in PR #6131: URL: https://github.com/apache/iceberg/pull/6131#discussion_r1015731779 ## python/pyiceberg/table/__init__.py: ## @@ -14,30 +14,43 @@ # KIND, either express or implied. See the License for the # specific language governing permissions and

[GitHub] [iceberg] Fokko commented on pull request #6131: Python: Add initial TableScan implementation

Fokko commented on PR #6131: URL: https://github.com/apache/iceberg/pull/6131#issuecomment-1305996028 I would also like: ```python scan = table.scan( row_filter=col("id") in [5, 6, 7], selected_fields=("id", "data"), snapshot_id=1234567890 ) ``` -- This is

[GitHub] [iceberg] Fokko opened a new pull request, #6140: Python: Fix Evaluator tests

Fokko opened a new pull request, #6140: URL: https://github.com/apache/iceberg/pull/6140 Instead of just supplying the unbound expression to the evaluator directly, we created a bogus one and replaced the bound expression with the one we wanted to test. But introduced a bug in the test beca

[GitHub] [iceberg] wmoustafa commented on a diff in pull request #4925: API: Add view interfaces

wmoustafa commented on code in PR #4925: URL: https://github.com/apache/iceberg/pull/4925#discussion_r1015762154 ## api/src/main/java/org/apache/iceberg/view/SQLViewRepresentation.java: ## @@ -0,0 +1,53 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or

[GitHub] [iceberg] ajantha-bhat commented on pull request #4826: Nessie: Use unique path for different table with same name

ajantha-bhat commented on PR #4826: URL: https://github.com/apache/iceberg/pull/4826#issuecomment-1306018512 @RussellSpitzer, @rdblue: Can we please merge this PR if it is ok? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [iceberg] rdblue commented on a diff in pull request #6140: Python: Fix Evaluator tests

rdblue commented on code in PR #6140: URL: https://github.com/apache/iceberg/pull/6140#discussion_r1015778820 ## python/tests/expressions/test_visitors.py: ## @@ -836,15 +843,23 @@ def _create_manifest_evaluator(bound_expr: BoundPredicate) -> _ManifestEvalVisit return eval

[GitHub] [iceberg] ajantha-bhat commented on pull request #6053: Build: Let revapi compare API compatibility against apache-iceberg-1.0.0

ajantha-bhat commented on PR #6053: URL: https://github.com/apache/iceberg/pull/6053#issuecomment-1306037207 > @ajantha-bhat this should work: - org.apache.iceberg:iceberg-api:apache-iceberg-0.14.0: "0.14.0" + org.apache.iceberg:iceberg-api:1.0.0: "1.0.0" a. Could you please ex

[GitHub] [iceberg] rdblue commented on a diff in pull request #6140: Python: Fix Evaluator tests

rdblue commented on code in PR #6140: URL: https://github.com/apache/iceberg/pull/6140#discussion_r1015778820 ## python/tests/expressions/test_visitors.py: ## @@ -836,15 +843,23 @@ def _create_manifest_evaluator(bound_expr: BoundPredicate) -> _ManifestEvalVisit return eval

[GitHub] [iceberg] jzhuge commented on a diff in pull request #4657: [WIP] API/Core: View support

jzhuge commented on code in PR #4657: URL: https://github.com/apache/iceberg/pull/4657#discussion_r1015780470 ## core/src/main/java/org/apache/iceberg/view/BaseViewDefinition.java: ## @@ -0,0 +1,197 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or mor

[GitHub] [iceberg] rdblue commented on a diff in pull request #6140: Python: Fix Evaluator tests

rdblue commented on code in PR #6140: URL: https://github.com/apache/iceberg/pull/6140#discussion_r1015781049 ## python/tests/expressions/test_visitors.py: ## @@ -853,19 +868,11 @@ def test_manifest_evaluator_less_than_no_overlap(): upper_bound=_to_byte_buffer(Strin

[GitHub] [iceberg] jzhuge commented on a diff in pull request #4925: API: Add view interfaces

jzhuge commented on code in PR #4925: URL: https://github.com/apache/iceberg/pull/4925#discussion_r1015780924 ## api/src/main/java/org/apache/iceberg/view/ViewBuilder.java: ## @@ -0,0 +1,151 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contri

[GitHub] [iceberg] jzhuge commented on a diff in pull request #4925: API: Add view interfaces

jzhuge commented on code in PR #4925: URL: https://github.com/apache/iceberg/pull/4925#discussion_r1015781565 ## api/src/main/java/org/apache/iceberg/view/ViewBuilder.java: ## @@ -0,0 +1,151 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contri

[GitHub] [iceberg] ajantha-bhat commented on pull request #6090: Core: Handle statistics file clean up from expireSnapshots

ajantha-bhat commented on PR #6090: URL: https://github.com/apache/iceberg/pull/6090#issuecomment-1306046322 > I think we should change the label of the snapshot-id entry in https://iceberg.apache.org/spec/#table-statistics (to level, not blob level) Sorry, I still didn't get how the

[GitHub] [iceberg] flyrain commented on pull request #6056: Parquet: Remove the row position since parquet row group has it natively

flyrain commented on PR #6056: URL: https://github.com/apache/iceberg/pull/6056#issuecomment-1306047299 > Do we trust this value from Parquet? The approach parquet used is similar to what @chenjunjiedada implemented in Iceberg repo. As long as it is reliable(no bug), I don't see a rea

[GitHub] [iceberg] Fokko opened a new pull request, #6141: Python: Make invalid Literal conversions explicit

Fokko opened a new pull request, #6141: URL: https://github.com/apache/iceberg/pull/6141 Currently, we silently turn Literals into None if we can't convert them, instead I prefer to raise an exception. This can cause silent bugs like: `EqualTo(Reference("id"), StringLiteral("123a"))` will t

[GitHub] [iceberg] Fokko commented on a diff in pull request #6140: Python: Fix Evaluator tests

Fokko commented on code in PR #6140: URL: https://github.com/apache/iceberg/pull/6140#discussion_r1015798033 ## python/tests/expressions/test_visitors.py: ## @@ -836,15 +843,23 @@ def _create_manifest_evaluator(bound_expr: BoundPredicate) -> _ManifestEvalVisit return evalu

[GitHub] [iceberg] Fokko commented on a diff in pull request #6140: Python: Fix Evaluator tests

Fokko commented on code in PR #6140: URL: https://github.com/apache/iceberg/pull/6140#discussion_r1015799107 ## python/tests/expressions/test_visitors.py: ## @@ -853,19 +868,11 @@ def test_manifest_evaluator_less_than_no_overlap(): upper_bound=_to_byte_buffer(String

[GitHub] [iceberg] aokolnychyi commented on pull request #2276: Core: Add option to combine tasks by partition

aokolnychyi commented on PR #2276: URL: https://github.com/apache/iceberg/pull/2276#issuecomment-1306079422 Let me take another look in a bit. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [iceberg] jzhuge commented on a diff in pull request #4925: API: Add view interfaces

jzhuge commented on code in PR #4925: URL: https://github.com/apache/iceberg/pull/4925#discussion_r1015834172 ## api/src/main/java/org/apache/iceberg/view/SQLViewRepresentation.java: ## @@ -0,0 +1,53 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or mo

[GitHub] [iceberg] rdblue commented on a diff in pull request #5432: AES GCM Stream Spec

rdblue commented on code in PR #5432: URL: https://github.com/apache/iceberg/pull/5432#discussion_r950908532 ## format/gcm-stream-spec.md: ## @@ -0,0 +1,87 @@ +--- +title: "AES GCM Stream Spec" +url: gcm-stream-spec +toc: true +disableSidebar: true +--- + + +# AES GCM Stream (AG

[GitHub] [iceberg] rdblue commented on a diff in pull request #5432: AES GCM Stream Spec

rdblue commented on code in PR #5432: URL: https://github.com/apache/iceberg/pull/5432#discussion_r1015856883 ## format/gcm-stream-spec.md: ## @@ -0,0 +1,87 @@ +--- +title: "AES GCM Stream Spec" +url: gcm-stream-spec +toc: true +disableSidebar: true +--- + + +# AES GCM Stream (A

[GitHub] [iceberg] rdblue commented on a diff in pull request #5432: AES GCM Stream Spec

rdblue commented on code in PR #5432: URL: https://github.com/apache/iceberg/pull/5432#discussion_r1015856676 ## format/gcm-stream-spec.md: ## @@ -0,0 +1,87 @@ +--- +title: "AES GCM Stream Spec" +url: gcm-stream-spec +toc: true +disableSidebar: true +--- + + +# AES GCM Stream (A

[GitHub] [iceberg] rdblue commented on a diff in pull request #5432: AES GCM Stream Spec

rdblue commented on code in PR #5432: URL: https://github.com/apache/iceberg/pull/5432#discussion_r1015857790 ## format/gcm-stream-spec.md: ## @@ -0,0 +1,87 @@ +--- +title: "AES GCM Stream Spec" +url: gcm-stream-spec +toc: true +disableSidebar: true +--- + + +# AES GCM Stream (A

[GitHub] [iceberg] rdblue commented on a diff in pull request #5432: AES GCM Stream Spec

rdblue commented on code in PR #5432: URL: https://github.com/apache/iceberg/pull/5432#discussion_r1015859348 ## format/gcm-stream-spec.md: ## @@ -0,0 +1,87 @@ +--- +title: "AES GCM Stream Spec" +url: gcm-stream-spec +toc: true +disableSidebar: true +--- + + +# AES GCM Stream (A

[GitHub] [iceberg] rdblue commented on a diff in pull request #4925: API: Add view interfaces

rdblue commented on code in PR #4925: URL: https://github.com/apache/iceberg/pull/4925#discussion_r1015864672 ## api/src/main/java/org/apache/iceberg/view/SQLViewRepresentation.java: ## @@ -0,0 +1,53 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or mo

[GitHub] [iceberg] rdblue commented on a diff in pull request #4925: API: Add view interfaces

rdblue commented on code in PR #4925: URL: https://github.com/apache/iceberg/pull/4925#discussion_r1015865455 ## api/src/main/java/org/apache/iceberg/view/ViewBuilder.java: ## @@ -0,0 +1,151 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contri

[GitHub] [iceberg] rdblue commented on a diff in pull request #6139: Python: Remove dataclasses

rdblue commented on code in PR #6139: URL: https://github.com/apache/iceberg/pull/6139#discussion_r1015867397 ## python/pyiceberg/expressions/__init__.py: ## @@ -32,6 +41,24 @@ B = TypeVar("B") +def _to_literal(lit: Optional[Union[T, Literal[T]]]) -> Optional[Literal[T]]: +

[GitHub] [iceberg] Fokko commented on pull request #6069: Python: TableScan Plan files API implementation without residual evaluation

Fokko commented on PR #6069: URL: https://github.com/apache/iceberg/pull/6069#issuecomment-1306148401 Hey @dhruv-pratap that makes a lot of sense. Maybe we should create issues on the list you mentioned above, to make sure that we're aligned on who's working on what. Smaller PRs make it muc

[GitHub] [iceberg] rdblue commented on a diff in pull request #6140: Python: Fix Evaluator tests

rdblue commented on code in PR #6140: URL: https://github.com/apache/iceberg/pull/6140#discussion_r1015875223 ## python/tests/expressions/test_visitors.py: ## @@ -853,19 +868,11 @@ def test_manifest_evaluator_less_than_no_overlap(): upper_bound=_to_byte_buffer(Strin

[GitHub] [iceberg] rdblue commented on a diff in pull request #6140: Python: Fix Evaluator tests

rdblue commented on code in PR #6140: URL: https://github.com/apache/iceberg/pull/6140#discussion_r1015876595 ## python/tests/expressions/test_visitors.py: ## @@ -874,36 +881,19 @@ def test_manifest_evaluator_less_than_overlap(): upper_bound=_to_byte_buffer(StringTy

[GitHub] [iceberg] rdblue commented on a diff in pull request #6140: Python: Fix Evaluator tests

rdblue commented on code in PR #6140: URL: https://github.com/apache/iceberg/pull/6140#discussion_r1015879735 ## python/tests/expressions/test_visitors.py: ## @@ -836,15 +843,23 @@ def _create_manifest_evaluator(bound_expr: BoundPredicate) -> _ManifestEvalVisit return eval

[GitHub] [iceberg] rdblue commented on a diff in pull request #6140: Python: Fix Evaluator tests

rdblue commented on code in PR #6140: URL: https://github.com/apache/iceberg/pull/6140#discussion_r1015880802 ## python/tests/expressions/test_visitors.py: ## @@ -853,19 +868,11 @@ def test_manifest_evaluator_less_than_no_overlap(): upper_bound=_to_byte_buffer(Strin

[GitHub] [iceberg] wmoustafa commented on a diff in pull request #6134: Spec: Add query-column-names to SQL view representation in view spec

wmoustafa commented on code in PR #6134: URL: https://github.com/apache/iceberg/pull/6134#discussion_r101588 ## format/view-spec.md: ## @@ -116,11 +116,19 @@ This type of representation stores the original view definition in SQL and its S | Optional | schema-id | ID of the

[GitHub] [iceberg] rdblue commented on a diff in pull request #6140: Python: Fix Evaluator tests

rdblue commented on code in PR #6140: URL: https://github.com/apache/iceberg/pull/6140#discussion_r1015882053 ## python/tests/expressions/test_visitors.py: ## @@ -874,36 +881,19 @@ def test_manifest_evaluator_less_than_overlap(): upper_bound=_to_byte_buffer(StringTy

[GitHub] [iceberg] wmoustafa commented on a diff in pull request #4925: API: Add view interfaces

wmoustafa commented on code in PR #4925: URL: https://github.com/apache/iceberg/pull/4925#discussion_r1015882452 ## api/src/main/java/org/apache/iceberg/view/SQLViewRepresentation.java: ## @@ -0,0 +1,53 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or

[GitHub] [iceberg] rdblue commented on a diff in pull request #6140: Python: Fix Evaluator tests

rdblue commented on code in PR #6140: URL: https://github.com/apache/iceberg/pull/6140#discussion_r1015882570 ## python/tests/expressions/test_visitors.py: ## @@ -912,19 +902,11 @@ def test_manifest_evaluator_less_than_no_match(): upper_bound=_to_byte_buffer(StringT

[GitHub] [iceberg] wmoustafa commented on a diff in pull request #4925: API: Add view interfaces

wmoustafa commented on code in PR #4925: URL: https://github.com/apache/iceberg/pull/4925#discussion_r1015888620 ## api/src/main/java/org/apache/iceberg/view/SQLViewRepresentation.java: ## @@ -0,0 +1,53 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or

[GitHub] [iceberg] Fokko commented on a diff in pull request #6140: Python: Fix Evaluator tests

Fokko commented on code in PR #6140: URL: https://github.com/apache/iceberg/pull/6140#discussion_r1015888905 ## python/tests/expressions/test_visitors.py: ## @@ -853,19 +868,11 @@ def test_manifest_evaluator_less_than_no_overlap(): upper_bound=_to_byte_buffer(String

[GitHub] [iceberg] Fokko commented on a diff in pull request #6140: Python: Fix Evaluator tests

Fokko commented on code in PR #6140: URL: https://github.com/apache/iceberg/pull/6140#discussion_r1015893351 ## python/tests/expressions/test_visitors.py: ## @@ -836,15 +843,23 @@ def _create_manifest_evaluator(bound_expr: BoundPredicate) -> _ManifestEvalVisit return evalu

[GitHub] [iceberg] Fokko commented on a diff in pull request #6140: Python: Fix Evaluator tests

Fokko commented on code in PR #6140: URL: https://github.com/apache/iceberg/pull/6140#discussion_r1015894495 ## python/tests/expressions/test_visitors.py: ## @@ -853,19 +868,11 @@ def test_manifest_evaluator_less_than_no_overlap(): upper_bound=_to_byte_buffer(String

[GitHub] [iceberg] Fokko commented on a diff in pull request #6140: Python: Fix Evaluator tests

Fokko commented on code in PR #6140: URL: https://github.com/apache/iceberg/pull/6140#discussion_r1015895913 ## python/tests/expressions/test_visitors.py: ## @@ -874,36 +881,19 @@ def test_manifest_evaluator_less_than_overlap(): upper_bound=_to_byte_buffer(StringTyp

[GitHub] [iceberg] Fokko commented on a diff in pull request #6139: Python: Remove dataclasses

Fokko commented on code in PR #6139: URL: https://github.com/apache/iceberg/pull/6139#discussion_r1015901564 ## python/pyiceberg/expressions/__init__.py: ## @@ -32,6 +41,24 @@ B = TypeVar("B") +def _to_literal(lit: Optional[Union[T, Literal[T]]]) -> Optional[Literal[T]]: +

[GitHub] [iceberg] Fokko commented on a diff in pull request #6139: Python: Remove dataclasses

Fokko commented on code in PR #6139: URL: https://github.com/apache/iceberg/pull/6139#discussion_r1015901564 ## python/pyiceberg/expressions/__init__.py: ## @@ -32,6 +41,24 @@ B = TypeVar("B") +def _to_literal(lit: Optional[Union[T, Literal[T]]]) -> Optional[Literal[T]]: +

[GitHub] [iceberg] haizhou-zhao commented on a diff in pull request #6045: [iceberg-hive-metastore] Support setting individual and group ownership for Namespace

haizhou-zhao commented on code in PR #6045: URL: https://github.com/apache/iceberg/pull/6045#discussion_r1013067243 ## hive-metastore/src/test/java/org/apache/iceberg/hive/TestHiveCatalog.java: ## @@ -448,6 +540,36 @@ public void testRemoveNamespaceProperties() throws TExceptio

[GitHub] [iceberg] github-actions[bot] closed issue #4633: Pyflink+Iceberg+Kinesis

github-actions[bot] closed issue #4633: Pyflink+Iceberg+Kinesis URL: https://github.com/apache/iceberg/issues/4633 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscrib

[GitHub] [iceberg] github-actions[bot] commented on issue #4633: Pyflink+Iceberg+Kinesis

github-actions[bot] commented on issue #4633: URL: https://github.com/apache/iceberg/issues/4633#issuecomment-1306403959 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

[GitHub] [iceberg] github-actions[bot] closed issue #3783: Cannot set TBLPROPERTIES when using HiveCatalog

github-actions[bot] closed issue #3783: Cannot set TBLPROPERTIES when using HiveCatalog URL: https://github.com/apache/iceberg/issues/3783 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

[GitHub] [iceberg] github-actions[bot] commented on issue #3783: Cannot set TBLPROPERTIES when using HiveCatalog

github-actions[bot] commented on issue #3783: URL: https://github.com/apache/iceberg/issues/3783#issuecomment-1306404062 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

[GitHub] [iceberg] rdblue commented on a diff in pull request #6140: Python: Fix Evaluator tests

rdblue commented on code in PR #6140: URL: https://github.com/apache/iceberg/pull/6140#discussion_r1016041196 ## python/tests/expressions/test_visitors.py: ## @@ -874,57 +872,32 @@ def test_manifest_evaluator_less_than_overlap(): upper_bound=_to_byte_buffer(StringTy

[GitHub] [iceberg] rdblue commented on a diff in pull request #6140: Python: Fix Evaluator tests

rdblue commented on code in PR #6140: URL: https://github.com/apache/iceberg/pull/6140#discussion_r1016041704 ## python/tests/expressions/test_visitors.py: ## @@ -954,36 +919,33 @@ def test_manifest_evaluator_less_than_or_equal_overlap(): upper_bound=_to_byte_buffer

[GitHub] [iceberg] rdblue commented on a diff in pull request #6140: Python: Fix Evaluator tests

rdblue commented on code in PR #6140: URL: https://github.com/apache/iceberg/pull/6140#discussion_r1016042277 ## python/tests/expressions/test_visitors.py: ## @@ -827,24 +834,23 @@ def _to_manifest_file(*partitions: PartitionFieldSummary) -> ManifestFile: ) -def _creat

[GitHub] [iceberg] stevenzwu commented on a diff in pull request #6111: Flink: Add 'cache.expiration-interval-ms' option to FlinkCatalog

stevenzwu commented on code in PR #6111: URL: https://github.com/apache/iceberg/pull/6111#discussion_r1016049656 ## flink/v1.14/flink/src/main/java/org/apache/iceberg/flink/FlinkCatalogFactory.java: ## @@ -145,8 +145,27 @@ protected Catalog createCatalog( baseNamespace =

<    9   10   11   12   13   14   15   16   17   18   >