[GitHub] [iceberg-docs] rdblue commented on pull request #174: Fix spark runtime jar link

rdblue commented on PR #174: URL: https://github.com/apache/iceberg-docs/pull/174#issuecomment-1319020172 Thanks, @vgupta3! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [iceberg] Fokko commented on a diff in pull request #6034: Python: GlueCatalog Full Implementation

Fokko commented on code in PR #6034: URL: https://github.com/apache/iceberg/pull/6034#discussion_r1025570314 ## python/pyiceberg/catalog/glue.py: ## @@ -0,0 +1,491 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See

[GitHub] [iceberg] Fokko commented on a diff in pull request #6034: Python: GlueCatalog Full Implementation

Fokko commented on code in PR #6034: URL: https://github.com/apache/iceberg/pull/6034#discussion_r1025571738 ## python/pyiceberg/catalog/__init__.py: ## @@ -70,9 +81,19 @@ def load_hive(name: str, conf: Properties) -> Catalog: raise NotInstalledError("Apache Hive suppor

[GitHub] [iceberg] Fokko commented on a diff in pull request #6034: Python: GlueCatalog Full Implementation

Fokko commented on code in PR #6034: URL: https://github.com/apache/iceberg/pull/6034#discussion_r1025572171 ## python/pyiceberg/catalog/__init__.py: ## @@ -134,6 +155,44 @@ def load_catalog(name: str, **properties: Optional[str]) -> Catalog: raise ValueError(f"Could not i

[GitHub] [iceberg] Fokko commented on a diff in pull request #6034: Python: GlueCatalog Full Implementation

Fokko commented on code in PR #6034: URL: https://github.com/apache/iceberg/pull/6034#discussion_r1025572564 ## python/pyiceberg/catalog/__init__.py: ## @@ -134,6 +155,44 @@ def load_catalog(name: str, **properties: Optional[str]) -> Catalog: raise ValueError(f"Could not i

[GitHub] [iceberg] Fokko commented on a diff in pull request #6034: Python: GlueCatalog Full Implementation

Fokko commented on code in PR #6034: URL: https://github.com/apache/iceberg/pull/6034#discussion_r1025575419 ## python/pyiceberg/catalog/glue.py: ## @@ -0,0 +1,491 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See

[GitHub] [iceberg] Fokko commented on pull request #6034: Python: GlueCatalog Full Implementation

Fokko commented on PR #6034: URL: https://github.com/apache/iceberg/pull/6034#issuecomment-1319061981 @ELHoussineT thanks for pinging me. I'll test the PR tomorrow against an actual glue instance, I think it is close! 👍🏻 -- This is an automated message from the Apache Git Service. To res

[GitHub] [iceberg] haizhou-zhao commented on a diff in pull request #6045: [iceberg-hive-metastore] Support setting individual and group ownership for Namespace

haizhou-zhao commented on code in PR #6045: URL: https://github.com/apache/iceberg/pull/6045#discussion_r1025582073 ## hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java: ## @@ -518,11 +541,26 @@ private Map convertToMetadata(Database database) { if (dat

[GitHub] [iceberg] alec-heif opened a new issue, #6210: pyiceberg BinaryDecoder does not correctly read int values

alec-heif opened a new issue, #6210: URL: https://github.com/apache/iceberg/issues/6210 ### Apache Iceberg version _No response_ ### Query engine _No response_ ### Please describe the bug 🐞 The logic in https://github.com/apache/iceberg/blob/master/python/p

[GitHub] [iceberg] islamismailov opened a new pull request, #6211: Allow dropping a column used by old SortOrders but not current SortOrder

islamismailov opened a new pull request, #6211: URL: https://github.com/apache/iceberg/pull/6211 Adding a unit test to make sure a column that USED to be a part of a sort order but NO LONGER is, is allowed to be dropped. -- This is an automated message from the Apache Git Service. To resp

[GitHub] [iceberg] alec-heif commented on issue #6210: pyiceberg BinaryDecoder does not correctly read int values

alec-heif commented on issue #6210: URL: https://github.com/apache/iceberg/issues/6210#issuecomment-1319098656 fyi @Fokko since this logic appears to have been introduced by https://github.com/apache/iceberg/pull/4920 -- This is an automated message from the Apache Git Service. To respond

[GitHub] [iceberg] krvikash opened a new pull request, #6212: Replace ImmutableMap.Builder.build() with buildOrThrow()

krvikash opened a new pull request, #6212: URL: https://github.com/apache/iceberg/pull/6212 Starting from guava 31.1, `ImmutableMap.Builder.build()` is discouraged to use and will soon be deprecated. Please see https://github.com/google/guava/commit/4bbe12c4e031b6c18074f933e444fa20a

[GitHub] [iceberg] islamismailov commented on issue #6204: Allow dropping a column used by old SortOrders

islamismailov commented on issue #6204: URL: https://github.com/apache/iceberg/issues/6204#issuecomment-1319124286 Added a PR https://github.com/apache/iceberg/pull/6211 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [iceberg] alec-heif commented on issue #6210: pyiceberg BinaryDecoder does not correctly read int values

alec-heif commented on issue #6210: URL: https://github.com/apache/iceberg/issues/6210#issuecomment-1319134076 somewhat disturbingly, the [unit test ](https://github.com/apache/iceberg/blob/master/python/tests/avro/test_decoder.py#L166)coverage for this function also verifies the (incorrect

[GitHub] [iceberg] Fokko commented on issue #6210: pyiceberg BinaryDecoder does not correctly read 4-byte little-endian int values

Fokko commented on issue #6210: URL: https://github.com/apache/iceberg/issues/6210#issuecomment-1319141897 Hey @alec-heif Thanks for opening this PR. I think we're mixing up different file types. The decoder that you pointed out in the example, is an Avro decoder that adheres to the [Avro

[GitHub] [iceberg] alec-heif commented on issue #6210: pyiceberg BinaryDecoder does not correctly read 4-byte little-endian int values

alec-heif commented on issue #6210: URL: https://github.com/apache/iceberg/issues/6210#issuecomment-1319146864 h wow ok that's embarrassing, thanks for correcting!! sorry for the false alarm here, i don't know how i missed this. -- This is an automated message from the Apache Git Serv

[GitHub] [iceberg] alec-heif closed issue #6210: pyiceberg BinaryDecoder does not correctly read 4-byte little-endian int values

alec-heif closed issue #6210: pyiceberg BinaryDecoder does not correctly read 4-byte little-endian int values URL: https://github.com/apache/iceberg/issues/6210 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

[GitHub] [iceberg] Fokko commented on issue #6210: pyiceberg BinaryDecoder does not correctly read 4-byte little-endian int values

Fokko commented on issue #6210: URL: https://github.com/apache/iceberg/issues/6210#issuecomment-1319152002 No problem at all, happy to help -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

[GitHub] [iceberg] Fokko commented on pull request #6206: Extract Flink package version programmatically for EnvironmentContext…

Fokko commented on PR #6206: URL: https://github.com/apache/iceberg/pull/6206#issuecomment-1319199002 I'm not super strong on it, but I would have added one, just to make sure that nothing changes on the Flink side. -- This is an automated message from the Apache Git Service. To respond t

[GitHub] [iceberg] haizhou-zhao commented on pull request #6045: [iceberg-hive-metastore] Support setting individual and group ownership for Namespace

haizhou-zhao commented on PR #6045: URL: https://github.com/apache/iceberg/pull/6045#issuecomment-1319223257 thanks @szehon-ho . All comments from the last round of review has been taken and committed. -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [iceberg] Fokko commented on a diff in pull request #6159: Python: Update mypy version

Fokko commented on code in PR #6159: URL: https://github.com/apache/iceberg/pull/6159#discussion_r1025777318 ## python/pyiceberg/utils/config.py: ## @@ -44,7 +44,7 @@ def merge_config(lhs: RecursiveDict, rhs: RecursiveDict) -> RecursiveDict: if isinstance(lhs_value

[GitHub] [iceberg] stevenzwu opened a new pull request, #6213: Flink: Add unit test for FlinkPackage util class

stevenzwu opened a new pull request, #6213: URL: https://github.com/apache/iceberg/pull/6213 follow-up to #6206 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscri

[GitHub] [iceberg] RussellSpitzer commented on a diff in pull request #5376: Core: Add readable metrics columns to files metadata tables

RussellSpitzer commented on code in PR #5376: URL: https://github.com/apache/iceberg/pull/5376#discussion_r1025803239 ## core/src/main/java/org/apache/iceberg/BaseFilesTable.java: ## @@ -140,42 +144,76 @@ protected CloseableIterable doPlanFiles() { } static class Manifes

[GitHub] [iceberg] RussellSpitzer commented on a diff in pull request #5376: Core: Add readable metrics columns to files metadata tables

RussellSpitzer commented on code in PR #5376: URL: https://github.com/apache/iceberg/pull/5376#discussion_r1025804938 ## core/src/main/java/org/apache/iceberg/BaseFilesTable.java: ## @@ -186,4 +224,43 @@ ManifestFile manifest() { return manifest; } } + + static c

[GitHub] [iceberg] RussellSpitzer commented on a diff in pull request #5376: Core: Add readable metrics columns to files metadata tables

RussellSpitzer commented on code in PR #5376: URL: https://github.com/apache/iceberg/pull/5376#discussion_r1025805720 ## core/src/main/java/org/apache/iceberg/BaseFilesTable.java: ## @@ -186,4 +224,43 @@ ManifestFile manifest() { return manifest; } } + + static c

[GitHub] [iceberg] RussellSpitzer commented on a diff in pull request #5376: Core: Add readable metrics columns to files metadata tables

RussellSpitzer commented on code in PR #5376: URL: https://github.com/apache/iceberg/pull/5376#discussion_r1025808348 ## core/src/main/java/org/apache/iceberg/MetricsUtil.java: ## @@ -56,4 +72,270 @@ public static MetricsModes.MetricsMode metricsMode( String columnName = in

[GitHub] [iceberg] RussellSpitzer commented on a diff in pull request #5376: Core: Add readable metrics columns to files metadata tables

RussellSpitzer commented on code in PR #5376: URL: https://github.com/apache/iceberg/pull/5376#discussion_r1025811376 ## core/src/main/java/org/apache/iceberg/MetricsUtil.java: ## @@ -56,4 +72,270 @@ public static MetricsModes.MetricsMode metricsMode( String columnName = in

[GitHub] [iceberg] RussellSpitzer commented on a diff in pull request #5376: Core: Add readable metrics columns to files metadata tables

RussellSpitzer commented on code in PR #5376: URL: https://github.com/apache/iceberg/pull/5376#discussion_r1025812897 ## core/src/main/java/org/apache/iceberg/MetricsUtil.java: ## @@ -56,4 +72,270 @@ public static MetricsModes.MetricsMode metricsMode( String columnName = in

[GitHub] [iceberg] RussellSpitzer commented on a diff in pull request #5376: Core: Add readable metrics columns to files metadata tables

RussellSpitzer commented on code in PR #5376: URL: https://github.com/apache/iceberg/pull/5376#discussion_r1025816613 ## spark/v3.2/spark/src/test/java/org/apache/iceberg/spark/data/TestHelpers.java: ## @@ -817,4 +824,93 @@ public static Set reachableManifestPaths(Table table)

[GitHub] [iceberg] fortarch commented on issue #5993: Quick start docker-compose demo doesn't work

fortarch commented on issue #5993: URL: https://github.com/apache/iceberg/issues/5993#issuecomment-1319451204 > version: "3" > > services: > spark-iceberg: > image: tabulario/spark-iceberg > container_name: spark-iceberg > build: spark/ > depends_on:

[GitHub] [iceberg-docs] ajantha-bhat commented on pull request #175: Docs: Update spark-3.0 removal

ajantha-bhat commented on PR #175: URL: https://github.com/apache/iceberg-docs/pull/175#issuecomment-1319504689 Rebased now. Thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

[GitHub] [iceberg] eemario commented on issue #6205: Using iceberg-parquet and iceberg-spark-runtime throws ClassCastException for MessageType

eemario commented on issue #6205: URL: https://github.com/apache/iceberg/issues/6205#issuecomment-1319537484 I think these two dependencies triggered the exception when I call `spark.read().format("iceberg").load(...)` : org.apache.iceberg iceberg-parque

[GitHub] [iceberg] eemario closed issue #6205: Using iceberg-parquet and iceberg-spark-runtime throws ClassCastException for MessageType

eemario closed issue #6205: Using iceberg-parquet and iceberg-spark-runtime throws ClassCastException for MessageType URL: https://github.com/apache/iceberg/issues/6205 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [iceberg] ajantha-bhat commented on issue #6205: Using iceberg-parquet and iceberg-spark-runtime throws ClassCastException for MessageType

ajantha-bhat commented on issue #6205: URL: https://github.com/apache/iceberg/issues/6205#issuecomment-1319538691 `iceberg-spark-runtime-3.2_2.12` dependency itself is enough. No need to depend on `iceberg-parquet` explicitly I think. -- This is an automated message from the Apache Git S

[GitHub] [iceberg] eemario commented on issue #6205: Using iceberg-parquet and iceberg-spark-runtime throws ClassCastException for MessageType

eemario commented on issue #6205: URL: https://github.com/apache/iceberg/issues/6205#issuecomment-1319546566 Yes, I have tried it too and that is another way to get around this. But we only use iceberg-spark-runtime to test. It is a bit too big to be included as a whole and so we choose

[GitHub] [iceberg] aokolnychyi commented on a diff in pull request #6207: Spark 3.3: Add years transform function

aokolnychyi commented on code in PR #6207: URL: https://github.com/apache/iceberg/pull/6207#discussion_r1026001944 ## spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/functions/YearsFunction.java: ## @@ -0,0 +1,129 @@ +/* + * Licensed to the Apache Software Foundation (AS

[GitHub] [iceberg] aokolnychyi commented on a diff in pull request #6207: Spark 3.3: Add years transform function

aokolnychyi commented on code in PR #6207: URL: https://github.com/apache/iceberg/pull/6207#discussion_r1026002989 ## spark/v3.3/spark/src/test/java/org/apache/iceberg/spark/sql/TestSparkYearsFunction.java: ## @@ -0,0 +1,116 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [iceberg] aokolnychyi commented on a diff in pull request #6207: Spark 3.3: Add years transform function

aokolnychyi commented on code in PR #6207: URL: https://github.com/apache/iceberg/pull/6207#discussion_r1026003401 ## spark/v3.3/spark/src/test/java/org/apache/iceberg/spark/sql/TestSparkYearsFunction.java: ## @@ -0,0 +1,116 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [iceberg] aokolnychyi commented on a diff in pull request #6207: Spark 3.3: Add years transform function

aokolnychyi commented on code in PR #6207: URL: https://github.com/apache/iceberg/pull/6207#discussion_r1026004169 ## spark/v3.3/spark/src/test/java/org/apache/iceberg/spark/sql/TestSparkYearsFunction.java: ## @@ -0,0 +1,116 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [iceberg] ConeyLiu commented on a diff in pull request #4627: Parquet: Fixes get null values for the nested field partition column

ConeyLiu commented on code in PR #4627: URL: https://github.com/apache/iceberg/pull/4627#discussion_r1026057680 ## parquet/src/main/java/org/apache/iceberg/data/parquet/BaseParquetReaders.java: ## @@ -149,11 +153,14 @@ public ParquetValueReader struct(Types.StructType expected,

[GitHub] [iceberg] ConeyLiu commented on a diff in pull request #4627: Parquet: Fixes get null values for the nested field partition column

ConeyLiu commented on code in PR #4627: URL: https://github.com/apache/iceberg/pull/4627#discussion_r1026059432 ## spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/data/SparkParquetReaders.java: ## @@ -148,6 +149,9 @@ public ParquetValueReader struct( int id =

[GitHub] [iceberg] pvary commented on pull request #6206: Extract Flink package version programmatically for EnvironmentContext…

pvary commented on PR #6206: URL: https://github.com/apache/iceberg/pull/6206#issuecomment-1319616607 @Fokko: Sorry, I have missed your comment about the unit tests. Based on the approvals just merged the change -- This is an automated message from the Apache Git Service. To respond to th

[GitHub] [iceberg] renshangtao commented on pull request #5026: Reduce the number of equity-deletes using bloom filter

renshangtao commented on PR #5026: URL: https://github.com/apache/iceberg/pull/5026#issuecomment-1319619526 @puneetzaroo here is the issue https://github.com/apache/iceberg/issues/5025 -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [iceberg] Fokko commented on pull request #6206: Extract Flink package version programmatically for EnvironmentContext…

Fokko commented on PR #6206: URL: https://github.com/apache/iceberg/pull/6206#issuecomment-1319624537 @pvary no problem, it was more of a nit, and therefore I already approved 👍🏻 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [iceberg] Fokko merged pull request #6213: Flink: Add unit test for FlinkPackage util class

Fokko merged PR #6213: URL: https://github.com/apache/iceberg/pull/6213 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apach

[GitHub] [iceberg] Fokko commented on pull request #6213: Flink: Add unit test for FlinkPackage util class

Fokko commented on PR #6213: URL: https://github.com/apache/iceberg/pull/6213#issuecomment-1319624890 Thanks @stevenzwu 👍🏻 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [iceberg] LuigiCerone commented on a diff in pull request #6159: Python: Update mypy version

LuigiCerone commented on code in PR #6159: URL: https://github.com/apache/iceberg/pull/6159#discussion_r1026091130 ## python/pyiceberg/utils/config.py: ## @@ -44,7 +44,7 @@ def merge_config(lhs: RecursiveDict, rhs: RecursiveDict) -> RecursiveDict: if isinstance(lhs

[GitHub] [iceberg] nastra commented on issue #5993: Quick start docker-compose demo doesn't work

nastra commented on issue #5993: URL: https://github.com/apache/iceberg/issues/5993#issuecomment-1319648430 Closing this as it's fixed by https://github.com/apache/iceberg-docs/pull/178 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [iceberg] nastra closed issue #5993: Quick start docker-compose demo doesn't work

nastra closed issue #5993: Quick start docker-compose demo doesn't work URL: https://github.com/apache/iceberg/issues/5993 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

[GitHub] [iceberg] gaborkaszab commented on pull request #6175: Hive: Add UGI to the key in CachedClientPool

gaborkaszab commented on PR #6175: URL: https://github.com/apache/iceberg/pull/6175#issuecomment-131917 > @lirui-apache: I am still a bit concerned that in Hive or Impala this would cause issues by not reusing the client pools. @gaborkaszab: Could you please check from Hive/Impala side?

[GitHub] [iceberg] gaborkaszab commented on pull request #5837: API,Core: Introduce metrics for data files by file format

gaborkaszab commented on PR #5837: URL: https://github.com/apache/iceberg/pull/5837#issuecomment-1319670877 > the code changes themselves LGTM but I'm still not sure that this is how we'd want to represent **dimensions** in metrics as this doesn't really scale to add a new metric field for

[GitHub] [iceberg] ajantha-bhat opened a new pull request, #6214: Docs: Update the view spec

ajantha-bhat opened a new pull request, #6214: URL: https://github.com/apache/iceberg/pull/6214 some follow-up from https://github.com/apache/iceberg/pull/3188 Other than some comments add detailed examples. I have handled all. Examples can be added in the follow-up PR. -- This is

[GitHub] [iceberg] ajantha-bhat commented on pull request #6214: Docs: Update the view spec

ajantha-bhat commented on PR #6214: URL: https://github.com/apache/iceberg/pull/6214#issuecomment-1319687679 cc: @rdblue , @jzhuge -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

[GitHub] [iceberg] ajantha-bhat commented on a diff in pull request #6214: Docs: Update the view spec

ajantha-bhat commented on code in PR #6214: URL: https://github.com/apache/iceberg/pull/6214#discussion_r1026144988 ## format/view-spec.md: ## @@ -58,16 +55,16 @@ Writers create view metadata files optimistically, assuming that the current met The view version metadata file

[GitHub] [iceberg] szehon-ho commented on a diff in pull request #4627: Parquet: Fixes get null values for the nested field partition column

szehon-ho commented on code in PR #4627: URL: https://github.com/apache/iceberg/pull/4627#discussion_r1026164504 ## spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/data/SparkParquetReaders.java: ## @@ -148,6 +149,9 @@ public ParquetValueReader struct( int id =

[GitHub] [iceberg] nastra commented on a diff in pull request #6207: Spark 3.3: Add years transform function

nastra commented on code in PR #6207: URL: https://github.com/apache/iceberg/pull/6207#discussion_r1026166812 ## spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/functions/YearsFunction.java: ## @@ -0,0 +1,129 @@ +/* + * Licensed to the Apache Software Foundation (ASF) un

[GitHub] [iceberg] grbinho opened a new issue, #6215: Accessing AWS Glue in different AWS Account

grbinho opened a new issue, #6215: URL: https://github.com/apache/iceberg/issues/6215 ### Apache Iceberg version 0.14.0 ### Query engine Other ### Please describe the bug 🐞 Hi We are using AWS Glue Jobs 3.0 (Spark 3.1) with Iceberg 14.0 (through Glue

[GitHub] [iceberg] chenwyi2 opened a new issue, #6216: write.metadata.metrics.default how to works?

chenwyi2 opened a new issue, #6216: URL: https://github.com/apache/iceberg/issues/6216 ### Query engine spark 3.xx ### Question i set write.metadata.metrics.default =full to an iceberg table, how can i see if it works? i log the upper and lower message, the object messag

[GitHub] [iceberg] nastra commented on issue #6216: write.metadata.metrics.default how to works?

nastra commented on issue #6216: URL: https://github.com/apache/iceberg/issues/6216#issuecomment-1319800904 You can look at the manifest file and it should contains metrics for `lower_bounds` / `upper_bounds`/ and a few others. Or is your question more specific to using `full` metrics mode?

[GitHub] [iceberg] AlvaroNavarroMora commented on issue #6105: Add tblproperties using alter table with spark

AlvaroNavarroMora commented on issue #6105: URL: https://github.com/apache/iceberg/issues/6105#issuecomment-1319803494 Hi, after doing some testing we have observed that the properties are being applied to the tables, however it seems to not be working properly. We have tables with those pr

[GitHub] [iceberg] nastra commented on issue #6105: Add tblproperties using alter table with spark

nastra commented on issue #6105: URL: https://github.com/apache/iceberg/issues/6105#issuecomment-1319810330 The same issue came up in a [Slack discussion](https://apache-iceberg.slack.com/archives/C025PH0G1D4/p1668714478198079?thread_ts=1668645354.671479&cid=C025PH0G1D4) and I've opened htt

[GitHub] [iceberg] singhpk234 commented on issue #6215: Accessing AWS Glue in different AWS Account

singhpk234 commented on issue #6215: URL: https://github.com/apache/iceberg/issues/6215#issuecomment-1319817987 > Can this also be added to the defaultWarehouseLocation method? I'm happy to open a PR, but wanted to check first if there is a specific reason this setting is omitted. I

[GitHub] [iceberg] ELHoussineT commented on issue #6105: Add tblproperties using alter table with spark

ELHoussineT commented on issue #6105: URL: https://github.com/apache/iceberg/issues/6105#issuecomment-1319834985 @AlvaroNavarroMora Interesting, we did as the example you shared in the main thread and we saw that the metadata were deleted. For verification of the deletion of the metadata, w

[GitHub] [iceberg] chenwyi2 commented on issue #6216: write.metadata.metrics.default how to works?

chenwyi2 commented on issue #6216: URL: https://github.com/apache/iceberg/issues/6216#issuecomment-1319835217 manifest file can see deleteFile.lowerBounds() ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

[GitHub] [iceberg] chenwyi2 commented on issue #6216: write.metadata.metrics.default how to works?

chenwyi2 commented on issue #6216: URL: https://github.com/apache/iceberg/issues/6216#issuecomment-1319836132  here, LOG.info("lower {}, fromByteBuffer{}",lower,Conversions.

[GitHub] [iceberg] lirui-apache commented on pull request #6175: Hive: Add UGI to the key in CachedClientPool

lirui-apache commented on PR #6175: URL: https://github.com/apache/iceberg/pull/6175#issuecomment-1319845848 The pool is still reused as long as the same UGI instance is in use. But I agree it might be desirable to reuse the pool across queries or sessions for the same user. Since it

[GitHub] [iceberg] szehon-ho commented on pull request #6175: Hive: Add UGI to the key in CachedClientPool

szehon-ho commented on PR #6175: URL: https://github.com/apache/iceberg/pull/6175#issuecomment-1319877477 I like that idea, to have a pluggable cache. As I mentioned before with things like https://github.com/apache/iceberg/pull/5378 , we have users wanting to get different HMS cli

[GitHub] [iceberg] szehon-ho commented on a diff in pull request #6045: [iceberg-hive-metastore] Support setting individual and group ownership for Namespace

szehon-ho commented on code in PR #6045: URL: https://github.com/apache/iceberg/pull/6045#discussion_r1026348753 ## hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java: ## @@ -356,6 +365,11 @@ public boolean dropNamespace(Namespace namespace) { @Override

[GitHub] [iceberg] szehon-ho commented on a diff in pull request #6045: [iceberg-hive-metastore] Support setting individual and group ownership for Namespace

szehon-ho commented on code in PR #6045: URL: https://github.com/apache/iceberg/pull/6045#discussion_r1026350489 ## hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java: ## @@ -518,11 +541,26 @@ private Map convertToMetadata(Database database) { if (databa

[GitHub] [iceberg] szehon-ho commented on a diff in pull request #6045: [iceberg-hive-metastore] Support setting individual and group ownership for Namespace

szehon-ho commented on code in PR #6045: URL: https://github.com/apache/iceberg/pull/6045#discussion_r1026348753 ## hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java: ## @@ -356,6 +365,11 @@ public boolean dropNamespace(Namespace namespace) { @Override

[GitHub] [iceberg] ConeyLiu commented on a diff in pull request #4627: Parquet: Fixes get null values for the nested field partition column

ConeyLiu commented on code in PR #4627: URL: https://github.com/apache/iceberg/pull/4627#discussion_r1026394236 ## spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/data/SparkParquetReaders.java: ## @@ -148,6 +149,9 @@ public ParquetValueReader struct( int id =

[GitHub] [iceberg] rbalamohan opened a new issue, #6218: Set COMMIT_MIN_RETRY_WAIT_MS_DEFAULT to 1000 ms instead of 100 ms to avoid too frequent commit exceptions

rbalamohan opened a new issue, #6218: URL: https://github.com/apache/iceberg/issues/6218 ### Apache Iceberg version _No response_ ### Query engine _No response_ ### Please describe the bug 🐞 COMMIT_MIN_RETRY_WAIT_MS_DEFAULT has been set to 100ms. Though it h

[GitHub] [iceberg-docs] hililiwei opened a new pull request, #179: Docs: Update Flink 1.13 removal and 1.16 addition.

hililiwei opened a new pull request, #179: URL: https://github.com/apache/iceberg-docs/pull/179 Update Flink 1.13 removal and 1.16 addition. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

[GitHub] [iceberg] Fokko closed pull request #6219: API: Fix breaking change in Partition Spec transform

Fokko closed pull request #6219: API: Fix breaking change in Partition Spec transform URL: https://github.com/apache/iceberg/pull/6219 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

[GitHub] [iceberg] szehon-ho commented on a diff in pull request #4627: Parquet: Fixes get null values for the nested field partition column

szehon-ho commented on code in PR #4627: URL: https://github.com/apache/iceberg/pull/4627#discussion_r1026542566 ## spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/data/SparkParquetReaders.java: ## @@ -148,6 +149,9 @@ public ParquetValueReader struct( int id =

[GitHub] [iceberg] szehon-ho closed issue #4626: Get null values for for the nested field partition column

szehon-ho closed issue #4626: Get null values for for the nested field partition column URL: https://github.com/apache/iceberg/issues/4626 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

[GitHub] [iceberg] szehon-ho merged pull request #4627: Parquet: Fixes get null values for the nested field partition column

szehon-ho merged PR #4627: URL: https://github.com/apache/iceberg/pull/4627 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.a

[GitHub] [iceberg] szehon-ho commented on pull request #4627: Parquet: Fixes get null values for the nested field partition column

szehon-ho commented on PR #4627: URL: https://github.com/apache/iceberg/pull/4627#issuecomment-1320148292 Thanks @ConeyLiu for fix, and @pvary and @kbendick for additional review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [iceberg] Fokko opened a new pull request, #6220: API: Pass in the types

Fokko opened a new pull request, #6220: URL: https://github.com/apache/iceberg/pull/6220 Debugging some regression that we have at Trino, this PR seems to fix a part of it. Looks like there is another codepath that also creates PartitionSpecs outside of the builder. Looking into that now

[GitHub] [iceberg] szehon-ho commented on a diff in pull request #5376: Core: Add readable metrics columns to files metadata tables

szehon-ho commented on code in PR #5376: URL: https://github.com/apache/iceberg/pull/5376#discussion_r1026570061 ## core/src/main/java/org/apache/iceberg/MetricsUtil.java: ## @@ -56,4 +72,270 @@ public static MetricsModes.MetricsMode metricsMode( String columnName = inputSc

[GitHub] [iceberg] szehon-ho commented on a diff in pull request #5376: Core: Add readable metrics columns to files metadata tables

szehon-ho commented on code in PR #5376: URL: https://github.com/apache/iceberg/pull/5376#discussion_r1026571367 ## core/src/main/java/org/apache/iceberg/MetricsUtil.java: ## @@ -56,4 +72,270 @@ public static MetricsModes.MetricsMode metricsMode( String columnName = inputSc

[GitHub] [iceberg] szehon-ho commented on a diff in pull request #5376: Core: Add readable metrics columns to files metadata tables

szehon-ho commented on code in PR #5376: URL: https://github.com/apache/iceberg/pull/5376#discussion_r1026573190 ## core/src/main/java/org/apache/iceberg/BaseFilesTable.java: ## @@ -186,4 +224,43 @@ ManifestFile manifest() { return manifest; } } + + static class

[GitHub] [iceberg] szehon-ho commented on a diff in pull request #5376: Core: Add readable metrics columns to files metadata tables

szehon-ho commented on code in PR #5376: URL: https://github.com/apache/iceberg/pull/5376#discussion_r1026574052 ## core/src/main/java/org/apache/iceberg/BaseFilesTable.java: ## @@ -186,4 +224,43 @@ ManifestFile manifest() { return manifest; } } + + static class

[GitHub] [iceberg] szehon-ho commented on a diff in pull request #5376: Core: Add readable metrics columns to files metadata tables

szehon-ho commented on code in PR #5376: URL: https://github.com/apache/iceberg/pull/5376#discussion_r1026578079 ## spark/v3.2/spark/src/test/java/org/apache/iceberg/spark/data/TestHelpers.java: ## @@ -817,4 +824,93 @@ public static Set reachableManifestPaths(Table table) {

[GitHub] [iceberg] szehon-ho commented on a diff in pull request #5376: Core: Add readable metrics columns to files metadata tables

szehon-ho commented on code in PR #5376: URL: https://github.com/apache/iceberg/pull/5376#discussion_r1026574052 ## core/src/main/java/org/apache/iceberg/BaseFilesTable.java: ## @@ -186,4 +224,43 @@ ManifestFile manifest() { return manifest; } } + + static class

[GitHub] [iceberg] szehon-ho commented on a diff in pull request #5376: Core: Add readable metrics columns to files metadata tables

szehon-ho commented on code in PR #5376: URL: https://github.com/apache/iceberg/pull/5376#discussion_r1026578079 ## spark/v3.2/spark/src/test/java/org/apache/iceberg/spark/data/TestHelpers.java: ## @@ -817,4 +824,93 @@ public static Set reachableManifestPaths(Table table) {

[GitHub] [iceberg] szehon-ho commented on a diff in pull request #5376: Core: Add readable metrics columns to files metadata tables

szehon-ho commented on code in PR #5376: URL: https://github.com/apache/iceberg/pull/5376#discussion_r1026578079 ## spark/v3.2/spark/src/test/java/org/apache/iceberg/spark/data/TestHelpers.java: ## @@ -817,4 +824,93 @@ public static Set reachableManifestPaths(Table table) {

[GitHub] [iceberg] szehon-ho commented on a diff in pull request #5376: Core: Add readable metrics columns to files metadata tables

szehon-ho commented on code in PR #5376: URL: https://github.com/apache/iceberg/pull/5376#discussion_r1026595133 ## spark/v3.2/spark/src/test/java/org/apache/iceberg/spark/data/TestHelpers.java: ## @@ -817,4 +824,93 @@ public static Set reachableManifestPaths(Table table) {

[GitHub] [iceberg] szehon-ho commented on a diff in pull request #5376: Core: Add readable metrics columns to files metadata tables

szehon-ho commented on code in PR #5376: URL: https://github.com/apache/iceberg/pull/5376#discussion_r1026573190 ## core/src/main/java/org/apache/iceberg/BaseFilesTable.java: ## @@ -186,4 +224,43 @@ ManifestFile manifest() { return manifest; } } + + static class

[GitHub] [iceberg] Fokko merged pull request #6159: Python: Update mypy version

Fokko merged PR #6159: URL: https://github.com/apache/iceberg/pull/6159 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apach

[GitHub] [iceberg] Fokko closed issue #6148: PyIceberg: Bump mypy to v0.990

Fokko closed issue #6148: PyIceberg: Bump mypy to v0.990 URL: https://github.com/apache/iceberg/issues/6148 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-ma

[GitHub] [iceberg-docs] stevenzwu merged pull request #179: Docs: Update Flink 1.13 removal and 1.16 addition.

stevenzwu merged PR #179: URL: https://github.com/apache/iceberg-docs/pull/179 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

[GitHub] [iceberg-docs] stevenzwu commented on pull request #179: Docs: Update Flink 1.13 removal and 1.16 addition.

stevenzwu commented on PR #179: URL: https://github.com/apache/iceberg-docs/pull/179#issuecomment-1320231209 thx @hililiwei -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [iceberg] Fokko commented on pull request #6220: API: Make the PartitionSpec less lazy

Fokko commented on PR #6220: URL: https://github.com/apache/iceberg/pull/6220#issuecomment-1320264231 Trino is now passing without changes on their side:  -- This is an a

[GitHub] [iceberg] JonasJ-ap commented on pull request #6034: Python: GlueCatalog Full Implementation

JonasJ-ap commented on PR #6034: URL: https://github.com/apache/iceberg/pull/6034#issuecomment-1320324309 > @ELHoussineT thanks for pinging me. I'll test the PR tomorrow against an actual glue instance, I think it is close! 👍🏻 @Fokko Thank you very much for your review

[GitHub] [iceberg] islamismailov commented on pull request #6211: Allow dropping a column used by old SortOrders but not current SortOrder

islamismailov commented on PR #6211: URL: https://github.com/apache/iceberg/pull/6211#issuecomment-1320325932 @Fokko can you have a look? I am just adding a unit test here since it seems like this issue does not affect latest Iceberg. -- This is an automated message from the Apache Git Se

[GitHub] [iceberg] flyrain commented on pull request #6175: Hive: Add UGI to the key in CachedClientPool

flyrain commented on PR #6175: URL: https://github.com/apache/iceberg/pull/6175#issuecomment-1320364877 #5378 is a similar solution. One of reasons we didn't continue is that we also think about removing the `CachedClientPool`, which might be a better solution. The `CachedClientPool` wa

[GitHub] [iceberg] flyrain commented on a diff in pull request #6012: Spark 3.3: Add a procedure to generate table changes

flyrain commented on code in PR #6012: URL: https://github.com/apache/iceberg/pull/6012#discussion_r1026810947 ## spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/procedures/GenerateChangesProcedure.java: ## @@ -0,0 +1,210 @@ +/* + * Licensed to the Apache Software Founda

[GitHub] [iceberg] flyrain commented on a diff in pull request #6012: Spark 3.3: Add a procedure to generate table changes

flyrain commented on code in PR #6012: URL: https://github.com/apache/iceberg/pull/6012#discussion_r1026812311 ## spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/procedures/GenerateChangesProcedure.java: ## @@ -0,0 +1,336 @@ +/* + * Licensed to the Apache Software Founda

[GitHub] [iceberg] haizhou-zhao commented on a diff in pull request #6045: [iceberg-hive-metastore] Support setting individual and group ownership for Namespace

haizhou-zhao commented on code in PR #6045: URL: https://github.com/apache/iceberg/pull/6045#discussion_r1026820406 ## hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java: ## @@ -356,6 +365,11 @@ public boolean dropNamespace(Namespace namespace) { @Overrid

<    18   19   20   21   22   23   24   25   26   27   >