Re: [I] Flink: Make Hadoop an optional dependency [iceberg]

github-actions[bot] commented on issue #7332: URL: https://github.com/apache/iceberg/issues/7332#issuecomment-2594191701 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] Avro referenced in metadata json file is missing [iceberg]

github-actions[bot] commented on issue #7137: URL: https://github.com/apache/iceberg/issues/7137#issuecomment-2594191683 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [PR] Core: Make metrics reporter serializable (alternative impl) [iceberg]

github-actions[bot] commented on PR #8032: URL: https://github.com/apache/iceberg/pull/8032#issuecomment-2594191816 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull

Re: [I] The snapshots_id is not found in the table.snapshots [iceberg]

github-actions[bot] closed issue #9140: The snapshots_id is not found in the table.snapshots URL: https://github.com/apache/iceberg/issues/9140 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

Re: [I] With spark and hive thrift server, inserts to iceberg table in one connection are not seen in another [iceberg]

github-actions[bot] commented on issue #9135: URL: https://github.com/apache/iceberg/issues/9135#issuecomment-2594192105 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Cannot delete files cleanly with CatalogUtil::dropTableData [iceberg]

github-actions[bot] commented on issue #9164: URL: https://github.com/apache/iceberg/issues/9164#issuecomment-2594192211 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Concerns with `String.toLowerCase()` in default Locale [iceberg]

github-actions[bot] commented on issue #9276: URL: https://github.com/apache/iceberg/issues/9276#issuecomment-2594192413 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] flink programs sometimes fail to write to icebergTable. The.avro file in metadata cannot be found [iceberg]

github-actions[bot] commented on issue #9168: URL: https://github.com/apache/iceberg/issues/9168#issuecomment-2594192255 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Querying metadata tables for a branch or tag [iceberg]

github-actions[bot] closed issue #9279: Querying metadata tables for a branch or tag URL: https://github.com/apache/iceberg/issues/9279 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [I] Create Branches / TAGS between 2 snapshots [iceberg]

github-actions[bot] commented on issue #9281: URL: https://github.com/apache/iceberg/issues/9281#issuecomment-2594192471 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Allow for - in Glue Catalog DB/Table names [iceberg]

github-actions[bot] closed issue #9169: Allow for - in Glue Catalog DB/Table names URL: https://github.com/apache/iceberg/issues/9169 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

Re: [I] metadata json conflict when streaming [iceberg]

github-actions[bot] commented on issue #9171: URL: https://github.com/apache/iceberg/issues/9171#issuecomment-2594192304 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] An exception occurred while writing iceberg data through Spark: org. apache. iceberg. exceptions. CommitFailedException: metadata location has changed [iceberg]

github-actions[bot] commented on issue #9178: URL: https://github.com/apache/iceberg/issues/9178#issuecomment-2594192321 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] Generate iceberg metadata file based on _spark_metadata [iceberg]

github-actions[bot] commented on issue #9270: URL: https://github.com/apache/iceberg/issues/9270#issuecomment-2594192360 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] TBLPROPERTIES('history.expire.max-snapshot-age-ms') doesn't work [iceberg]

github-actions[bot] commented on issue #9123: URL: https://github.com/apache/iceberg/issues/9123#issuecomment-2594192037 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] `GlueTableOperations` retries on Access Denied exceptions from S3, and does not support configuration of exception retry logic [iceberg]

github-actions[bot] commented on issue #9124: URL: https://github.com/apache/iceberg/issues/9124#issuecomment-2594192062 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] `GlueTableOperations` retries on Access Denied exceptions from S3, and does not support configuration of exception retry logic [iceberg]

github-actions[bot] closed issue #9124: `GlueTableOperations` retries on Access Denied exceptions from S3, and does not support configuration of exception retry logic URL: https://github.com/apache/iceberg/issues/9124 -- This is an automated message from the Apache Git Service. To respond to

Re: [I] With spark and hive thrift server, inserts to iceberg table in one connection are not seen in another [iceberg]

github-actions[bot] closed issue #9135: With spark and hive thrift server, inserts to iceberg table in one connection are not seen in another URL: https://github.com/apache/iceberg/issues/9135 -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [I] The query result of `col > x` may be incorrect when there are NaN values in the column `col` [iceberg]

github-actions[bot] commented on issue #9130: URL: https://github.com/apache/iceberg/issues/9130#issuecomment-2594192084 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] encountered a problem while making multiple bucket maybe a bug [iceberg]

github-actions[bot] closed issue #9167: encountered a problem while making multiple bucket maybe a bug URL: https://github.com/apache/iceberg/issues/9167 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [I] Unclosed input streams when writing with high throughput [iceberg]

github-actions[bot] closed issue #9148: Unclosed input streams when writing with high throughput URL: https://github.com/apache/iceberg/issues/9148 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

Re: [I] java.lang.ClassNotFoundException: Failed to find data source: iceberg. Issue when we are using Java Custom Catalog [iceberg]

github-actions[bot] closed issue #9275: java.lang.ClassNotFoundException: Failed to find data source: iceberg. Issue when we are using Java Custom Catalog URL: https://github.com/apache/iceberg/issues/9275 -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [I] Considering adjust the default row-group size of Parquet position delete file [iceberg]

github-actions[bot] closed issue #9149: Considering adjust the default row-group size of Parquet position delete file URL: https://github.com/apache/iceberg/issues/9149 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] Core: Parsing and Writing Tests for V3 Metadata [iceberg]

rdblue commented on code in PR #11947: URL: https://github.com/apache/iceberg/pull/11947#discussion_r1917404736 ## core/src/test/java/org/apache/iceberg/MetadataTestUtils.java: ## @@ -0,0 +1,428 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more co

Re: [PR] Core: Parsing and Writing Tests for V3 Metadata [iceberg]

rdblue commented on code in PR #11947: URL: https://github.com/apache/iceberg/pull/11947#discussion_r1917409546 ## core/src/test/java/org/apache/iceberg/MetadataTestUtils.java: ## @@ -0,0 +1,428 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more co

Re: [PR] Core: Parsing and Writing Tests for V3 Metadata [iceberg]

rdblue commented on code in PR #11947: URL: https://github.com/apache/iceberg/pull/11947#discussion_r1917406983 ## core/src/test/java/org/apache/iceberg/MetadataTestUtils.java: ## @@ -0,0 +1,428 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more co

Re: [PR] Core: Parsing and Writing Tests for V3 Metadata [iceberg]

rdblue commented on code in PR #11947: URL: https://github.com/apache/iceberg/pull/11947#discussion_r1917421391 ## core/src/test/java/org/apache/iceberg/MetadataTestUtils.java: ## @@ -0,0 +1,336 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more co

Re: [PR] ADLS: Support Vended Credentials [iceberg-python]

corleyma commented on PR #1520: URL: https://github.com/apache/iceberg-python/pull/1520#issuecomment-2594171646 I believe support for SAS token was added to the Arrow azure filesystem, but I don't think it has been released yet: https://github.com/apache/arrow/pull/45021 looks like a

Re: [I] position delete in BaseEqualityDeltaWriter write function will lead to unstable result when equalityFieldColumns is not null and upsert is false [iceberg]

github-actions[bot] commented on issue #9299: URL: https://github.com/apache/iceberg/issues/9299#issuecomment-2594192548 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [PR] Build: Add plugin to generate license and notice files [iceberg]

bryanck commented on PR #11977: URL: https://github.com/apache/iceberg/pull/11977#issuecomment-2594205352 > I liked the idea that Ryan suggested this morning, where our plugin would always generate and check the generated files against a set of committed existing files and report an error i

Re: [PR] Spec: Add added-rows field to Snapshot [iceberg]

dramaticlly commented on code in PR #11976: URL: https://github.com/apache/iceberg/pull/11976#discussion_r1917506904 ## format/spec.md: ## @@ -654,6 +656,7 @@ A snapshot consists of the following fields: | _optional_ | _required_ | _required_ | **`summary`**| A

Re: [PR] Spark 3.5: Fix flaky tests `withSnapshotIsolation` [iceberg]

manuzhang commented on PR #11974: URL: https://github.com/apache/iceberg/pull/11974#issuecomment-2594264595 @RussellSpitzer The maximum wait time is 1000 milliseconds, so the total wait time could be around 2000 milliseconds, or 2 seconds. -- This is an automated message from the Apache G

Re: [PR] WIP: Add headers for type/field/schema [iceberg-cpp]

lidavidm commented on PR #31: URL: https://github.com/apache/iceberg-cpp/pull/31#issuecomment-2594267414 Note: we could simplify the data type definition greatly if we adopt something like Arrow Java/[cuDF](https://docs.rapids.ai/api/libcudf/stable/classcudf_1_1data__type). Type would be a

Re: [PR] feat: add file_io and local fs impl [iceberg-cpp]

zhjwpku commented on PR #30: URL: https://github.com/apache/iceberg-cpp/pull/30#issuecomment-2594324922 > Has iceberg-cpp decided on an IO strategy already? > > It might be more productive to start writing the IO-less components, such as parsing the various metadata files, etc.

Re: [PR] Core: Parsing and Writing Tests for V3 Metadata [iceberg]

rdblue commented on code in PR #11947: URL: https://github.com/apache/iceberg/pull/11947#discussion_r1917413799 ## core/src/test/java/org/apache/iceberg/TestSnapshotJson.java: ## @@ -56,13 +56,16 @@ public void testJsonConversion() throws IOException { @Test public void

Re: [PR] Core: Parsing and Writing Tests for V3 Metadata [iceberg]

rdblue commented on code in PR #11947: URL: https://github.com/apache/iceberg/pull/11947#discussion_r1917414530 ## core/src/test/java/org/apache/iceberg/TestSnapshotJson.java: ## @@ -56,13 +56,16 @@ public void testJsonConversion() throws IOException { @Test public void

Re: [PR] Spec: Document Snapshot Summary Optional Fields for Standardization [iceberg]

HonahX commented on code in PR #11660: URL: https://github.com/apache/iceberg/pull/11660#discussion_r1917478351 ## format/spec.md: ## @@ -1633,3 +1633,50 @@ might indicate different snapshot IDs for a specific timestamp. The discrepancie When processing point in time queries

Re: [PR] Core, API, Spec: Metadata Row Lineage [iceberg]

stevenzwu commented on code in PR #11948: URL: https://github.com/apache/iceberg/pull/11948#discussion_r1917465682 ## core/src/main/java/org/apache/iceberg/TableMetadata.java: ## @@ -1468,6 +1515,45 @@ public Builder setPreviousFileLocation(String previousFileLocation) {

Re: [PR] Spec: add variant type [iceberg]

aihuaxu commented on code in PR #10831: URL: https://github.com/apache/iceberg/pull/10831#discussion_r1917457914 ## format/spec.md: ## @@ -182,6 +182,21 @@ A **`list`** is a collection of values with some element type. The element field A **`map`** is a collection of key-val

Re: [PR] feat: add file_io and local fs impl [iceberg-cpp]

lidavidm commented on PR #30: URL: https://github.com/apache/iceberg-cpp/pull/30#issuecomment-2594348439 For instance, async vs sync: https://github.com/apache/iceberg-cpp/issues/2#issuecomment-2522607394 Or, whether the core library should do any IO at all: https://github.com/apache

Re: [PR] FIX: retry REST catalog on 401 UnauthorizedError with refresh token [iceberg-python]

sungwy commented on PR #1517: URL: https://github.com/apache/iceberg-python/pull/1517#issuecomment-2594353847 I agree with @kevinjqliu here. I've created this issue on polaris github to fix the root cause on the catalog instead: https://github.com/apache/polaris/issues/791 -- This is an

Re: [PR] Spark 3.5: Display write metrics on SQL UI [iceberg]

manuzhang commented on PR #11340: URL: https://github.com/apache/iceberg/pull/11340#issuecomment-2594354446 @wypoon I plan to add metrics for all write operations, but I'd like to get the interfaces right at first. I'm not sure whether this is the best way to propagate a `metricsReporter`.

Re: [PR] feat: add file_io and local fs impl [iceberg-cpp]

MisterRaindrop commented on PR #30: URL: https://github.com/apache/iceberg-cpp/pull/30#issuecomment-2594372750 Is my understanding correct that this FileIo pertains to locations that are local, on HDFS, or on S3? -- This is an automated message from the Apache Git Service. To respond to t

Re: [PR] just a simple test [iceberg]

rodmeneses closed pull request #11978: just a simple test URL: https://github.com/apache/iceberg/pull/11978 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-ma

Re: [PR] WIP: Add headers for type/field/schema [iceberg-cpp]

lidavidm commented on PR #31: URL: https://github.com/apache/iceberg-cpp/pull/31#issuecomment-2594268386 For here I've chosen something somewhat resembling Arrow C++. However I think Schema and Field (unlike Arrow) should not always be wrapped in smart pointers. Type is still in a smart poi

Re: [PR] WIP: Add headers for type/field/schema [iceberg-cpp]

lidavidm commented on code in PR #31: URL: https://github.com/apache/iceberg-cpp/pull/31#discussion_r1917537751 ## src/iceberg/type_fwd.h: ## @@ -0,0 +1,55 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the

Re: [PR] WIP: Add headers for type/field/schema [iceberg-cpp]

lidavidm commented on code in PR #31: URL: https://github.com/apache/iceberg-cpp/pull/31#discussion_r1917538165 ## src/iceberg/schema.h: ## @@ -0,0 +1,60 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NO

Re: [I] [DISCUSSION] Project Goal [iceberg-cpp]

MisterRaindrop commented on issue #2: URL: https://github.com/apache/iceberg-cpp/issues/2#issuecomment-2594306540 Actually parquet as a storage format, doesn’t change frequently. Therefore I split parquet from arrow. To make it cleaner. -- This is an automated message from the Apache Git

Re: [PR] feat: add file_io and local fs impl [iceberg-cpp]

zhjwpku commented on PR #30: URL: https://github.com/apache/iceberg-cpp/pull/30#issuecomment-2594455927 > Is my understanding correct that this FileIo pertains to locations that are local, on HDFS, or on S3? Yeah, I hope this FileIO to be extended to other storages. -- This is an a

Re: [PR] feat: Support Bucket and Truncate transforms on write [iceberg-python]

kevinjqliu commented on code in PR #1345: URL: https://github.com/apache/iceberg-python/pull/1345#discussion_r1917748644 ## tests/integration/test_writes/test_partitioned_writes.py: ## @@ -760,50 +760,104 @@ def test_invalid_arguments(spark: SparkSession, session_catalog: Catal

Re: [PR] Bump up spark to 3.5.4 [iceberg-python]

ndrluis closed pull request #1521: Bump up spark to 3.5.4 URL: https://github.com/apache/iceberg-python/pull/1521 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe

[I] Validation Error in ConfigResponse Model When connecting Nessie with PyIceberg using RestCatalog [iceberg-python]

heman026 opened a new issue, #1524: URL: https://github.com/apache/iceberg-python/issues/1524 ### Question I tried connecting to Nessie using load_catalog and RestCatalog() from pyiceberg, but I am getting the below error in Config Response Model, > pydantic_core._pydantic_core

Re: [PR] Build: Bump mypy-boto3-glue from 1.35.93 to 1.36.0 [iceberg-python]

Fokko merged PR #1522: URL: https://github.com/apache/iceberg-python/pull/1522 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [I] Validation Error in ConfigResponse Model When connecting Nessie with PyIceberg using RestCatalog [iceberg-python]

Fokko commented on issue #1524: URL: https://github.com/apache/iceberg-python/issues/1524#issuecomment-2594666872 Thanks @heman026 for the quick reply. It looks like fields are missing from the `config` response. Could you share the JSON response? This can be done by adding a `print` state

Re: [PR] Core, API, Spec: Metadata Row Lineage [iceberg]

nastra commented on code in PR #11948: URL: https://github.com/apache/iceberg/pull/11948#discussion_r1917881876 ## core/src/main/java/org/apache/iceberg/BaseSnapshot.java: ## @@ -61,7 +63,9 @@ class BaseSnapshot implements Snapshot { String operation, Map summary,

Re: [PR] Add Doxygen for generating API documentation [iceberg-cpp]

Fokko commented on code in PR #27: URL: https://github.com/apache/iceberg-cpp/pull/27#discussion_r1917816727 ## docs/README.md: ## @@ -0,0 +1,27 @@ + + +# Documentation + +To build the documentation: + +#. Install [Doxygen][doxygen]. Review Comment: nit: I prefer inline link

Re: [I] Validation Error in ConfigResponse Model When connecting Nessie with PyIceberg using RestCatalog [iceberg-python]

heman026 commented on issue #1524: URL: https://github.com/apache/iceberg-python/issues/1524#issuecomment-2594674319 DEBUG:urllib3.connectionpool:http://10.xx.xx.xx:19120 "GET /api/v1/config?warehouse=s3a%3A%2F%2Ficeberg-datalake HTTP/11" 200 84 `{'defaultBranch': 'main', 'maxSupport

Re: [PR] feat: Support Bucket and Truncate transforms on write [iceberg-python]

Fokko commented on code in PR #1345: URL: https://github.com/apache/iceberg-python/pull/1345#discussion_r1917829595 ## pyiceberg/transforms.py: ## @@ -193,6 +195,24 @@ def supports_pyarrow_transform(self) -> bool: @abstractmethod def pyarrow_transform(self, source: Ice

Re: [PR] Core, Rest: Enable useSystemProperties on RESTClient [iceberg]

munendrasn commented on PR #11548: URL: https://github.com/apache/iceberg/pull/11548#issuecomment-2594635312 @nastra Created another PR with possible alternative solution https://github.com/apache/iceberg/pull/11979 -- This is an automated message from the Apache Git Service. To respo

Re: [PR] Add Doxygen for generating API documentation [iceberg-cpp]

Fokko commented on code in PR #27: URL: https://github.com/apache/iceberg-cpp/pull/27#discussion_r1917817748 ## docs/README.md: ## @@ -0,0 +1,29 @@ + + +# Documentation + +To build the documentation: + +#. Install [Doxygen][doxygen]. +#. From this directory, run `doxygen`. Revi

Re: [PR] [infra] download Spark from `archive.apache.org` [iceberg-python]

Fokko commented on PR #1523: URL: https://github.com/apache/iceberg-python/pull/1523#issuecomment-2594643191 Hopefully it is more stable now, thanks for fixing this @kevinjqliu -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [I] Validation Error in ConfigResponse Model When connecting Nessie with PyIceberg using RestCatalog [iceberg-python]

Fokko commented on issue #1524: URL: https://github.com/apache/iceberg-python/issues/1524#issuecomment-2594642043 @heman026 Thanks for raising this, and happy to help. Do you have a full stack-trace? -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] [infra] download Spark from `archive.apache.org` [iceberg-python]

Fokko merged PR #1523: URL: https://github.com/apache/iceberg-python/pull/1523 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [PR] Build: Bump mypy-boto3-glue from 1.35.93 to 1.36.0 [iceberg-python]

Fokko commented on PR #1522: URL: https://github.com/apache/iceberg-python/pull/1522#issuecomment-2594643477 @dependabot rebase -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

Re: [PR] Spark: Fix empty scan issue when start timestamp retrieves root snapshot and end timestamp is missing [iceberg]

lliangyu-lin commented on code in PR #11967: URL: https://github.com/apache/iceberg/pull/11967#discussion_r1917821160 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/SparkScanBuilder.java: ## @@ -561,14 +561,11 @@ public Scan buildChangelogScan() { boole

Re: [I] Validation Error in ConfigResponse Model When connecting Nessie with PyIceberg using RestCatalog [iceberg-python]

Fokko commented on issue #1524: URL: https://github.com/apache/iceberg-python/issues/1524#issuecomment-2594702838 That doesn't look like the Iceberg REST protocol at all. I'm not an expert on Nessie, but maybe we can debug it together. What endpoint did you configure in PyIceberg? -- Th

Re: [PR] Core: Parsing and Writing Tests for V3 Metadata [iceberg]

HonahX commented on code in PR #11947: URL: https://github.com/apache/iceberg/pull/11947#discussion_r1917865349 ## core/src/test/java/org/apache/iceberg/MetadataTestUtils.java: ## @@ -0,0 +1,336 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more co

Re: [PR] Core: Parsing and Writing Tests for V3 Metadata [iceberg]

HonahX commented on code in PR #11947: URL: https://github.com/apache/iceberg/pull/11947#discussion_r1917865349 ## core/src/test/java/org/apache/iceberg/MetadataTestUtils.java: ## @@ -0,0 +1,336 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more co

Re: [PR] Core: Parsing and Writing Tests for V3 Metadata [iceberg]

HonahX commented on code in PR #11947: URL: https://github.com/apache/iceberg/pull/11947#discussion_r1917873174 ## core/src/main/java/org/apache/iceberg/TableMetadataParser.java: ## @@ -372,6 +373,7 @@ public static TableMetadata fromJson(String metadataLocation, JsonNode node)

Re: [I] cannot load table thru glue catalog [iceberg-python]

xpj01 commented on issue #1501: URL: https://github.com/apache/iceberg-python/issues/1501#issuecomment-2594573841 I've found the root cause that the VPC was setup the s3 endpoint policy and put few buckets in it. I change the policy to include the bucket I used. The issue got resolved.

Re: [I] Validation Error in ConfigResponse Model When connecting Nessie with PyIceberg using RestCatalog [iceberg-python]

heman026 commented on issue #1524: URL: https://github.com/apache/iceberg-python/issues/1524#issuecomment-2594650407 > [@heman026](https://github.com/heman026) Thanks for raising this, and happy to help. Do you have a full stack-trace? Traceback (most recent call last): File "C:

Re: [I] Iceberg functions doesnt work [iceberg]

nastra commented on issue #11951: URL: https://github.com/apache/iceberg/issues/11951#issuecomment-2594758950 What error are you getting when you e.g. call the `expire_snapshots` procedure or when you create a tag? -- This is an automated message from the Apache Git Service. To respond to

Re: [PR] Spark 3.5: Display write metrics on SQL UI [iceberg]

wypoon commented on PR #11340: URL: https://github.com/apache/iceberg/pull/11340#issuecomment-2593778165 @manuzhang I am happy to see that someone is working on adding write-side Iceberg metrics to the Spark SQL UI! I realize that this is still in a draft state, but I have some questions

Re: [PR] Spark 3.5: Display write metrics on SQL UI [iceberg]

wypoon commented on code in PR #11340: URL: https://github.com/apache/iceberg/pull/11340#discussion_r1917245679 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/metrics/TotalDataFiles.java: ## @@ -0,0 +1,36 @@ +/* + * Licensed to the Apache Software Foundation (

Re: [I] DeleteOrphanFilesSparkAction.listDirRecursively - No FileSystem for scheme "s3" [iceberg]

sherman commented on issue #10539: URL: https://github.com/apache/iceberg/issues/10539#issuecomment-2593822642 BTW, you might try this (use S3AFileSystem for **s3** prefix): ``` .set("spark.hadoop.fs.s3.impl", "org.apache.hadoop.fs.s3a.S3AFileSystem") ``` -- This is an aut

Re: [PR] Parquet: Add readers and writers for the internal object model [iceberg]

rdblue commented on code in PR #11904: URL: https://github.com/apache/iceberg/pull/11904#discussion_r1917266691 ## data/src/test/java/org/apache/iceberg/data/parquet/TestInternalData.java: ## @@ -0,0 +1,94 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + *

Re: [PR] Parquet: Add readers and writers for the internal object model [iceberg]

rdblue commented on code in PR #11904: URL: https://github.com/apache/iceberg/pull/11904#discussion_r1917278339 ## data/src/test/java/org/apache/iceberg/data/parquet/TestInternalData.java: ## @@ -0,0 +1,94 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + *

Re: [PR] Parquet: Add readers and writers for the internal object model [iceberg]

rdblue commented on code in PR #11904: URL: https://github.com/apache/iceberg/pull/11904#discussion_r1917264392 ## api/src/test/java/org/apache/iceberg/util/RandomUtil.java: ## @@ -246,15 +246,15 @@ public static List generateList( if (list.isElementOptional() && random.n

[PR] Revert "Add support for lowercase `FileFormat`(#1362)" [iceberg-python]

Fokko opened a new pull request, #1518: URL: https://github.com/apache/iceberg-python/pull/1518 This reverts commit 4e755996c11e1768a63d3f3f663bfa77994648b7 which causes a sad CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

Re: [PR] Parquet: Add readers and writers for the internal object model [iceberg]

rdblue commented on code in PR #11904: URL: https://github.com/apache/iceberg/pull/11904#discussion_r1917277358 ## data/src/test/java/org/apache/iceberg/data/parquet/TestInternalData.java: ## @@ -0,0 +1,94 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + *

Re: [PR] Build: Add plugin to generate license and notice files [iceberg]

bryanck commented on code in PR #11977: URL: https://github.com/apache/iceberg/pull/11977#discussion_r1917278321 ## buildSrc/build.gradle: ## @@ -0,0 +1,26 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the

Re: [PR] Parquet: Add readers and writers for the internal object model [iceberg]

rdblue commented on code in PR #11904: URL: https://github.com/apache/iceberg/pull/11904#discussion_r1917279635 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/data/SparkParquetReaders.java: ## @@ -373,41 +371,6 @@ public Decimal read(Decimal ignored) { } }

Re: [PR] Parquet: Add readers and writers for the internal object model [iceberg]

rdblue commented on code in PR #11904: URL: https://github.com/apache/iceberg/pull/11904#discussion_r1917278880 ## data/src/test/java/org/apache/iceberg/data/parquet/TestInternalData.java: ## @@ -0,0 +1,94 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + *

Re: [PR] Revert "Add support for lowercase `FileFormat`(#1362)" [iceberg-python]

Fokko merged PR #1518: URL: https://github.com/apache/iceberg-python/pull/1518 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [PR] Revert "Add support for lowercase `FileFormat`(#1362)" [iceberg-python]

Fokko commented on PR #1518: URL: https://github.com/apache/iceberg-python/pull/1518#issuecomment-2593853426 Thanks @kevinjqliu for the quick turnaround 👍 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[PR] IO: Remove deprecations [iceberg-python]

Fokko opened a new pull request, #1519: URL: https://github.com/apache/iceberg-python/pull/1519 Less is more! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe

Re: [PR] Parquet: Add readers and writers for the internal object model [iceberg]

rdblue commented on code in PR #11904: URL: https://github.com/apache/iceberg/pull/11904#discussion_r1917285311 ## parquet/src/main/java/org/apache/iceberg/data/parquet/BaseParquetReaders.java: ## @@ -397,7 +404,7 @@ public ParquetValueReader primitive( case INT96:

Re: [PR] Parquet: Add readers and writers for the internal object model [iceberg]

rdblue commented on code in PR #11904: URL: https://github.com/apache/iceberg/pull/11904#discussion_r1917263968 ## api/src/test/java/org/apache/iceberg/util/RandomUtil.java: ## @@ -237,7 +237,7 @@ private static BigInteger randomUnscaled(int precision, Random random) { }

Re: [PR] Auth Manager API part 3: OAuth2 Manager [iceberg]

adutra commented on code in PR #11844: URL: https://github.com/apache/iceberg/pull/11844#discussion_r1916787886 ## core/src/main/java/org/apache/iceberg/rest/auth/AuthSessionCache.java: ## @@ -0,0 +1,130 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * o

Re: [PR] Core, API, Spec: Metadata Row Lineage [iceberg]

nastra commented on code in PR #11948: URL: https://github.com/apache/iceberg/pull/11948#discussion_r1916821207 ## core/src/main/java/org/apache/iceberg/TableMetadata.java: ## @@ -563,6 +578,14 @@ public TableMetadata withUUID() { return new Builder(this).assignUUID().build

Re: [PR] Added support for lowercase FileFormat for Issue #1340 [iceberg-python]

kevinjqliu commented on PR #1362: URL: https://github.com/apache/iceberg-python/pull/1362#issuecomment-2593417593 @Fokko this PR is failing CI https://github.com/apache/iceberg-python/actions/runs/12792587045/job/35663397601 I also tested that it fails locally. We might need to revert thi

Re: [PR] Core, API, Spec: Metadata Row Lineage [iceberg]

RussellSpitzer commented on code in PR #11948: URL: https://github.com/apache/iceberg/pull/11948#discussion_r1917006502 ## format/spec.md: ## @@ -654,17 +656,18 @@ The `first_row_id` is only inherited for added data files. The inherited value m A snapshot consists of the fol

Re: [PR] WIP: Deletion vectors [iceberg-python]

kevinjqliu commented on code in PR #1516: URL: https://github.com/apache/iceberg-python/pull/1516#discussion_r1917014423 ## dev/provision.py: ## @@ -401,3 +401,43 @@ ) spark.sql(f"ALTER TABLE {catalog_name}.default.test_empty_scan_ordered_str WRITE ORDERED BY id")

Re: [PR] Auth Manager API part 3: OAuth2 Manager [iceberg]

danielcweeks commented on code in PR #11844: URL: https://github.com/apache/iceberg/pull/11844#discussion_r1917015969 ## core/src/main/java/org/apache/iceberg/rest/auth/AuthSessionCache.java: ## @@ -0,0 +1,130 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

Re: [PR] IO: Remove deprecations [iceberg-python]

Fokko commented on PR #1519: URL: https://github.com/apache/iceberg-python/pull/1519#issuecomment-2593877228 Thanks again @kevinjqliu 🙌 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

Re: [PR] IO: Remove deprecations [iceberg-python]

Fokko merged PR #1519: URL: https://github.com/apache/iceberg-python/pull/1519 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [PR] Parquet: Add readers and writers for the internal object model [iceberg]

rdblue commented on code in PR #11904: URL: https://github.com/apache/iceberg/pull/11904#discussion_r1917299961 ## parquet/src/main/java/org/apache/iceberg/data/parquet/BaseParquetReaders.java: ## @@ -76,6 +67,31 @@ protected ParquetValueReader createReader( protected abstrac

Re: [PR] Parquet: Add readers and writers for the internal object model [iceberg]

rdblue commented on code in PR #11904: URL: https://github.com/apache/iceberg/pull/11904#discussion_r1917303932 ## parquet/src/main/java/org/apache/iceberg/data/parquet/BaseParquetReaders.java: ## @@ -76,6 +67,31 @@ protected ParquetValueReader createReader( protected abstrac

Re: [PR] Parquet: Add readers and writers for the internal object model [iceberg]

rdblue commented on code in PR #11904: URL: https://github.com/apache/iceberg/pull/11904#discussion_r1917305124 ## parquet/src/main/java/org/apache/iceberg/data/parquet/BaseParquetReaders.java: ## @@ -76,6 +67,31 @@ protected ParquetValueReader createReader( protected abstrac

Re: [PR] Parquet: Add readers and writers for the internal object model [iceberg]

rdblue commented on code in PR #11904: URL: https://github.com/apache/iceberg/pull/11904#discussion_r1917308787 ## parquet/src/main/java/org/apache/iceberg/data/parquet/GenericParquetReaders.java: ## @@ -92,4 +151,124 @@ protected void set(Record struct, int pos, Object value) {

<    1   2   3   >