Re: [I] Support Nessie catalog [iceberg-python]

cee-shubham commented on issue #19: URL: https://github.com/apache/iceberg-python/issues/19#issuecomment-2379058126 my coder is: from pyiceberg.catalog import load_catalog from pyiceberg.exceptions import NamespaceAlreadyExistsError # Define the catalog configuration catalog_t

Re: [PR] Arrow: add support for null vectors [iceberg]

nastra commented on code in PR #10953: URL: https://github.com/apache/iceberg/pull/10953#discussion_r1778476498 ## arrow/src/main/java/org/apache/iceberg/arrow/vectorized/VectorHolder.java: ## @@ -128,7 +129,7 @@ public static VectorHolder dummyHolder(int numRows) { } pu

Re: [I] Delete Files in Table Scans [iceberg-rust]

sdd commented on issue #630: URL: https://github.com/apache/iceberg-rust/issues/630#issuecomment-2378632325 Thanks for taking a look at the above, @liurenjie1024. I've just submitted a draft PR which outlines the second part of the approach - how we extend the filtering in the arrow reader

Re: [PR] Core: Add DataFileSet / DeleteFileSet [iceberg]

nastra commented on code in PR #11195: URL: https://github.com/apache/iceberg/pull/11195#discussion_r1778263784 ## api/src/main/java/org/apache/iceberg/util/WrapperSet.java: ## @@ -0,0 +1,185 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contr

Re: [PR] Core: Add DataFileSet / DeleteFileSet [iceberg]

nastra commented on code in PR #11195: URL: https://github.com/apache/iceberg/pull/11195#discussion_r1778103857 ## api/src/main/java/org/apache/iceberg/util/WrapperSet.java: ## @@ -0,0 +1,185 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contr

Re: [PR] Core: Add DataFileSet / DeleteFileSet [iceberg]

nastra commented on code in PR #11195: URL: https://github.com/apache/iceberg/pull/11195#discussion_r1778454236 ## api/src/main/java/org/apache/iceberg/util/DataFileSet.java: ## @@ -0,0 +1,113 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more cont

Re: [PR] API, Core: Add scan planning api request and response models [iceberg]

rahil-c commented on code in PR #11180: URL: https://github.com/apache/iceberg/pull/11180#discussion_r1779211148 ## core/src/main/java/org/apache/iceberg/rest/responses/FetchPlanningResultResponse.java: ## @@ -0,0 +1,51 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] API, Core: Add scan planning api request and response models [iceberg]

rahil-c commented on code in PR #11180: URL: https://github.com/apache/iceberg/pull/11180#discussion_r1779222812 ## core/src/main/java/org/apache/iceberg/rest/responses/PlanTableScanResponse.java: ## @@ -0,0 +1,54 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] fix: DayTransform result type override and docs [iceberg-python]

kevinzwang commented on PR #1208: URL: https://github.com/apache/iceberg-python/pull/1208#issuecomment-2379885390 Perhaps let's just wait on the response on the devlist first. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

Re: [PR] Core: fix NPE with HadoopFileIO because FileIOParser doesn't serialize Hadoop configuration [iceberg]

rdblue commented on PR #10926: URL: https://github.com/apache/iceberg/pull/10926#issuecomment-2379910160 I don't think that we should change how this works. A Hadoop Configuration MUST be provided externally. FileIO serialization is not intended to send the entire Hadoop Configuration and s

Re: [PR] Spec: Add v3 types and type promotion [iceberg]

rdblue commented on code in PR #10955: URL: https://github.com/apache/iceberg/pull/10955#discussion_r1779172676 ## format/spec.md: ## @@ -950,6 +977,7 @@ Maps with non-string keys must use an array representation with the `map` logica |**`uuid`**|`{ "type": "fixed",`  `"size":

Re: [PR] Spec: Add v3 types and type promotion [iceberg]

rdblue commented on code in PR #10955: URL: https://github.com/apache/iceberg/pull/10955#discussion_r1779173577 ## format/spec.md: ## @@ -222,11 +228,31 @@ Schemas may be evolved by type promotion or adding, deleting, renaming, or reord Evolution applies changes to the table

Re: [PR] Spec: Add v3 types and type promotion [iceberg]

rdblue commented on code in PR #10955: URL: https://github.com/apache/iceberg/pull/10955#discussion_r1779173577 ## format/spec.md: ## @@ -222,11 +228,31 @@ Schemas may be evolved by type promotion or adding, deleting, renaming, or reord Evolution applies changes to the table

Re: [PR] API, Core: Add scan planning api request and response models [iceberg]

rahil-c commented on code in PR #11180: URL: https://github.com/apache/iceberg/pull/11180#discussion_r1778831626 ## core/src/main/java/org/apache/iceberg/rest/requests/FetchScanTasksRequest.java: ## @@ -0,0 +1,35 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] Spec: Add v3 types and type promotion [iceberg]

rdblue commented on PR #10955: URL: https://github.com/apache/iceberg/pull/10955#issuecomment-2380082777 @aihuaxu, @RussellSpitzer, I've removed variant from this PR so that we can make progress on the type promotion. Now this just includes `unknown` and the new promotions. We can make prog

Re: [PR] Spec: Add v3 types and type promotion [iceberg]

rdblue commented on code in PR #10955: URL: https://github.com/apache/iceberg/pull/10955#discussion_r177917 ## format/spec.md: ## @@ -174,6 +174,7 @@ Supported primitive types are defined in the table below. Primitive types added | Added by version | Primitive type |

Re: [PR] Spec: Add v3 types and type promotion [iceberg]

rdblue commented on code in PR #10955: URL: https://github.com/apache/iceberg/pull/10955#discussion_r1779174675 ## format/spec.md: ## @@ -222,11 +228,31 @@ Schemas may be evolved by type promotion or adding, deleting, renaming, or reord Evolution applies changes to the table

Re: [PR] Core: Add DataFileSet / DeleteFileSet [iceberg]

nastra commented on code in PR #11195: URL: https://github.com/apache/iceberg/pull/11195#discussion_r1778254044 ## api/src/main/java/org/apache/iceberg/util/DataFileSet.java: ## @@ -0,0 +1,113 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more cont

Re: [PR] test (datafusion): add test for table provider creation [iceberg-rust]

FANNG1 commented on code in PR #651: URL: https://github.com/apache/iceberg-rust/pull/651#discussion_r1778674865 ## crates/integrations/datafusion/src/table.rs: ## @@ -110,3 +110,43 @@ impl TableProvider for IcebergTableProvider { Ok(filter_support) } } + +#[cfg(t

Re: [I] Spark SQL UI can't show scan metrics. [iceberg]

gmurro-armani commented on issue #11191: URL: https://github.com/apache/iceberg/issues/11191#issuecomment-2378946855 Using the same version of Spark and Iceberg, I'm experiencing the same problem. I've checked the jar I'm using (iceberg-spark-runtime-3.5_2.12-1.6.1). It has the TotalDat

Re: [PR] Spec: Add section on `null_value_counts` [iceberg]

zhongyujiang commented on code in PR #8611: URL: https://github.com/apache/iceberg/pull/8611#discussion_r1778399294 ## format/spec.md: ## @@ -434,7 +434,8 @@ The schema of a manifest file is a struct called `manifest_entry` with the follo | _optional_ || ~~**`107

Re: [PR] Core: Add DataFileSet / DeleteFileSet [iceberg]

nastra commented on code in PR #11195: URL: https://github.com/apache/iceberg/pull/11195#discussion_r1778689365 ## api/src/main/java/org/apache/iceberg/util/DataFileSet.java: ## @@ -0,0 +1,113 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more cont

Re: [PR] Core: Add DataFileSet / DeleteFileSet [iceberg]

nastra commented on code in PR #11195: URL: https://github.com/apache/iceberg/pull/11195#discussion_r1778689365 ## api/src/main/java/org/apache/iceberg/util/DataFileSet.java: ## @@ -0,0 +1,113 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more cont

Re: [PR] test (datafusion): add test for table provider creation [iceberg-rust]

a-agmon commented on code in PR #651: URL: https://github.com/apache/iceberg-rust/pull/651#discussion_r1778693746 ## crates/integrations/datafusion/src/table.rs: ## @@ -110,3 +110,43 @@ impl TableProvider for IcebergTableProvider { Ok(filter_support) } } + +#[cfg(

Re: [PR] Core: Add DataFileSet / DeleteFileSet [iceberg]

nastra commented on code in PR #11195: URL: https://github.com/apache/iceberg/pull/11195#discussion_r1778100059 ## api/src/main/java/org/apache/iceberg/util/DataFileSet.java: ## @@ -0,0 +1,113 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more cont

Re: [PR] Spark partial limit push down [iceberg]

huaxingao commented on code in PR #10943: URL: https://github.com/apache/iceberg/pull/10943#discussion_r1778973376 ## parquet/src/main/java/org/apache/iceberg/parquet/Parquet.java: ## @@ -1151,6 +1152,11 @@ public ReadBuilder withAADPrefix(ByteBuffer aadPrefix) { return t

Re: [PR] Spark partial limit push down [iceberg]

huaxingao commented on code in PR #10943: URL: https://github.com/apache/iceberg/pull/10943#discussion_r1778973584 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/SparkScanBuilder.java: ## @@ -407,14 +422,35 @@ public Scan build() { private Scan buildBatch

Re: [PR] API, Core: Add scan planning api request and response models [iceberg]

rahil-c commented on code in PR #11180: URL: https://github.com/apache/iceberg/pull/11180#discussion_r1778821918 ## core/src/main/java/org/apache/iceberg/rest/PlanStatus.java: ## @@ -0,0 +1,36 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more cont

Re: [PR] Spark partial limit push down [iceberg]

RussellSpitzer commented on code in PR #10943: URL: https://github.com/apache/iceberg/pull/10943#discussion_r1779152923 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/SparkPartitioningAwareScan.java: ## @@ -78,7 +78,19 @@ abstract class SparkPartitioningAwareS

Re: [PR] Arrow: add support for null vectors [iceberg]

slessard commented on code in PR #10953: URL: https://github.com/apache/iceberg/pull/10953#discussion_r1778901825 ## arrow/src/main/java/org/apache/iceberg/arrow/vectorized/VectorHolder.java: ## @@ -140,12 +141,18 @@ public static class ConstantVectorHolder extends VectorHolder

[I] Spark vectorized read of Parquet produces incorrect result for a decimal column [iceberg]

wypoon opened a new issue, #11221: URL: https://github.com/apache/iceberg/issues/11221 ### Apache Iceberg version main (development) ### Query engine Spark ### Please describe the bug 🐞 The bug is present in Iceberg 1.2 and later (and is in main). A custo

Re: [I] Spark vectorized read of Parquet produces incorrect result for a decimal column [iceberg]

wypoon commented on issue #11221: URL: https://github.com/apache/iceberg/issues/11221#issuecomment-2379964488 In Iceberg 1.1, a different bug occurs when reading the Iceberg table; the read fails altogether due to: ``` ERROR org.apache.iceberg.spark.source.BaseReader - Error reading fi

Re: [PR] Spec: Adds Row Lineage [iceberg]

flyrain commented on code in PR #11130: URL: https://github.com/apache/iceberg/pull/11130#discussion_r1777877117 ## format/spec.md: ## @@ -298,16 +298,143 @@ Iceberg tables must not use field ids greater than 2147483447 (`Integer.MAX_VALU The set of metadata columns is: -|

Re: [PR] API, Core: Add scan planning api request and response models [iceberg]

amogh-jahagirdar commented on code in PR #11180: URL: https://github.com/apache/iceberg/pull/11180#discussion_r1778956576 ## core/src/main/java/org/apache/iceberg/rest/responses/FetchPlanningResultResponse.java: ## @@ -0,0 +1,51 @@ +/* + * Licensed to the Apache Software Foundat

[PR] Update TableMetadataParser to close streams [iceberg]

erik-grepr opened a new pull request, #11220: URL: https://github.com/apache/iceberg/pull/11220 Minor try-with-resources tweak to fix an unclosed stream memory leak I encountered. More specifically, this occurs when the file is Gzipped and file.newStream().read() throws an exception.

Re: [PR] API, Core: Add scan planning api request and response models [iceberg]

rahil-c commented on code in PR #11180: URL: https://github.com/apache/iceberg/pull/11180#discussion_r1778921691 ## core/src/main/java/org/apache/iceberg/rest/responses/FetchPlanningResultResponse.java: ## @@ -0,0 +1,51 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] API, Core: Add scan planning api request and response models [iceberg]

amogh-jahagirdar commented on code in PR #11180: URL: https://github.com/apache/iceberg/pull/11180#discussion_r1778956576 ## core/src/main/java/org/apache/iceberg/rest/responses/FetchPlanningResultResponse.java: ## @@ -0,0 +1,51 @@ +/* + * Licensed to the Apache Software Foundat

Re: [I] Spark vectorized read of Parquet produces incorrect result for a decimal column [iceberg]

wypoon commented on issue #11221: URL: https://github.com/apache/iceberg/issues/11221#issuecomment-2379978697 cc @nastra -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] Core: fix NPE with HadoopFileIO because FileIOParser doesn't serialize Hadoop configuration [iceberg]

stevenzwu commented on PR #10926: URL: https://github.com/apache/iceberg/pull/10926#issuecomment-2379987754 > I don't think that we should change how this works. A Hadoop Configuration MUST be provided externally. This makes sense. We already have some consensus that the current PR ne

Re: [PR] Spec: Adds Row Lineage [iceberg]

RussellSpitzer commented on code in PR #11130: URL: https://github.com/apache/iceberg/pull/11130#discussion_r1779110605 ## format/spec.md: ## @@ -298,16 +298,143 @@ Iceberg tables must not use field ids greater than 2147483447 (`Integer.MAX_VALU The set of metadata columns i

Re: [PR] Spec: Adds Row Lineage [iceberg]

RussellSpitzer commented on code in PR #11130: URL: https://github.com/apache/iceberg/pull/11130#discussion_r1779115055 ## format/spec.md: ## @@ -298,16 +298,143 @@ Iceberg tables must not use field ids greater than 2147483447 (`Integer.MAX_VALU The set of metadata columns i

Re: [PR] Spec: Adds Row Lineage [iceberg]

RussellSpitzer commented on code in PR #11130: URL: https://github.com/apache/iceberg/pull/11130#discussion_r1779115886 ## format/spec.md: ## @@ -298,16 +298,143 @@ Iceberg tables must not use field ids greater than 2147483447 (`Integer.MAX_VALU The set of metadata columns i

Re: [PR] Spec: Adds Row Lineage [iceberg]

RussellSpitzer commented on code in PR #11130: URL: https://github.com/apache/iceberg/pull/11130#discussion_r1779120253 ## format/spec.md: ## @@ -598,6 +702,14 @@ Notes: 1. Lower and upper bounds are serialized to bytes using the single-object serialization in Appendix D. The

Re: [I] Spark vectorized read of Parquet produces incorrect result for a decimal column [iceberg]

wypoon commented on issue #11221: URL: https://github.com/apache/iceberg/issues/11221#issuecomment-2379978266 [impala_test_data.txt](https://github.com/user-attachments/files/17169180/impala_test_data.txt) Please rename the file (I used a .txt extension just to workaround my Mac prev

Re: [PR] Spec: Adds Row Lineage [iceberg]

RussellSpitzer commented on code in PR #11130: URL: https://github.com/apache/iceberg/pull/11130#discussion_r1779125152 ## format/spec.md: ## @@ -298,16 +298,101 @@ Iceberg tables must not use field ids greater than 2147483447 (`Integer.MAX_VALU The set of metadata columns i

Re: [PR] Spec: Add v3 types and type promotion [iceberg]

emkornfield commented on code in PR #10955: URL: https://github.com/apache/iceberg/pull/10955#discussion_r1779248853 ## format/spec.md: ## @@ -230,11 +233,31 @@ Schemas may be evolved by type promotion or adding, deleting, renaming, or reord Evolution applies changes to the

Re: [PR] Spec: Add v3 types and type promotion [iceberg]

emkornfield commented on code in PR #10955: URL: https://github.com/apache/iceberg/pull/10955#discussion_r1779268258 ## format/spec.md: ## @@ -174,6 +174,7 @@ Supported primitive types are defined in the table below. Primitive types added | Added by version | Primitive type

Re: [PR] Spec: Add v3 types and type promotion [iceberg]

emkornfield commented on code in PR #10955: URL: https://github.com/apache/iceberg/pull/10955#discussion_r1779268706 ## format/spec.md: ## @@ -230,11 +233,31 @@ Schemas may be evolved by type promotion or adding, deleting, renaming, or reord Evolution applies changes to the

Re: [PR] Spec: Adds Row Lineage [iceberg]

RussellSpitzer commented on code in PR #11130: URL: https://github.com/apache/iceberg/pull/11130#discussion_r1779126152 ## format/spec.md: ## @@ -298,16 +298,101 @@ Iceberg tables must not use field ids greater than 2147483447 (`Integer.MAX_VALU The set of metadata columns i

Re: [PR] Core: Add DataFileSet / DeleteFileSet [iceberg]

nastra commented on code in PR #11195: URL: https://github.com/apache/iceberg/pull/11195#discussion_r1778251131 ## api/src/main/java/org/apache/iceberg/util/WrapperSet.java: ## @@ -0,0 +1,185 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contr

Re: [PR] Core: Add DataFileSet / DeleteFileSet [iceberg]

nastra commented on code in PR #11195: URL: https://github.com/apache/iceberg/pull/11195#discussion_r1778254044 ## api/src/main/java/org/apache/iceberg/util/DataFileSet.java: ## @@ -0,0 +1,113 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more cont

[PR] WIP: Table Scan Delete File Handling [iceberg-rust]

sdd opened a new pull request, #652: URL: https://github.com/apache/iceberg-rust/pull/652 This Draft PR outlines an approach to add support for proper handling of delete files within table scans. The approach taken is to include a list of delete file paths in every `FileScanTask`. At

Re: [PR] Spec: Add section on `null_value_counts` [iceberg]

zhongyujiang commented on code in PR #8611: URL: https://github.com/apache/iceberg/pull/8611#discussion_r1778399294 ## format/spec.md: ## @@ -434,7 +434,8 @@ The schema of a manifest file is a struct called `manifest_entry` with the follo | _optional_ || ~~**`107

Re: [I] Support Nessie catalog [iceberg-python]

XN137 commented on issue #19: URL: https://github.com/apache/iceberg-python/issues/19#issuecomment-2378950088 `RestCatalog` class seems to live in `pyiceberg.catalog.rest`: https://github.com/apache/iceberg-python/blob/c30e43adf94a82ec1a225d3a1bf69fface592cfd/pyiceberg/catalog/rest.py#L24

Re: [PR] Core: Add DataFileSet / DeleteFileSet [iceberg]

nastra commented on code in PR #11195: URL: https://github.com/apache/iceberg/pull/11195#discussion_r1778454236 ## api/src/main/java/org/apache/iceberg/util/DataFileSet.java: ## @@ -0,0 +1,113 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more cont

Re: [I] Do not deprecate Botocore Session in upcoming release (0.8) [iceberg-python]

cshenrik commented on issue #1104: URL: https://github.com/apache/iceberg-python/issues/1104#issuecomment-2378587661 Thanks for sharing that, @BTheunissen. I have to call a bespoke webservice for retrieving AWS credentials, so I can't use that implementation directly, but it's still

Re: [PR] Manifest list encryption [iceberg]

ggershinsky commented on PR #7770: URL: https://github.com/apache/iceberg/pull/7770#issuecomment-2379298550 Hi @rdblue , I've built the integration code with the latest version of this patch, works ok. Can we merge this PR? -- This is an automated message from the Apache Git Service. To r

Re: [PR] test (datafusion): add test for table provider creation [iceberg-rust]

a-agmon commented on code in PR #651: URL: https://github.com/apache/iceberg-rust/pull/651#discussion_r1778693746 ## crates/integrations/datafusion/src/table.rs: ## @@ -110,3 +110,43 @@ impl TableProvider for IcebergTableProvider { Ok(filter_support) } } + +#[cfg(

Re: [PR] AWS: Set better defaults for S3 retry behaviour [iceberg]

danielcweeks commented on PR #11052: URL: https://github.com/apache/iceberg/pull/11052#issuecomment-2379484397 @ookumuso After thinking about this a little more, I'm increasingly concerned about the high value for defaults. If we look at this in isolation, it seems like the right thing to

Re: [PR] Spark: Add RewriteTablePath action interface [iceberg]

laithalzyoud commented on PR #10920: URL: https://github.com/apache/iceberg/pull/10920#issuecomment-2379580439 Hey @nastra @manuzhang @anuragmantri @flyrain! Can we merge this now? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

Re: [PR] API, Core: Add scan planning api request and response models [iceberg]

rahil-c commented on code in PR #11180: URL: https://github.com/apache/iceberg/pull/11180#discussion_r1778837559 ## core/src/main/java/org/apache/iceberg/rest/requests/PlanTableScanRequest.java: ## @@ -0,0 +1,67 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under o

Re: [I] Support Nessie catalog [iceberg-python]

cee-shubham commented on issue #19: URL: https://github.com/apache/iceberg-python/issues/19#issuecomment-2378847628 generally speaking you use the REST catalog these docs may help: https://py.iceberg.apache.org/configuration/#rest-catalog i have already tried it but it did not wo

Re: [PR] Spec: Adds Row Lineage [iceberg]

RussellSpitzer commented on code in PR #11130: URL: https://github.com/apache/iceberg/pull/11130#discussion_r1778877373 ## format/spec.md: ## @@ -298,16 +298,137 @@ Iceberg tables must not use field ids greater than 2147483447 (`Integer.MAX_VALU The set of metadata columns i

Re: [PR] API, Core: Add scan planning api request and response models [iceberg]

rahil-c commented on code in PR #11180: URL: https://github.com/apache/iceberg/pull/11180#discussion_r1778836954 ## core/src/main/java/org/apache/iceberg/rest/requests/PlanTableScanRequest.java: ## @@ -0,0 +1,67 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under o

Re: [I] Support Nessie catalog [iceberg-python]

cee-shubham commented on issue #19: URL: https://github.com/apache/iceberg-python/issues/19#issuecomment-2379053049 RestCatalog class seems to live in pyiceberg.catalog.rest: yeah i tried that only Tried this also:catalog = load_catalog("rest", ) but while giving optional config

Re: [PR] Core: Add internal Avro reader [iceberg]

aokolnychyi commented on code in PR #11108: URL: https://github.com/apache/iceberg/pull/11108#discussion_r1779286019 ## core/src/main/java/org/apache/iceberg/ManifestReader.java: ## @@ -261,12 +262,12 @@ private CloseableIterable> open(Schema projection) { AvroIterable

Re: [PR] Core: Add internal Avro reader [iceberg]

aokolnychyi commented on code in PR #11108: URL: https://github.com/apache/iceberg/pull/11108#discussion_r1779299588 ## core/src/test/java/org/apache/iceberg/TestManifestReader.java: ## @@ -44,7 +44,8 @@ public class TestManifestReader extends TestBase { "fileOrdi

Re: [PR] Spark partial limit push down [iceberg]

huaxingao commented on code in PR #10943: URL: https://github.com/apache/iceberg/pull/10943#discussion_r1779324357 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/SparkPartitioningAwareScan.java: ## @@ -78,7 +78,19 @@ abstract class SparkPartitioningAwareScan e

Re: [I] discuss: re-export arrow types [iceberg-rust]

Xuanwo commented on issue #636: URL: https://github.com/apache/iceberg-rust/issues/636#issuecomment-2380432758 -0 on this. I'm not in favor of re-exporting other crates based on my own experience. However, as @xxchan mentioned, there aren't many drawbacks, so I'm okay if someone want

[PR] Core: Support combining position deletes during writes [iceberg]

aokolnychyi opened a new pull request, #11222: URL: https://github.com/apache/iceberg/pull/11222 This PR adds support for combing historical position deletes in writers, enabling sync maintenance. -- This is an automated message from the Apache Git Service. To respond to the message, plea

Re: [PR] Core: Support combining position deletes during writes [iceberg]

aokolnychyi commented on code in PR #11222: URL: https://github.com/apache/iceberg/pull/11222#discussion_r1779279363 ## core/src/main/java/org/apache/iceberg/deletes/Deletes.java: ## @@ -130,26 +131,32 @@ public static StructLikeSet toEqualitySet( } } + public static

Re: [PR] Core: Add DataFileSet / DeleteFileSet [iceberg]

aokolnychyi commented on code in PR #11195: URL: https://github.com/apache/iceberg/pull/11195#discussion_r1779304973 ## api/src/main/java/org/apache/iceberg/util/WrapperSet.java: ## @@ -0,0 +1,185 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more

Re: [PR] Core: Add DataFileSet / DeleteFileSet [iceberg]

aokolnychyi commented on code in PR #11195: URL: https://github.com/apache/iceberg/pull/11195#discussion_r1779311673 ## api/src/main/java/org/apache/iceberg/util/DataFileSet.java: ## @@ -0,0 +1,149 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more

Re: [PR] Core: Support combining position deletes during writes [iceberg]

aokolnychyi commented on code in PR #11222: URL: https://github.com/apache/iceberg/pull/11222#discussion_r1779279363 ## core/src/main/java/org/apache/iceberg/deletes/Deletes.java: ## @@ -130,26 +131,32 @@ public static StructLikeSet toEqualitySet( } } + public static

Re: [I] org.apache.iceberg.spark.source.SerializableTableWithSize cannot be cast to org.apache.iceberg.Table [iceberg]

github-actions[bot] commented on issue #8978: URL: https://github.com/apache/iceberg/issues/8978#issuecomment-2380308568 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] View is no longer in sync with table after catalog cache entry expires [iceberg]

github-actions[bot] commented on issue #8977: URL: https://github.com/apache/iceberg/issues/8977#issuecomment-2380308560 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] Why are updateSchema and UpdatePartitionSpec commit not retried？ [iceberg]

github-actions[bot] commented on issue #8964: URL: https://github.com/apache/iceberg/issues/8964#issuecomment-2380308493 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] Question on BaseMetastoreViewCatalog#buildView [iceberg]

github-actions[bot] commented on issue #8967: URL: https://github.com/apache/iceberg/issues/8967#issuecomment-2380308518 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] flink1.13.2+iceberg0.13.0+hive-metastore3.0.0+minio（S3) Forbidden (Service: Amazon S3; Status Code: 403 [iceberg]

github-actions[bot] commented on issue #8968: URL: https://github.com/apache/iceberg/issues/8968#issuecomment-2380308532 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] Long overflow when Iceberg reading INT96 timestamp column from Spark parquet table [iceberg]

github-actions[bot] commented on issue #8949: URL: https://github.com/apache/iceberg/issues/8949#issuecomment-2380308468 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] Does the Java API support primary keys for creating tables [iceberg]

github-actions[bot] commented on issue #8950: URL: https://github.com/apache/iceberg/issues/8950#issuecomment-2380308477 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] equality delete files can be removed immediately after rewrite? [iceberg]

github-actions[bot] commented on issue #8933: URL: https://github.com/apache/iceberg/issues/8933#issuecomment-2380308452 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] Support adding an additional `opType` column when creating a table [iceberg]

github-actions[bot] commented on issue #8973: URL: https://github.com/apache/iceberg/issues/8973#issuecomment-2380308537 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] Support MOR CDC view [iceberg]

github-actions[bot] commented on issue #8975: URL: https://github.com/apache/iceberg/issues/8975#issuecomment-2380308549 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] Docs: Fix MkDocs ASF nav links [iceberg]

github-actions[bot] commented on issue #8965: URL: https://github.com/apache/iceberg/issues/8965#issuecomment-2380308505 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [PR] API, Core: Add scan planning api request and response models [iceberg]

rahil-c commented on code in PR #11180: URL: https://github.com/apache/iceberg/pull/11180#discussion_r1778836315 ## core/src/main/java/org/apache/iceberg/rest/requests/FetchScanTasksRequest.java: ## @@ -0,0 +1,35 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] API, Core: Add scan planning api request and response models [iceberg]

rahil-c commented on code in PR #11180: URL: https://github.com/apache/iceberg/pull/11180#discussion_r1778821628 ## api/src/main/java/org/apache/iceberg/exceptions/NoSuchPlanIdException.java: ## @@ -0,0 +1,33 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

Re: [PR] test (datafusion): add test for table provider creation [iceberg-rust]

a-agmon commented on code in PR #651: URL: https://github.com/apache/iceberg-rust/pull/651#discussion_r1778719810 ## crates/integrations/datafusion/src/table.rs: ## @@ -110,3 +110,43 @@ impl TableProvider for IcebergTableProvider { Ok(filter_support) } } + +#[cfg(

Re: [I] Why not use the profile name when initialising the S3FileSystem class? [iceberg-python]

wudihero2 commented on issue #1207: URL: https://github.com/apache/iceberg-python/issues/1207#issuecomment-2379659325 Hi all, I checked the code of [pyarrow](https://github.com/apache/arrow/blob/6f64af54ea3662b7e1685b84a81429df14beace1/python/pyarrow/_s3fs.pyx#L278-L288) and found that the