Re: [PR] Core: Add a util to compute partition stats [iceberg]

aokolnychyi commented on code in PR #11146: URL: https://github.com/apache/iceberg/pull/11146#discussion_r1772724334 ## core/src/main/java/org/apache/iceberg/PartitionStatsUtil.java: ## @@ -0,0 +1,136 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or m

Re: [PR] Always update metadata when Table.refresh() is called [iceberg]

eric-maynard closed pull request #11193: Always update metadata when Table.refresh() is called URL: https://github.com/apache/iceberg/pull/11193 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

Re: [I] DatasourceV2 does not prune columns after V2ScanRelationPushDown [iceberg]

IgorBerman commented on issue #9268: URL: https://github.com/apache/iceberg/issues/9268#issuecomment-2370296642 Hi @rdblue and @aokolnychyi Do you have new ideas regarding this issue and in general maybe you can provide pointers if Iceberg implements column pruning for highly nested sch

Re: [PR] Updating SparkScan to only read Apache DataSketches [iceberg]

aokolnychyi commented on PR #11035: URL: https://github.com/apache/iceberg/pull/11035#issuecomment-2370250512 I'll check tomorrow. Sorry for the delay! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

Re: [PR] feat: Add NamespaceIdent.parent() [iceberg-rust]

Xuanwo commented on code in PR #641: URL: https://github.com/apache/iceberg-rust/pull/641#discussion_r1772486755 ## crates/iceberg/src/catalog/mod.rs: ## @@ -133,6 +133,16 @@ impl NamespaceIdent { pub fn inner(self) -> Vec { self.0 } + +/// Get the parent

Re: [PR] feat: Add NamespaceIdent.parent() [iceberg-rust]

Xuanwo merged PR #641: URL: https://github.com/apache/iceberg-rust/pull/641 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.a

[PR] Always update table metadata when `refresh` is called [iceberg]

eric-maynard opened a new pull request, #11194: URL: https://github.com/apache/iceberg/pull/11194 In PyIceberg, calling `refresh` on a table [always updates the metadata](https://github.com/apache/iceberg-python/blob/620ad9f64307193ec0d26846b48f4e063b5da904/pyiceberg/table/__init__.py#L803).

Re: [PR] [Docs] Update AWS docs to add more AWS engines that supports iceberg [iceberg]

singhpk234 commented on PR #11192: URL: https://github.com/apache/iceberg/pull/11192#issuecomment-2369970995 Thanks @amogh-jahagirdar ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

Re: [PR] [Docs] Update AWS docs to add more AWS engines that supports iceberg [iceberg]

amogh-jahagirdar merged PR #11192: URL: https://github.com/apache/iceberg/pull/11192 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@

Re: [I] Pyiceberg support the query without provided snapshot_id [iceberg-python]

github-actions[bot] commented on issue #553: URL: https://github.com/apache/iceberg-python/issues/553#issuecomment-2369826677 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity oc

Re: [I] support meta column query on staged scan [iceberg]

github-actions[bot] commented on issue #8866: URL: https://github.com/apache/iceberg/issues/8866#issuecomment-2369824156 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [PR] Core: Optimize the process of find start delete file index [iceberg]

github-actions[bot] commented on PR #: URL: https://github.com/apache/iceberg/pull/#issuecomment-2369824230 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull

Re: [I] fast_forward command not merging branches within AWS Glue [iceberg]

github-actions[bot] commented on issue #8881: URL: https://github.com/apache/iceberg/issues/8881#issuecomment-2369824201 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] java.lang.IllegalArgumentException: requirement failed while read migrated parquet table [iceberg]

github-actions[bot] commented on issue #8863: URL: https://github.com/apache/iceberg/issues/8863#issuecomment-2369824134 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] Apache hive 3 with Tez engine select table no empty [iceberg]

github-actions[bot] commented on issue #8891: URL: https://github.com/apache/iceberg/issues/8891#issuecomment-2369824250 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] Flink: OverflowError: value too large to convert to int32_t [iceberg]

github-actions[bot] commented on issue #8874: URL: https://github.com/apache/iceberg/issues/8874#issuecomment-2369824175 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [PR] Cache filesToImport variable in importSparkPartitions to avoid duplicated compute [iceberg]

github-actions[bot] commented on PR #8505: URL: https://github.com/apache/iceberg/pull/8505#issuecomment-2369823706 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Add metricsConfig when build writer [iceberg]

github-actions[bot] commented on PR #8498: URL: https://github.com/apache/iceberg/pull/8498#issuecomment-2369823682 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Cache filesToImport variable in importSparkPartitions to avoid duplicated compute [iceberg]

github-actions[bot] closed pull request #8505: Cache filesToImport variable in importSparkPartitions to avoid duplicated compute URL: https://github.com/apache/iceberg/pull/8505 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] Add metricsConfig when build writer [iceberg]

github-actions[bot] closed pull request #8498: Add metricsConfig when build writer URL: https://github.com/apache/iceberg/pull/8498 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

Re: [I] Create table should take in sort order/ distribution mode [iceberg]

github-actions[bot] closed issue #8179: Create table should take in sort order/ distribution mode URL: https://github.com/apache/iceberg/issues/8179 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

Re: [I] Why the logical types are handled differently between Iceberg-Avro and Iceberg-Parquet? [iceberg]

github-actions[bot] closed issue #8176: Why the logical types are handled differently between Iceberg-Avro and Iceberg-Parquet? URL: https://github.com/apache/iceberg/issues/8176 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

Re: [I] Create table should take in sort order/ distribution mode [iceberg]

github-actions[bot] commented on issue #8179: URL: https://github.com/apache/iceberg/issues/8179#issuecomment-2369823576 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Why the logical types are handled differently between Iceberg-Avro and Iceberg-Parquet? [iceberg]

github-actions[bot] commented on issue #8176: URL: https://github.com/apache/iceberg/issues/8176#issuecomment-2369823551 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Support partitioning and sorting on nested struct [iceberg]

github-actions[bot] commented on issue #8175: URL: https://github.com/apache/iceberg/issues/8175#issuecomment-2369823524 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Support partitioning and sorting on nested struct [iceberg]

github-actions[bot] closed issue #8175: Support partitioning and sorting on nested struct URL: https://github.com/apache/iceberg/issues/8175 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

Re: [PR] [Docs] Update AWS docs to add more AWS engines that supports iceberg [iceberg]

amogh-jahagirdar commented on code in PR #11192: URL: https://github.com/apache/iceberg/pull/11192#discussion_r1772351307 ## docs/docs/aws.md: ## @@ -685,3 +685,9 @@ Search the [Iceberg blogs](../../blogs.md) page for tutorials around running Ice [Amazon Kinesis Data Analyt

Re: [PR] Spec: Adds Row Lineage [iceberg]

rdblue commented on code in PR #11130: URL: https://github.com/apache/iceberg/pull/11130#discussion_r1772352197 ## format/spec.md: ## @@ -298,16 +298,143 @@ Iceberg tables must not use field ids greater than 2147483447 (`Integer.MAX_VALU The set of metadata columns is: -|

Re: [PR] Spec: Adds Row Lineage [iceberg]

rdblue commented on code in PR #11130: URL: https://github.com/apache/iceberg/pull/11130#discussion_r1772351230 ## format/spec.md: ## @@ -298,16 +298,143 @@ Iceberg tables must not use field ids greater than 2147483447 (`Integer.MAX_VALU The set of metadata columns is: -|

Re: [PR] Spec: Adds Row Lineage [iceberg]

rdblue commented on code in PR #11130: URL: https://github.com/apache/iceberg/pull/11130#discussion_r1772351230 ## format/spec.md: ## @@ -298,16 +298,143 @@ Iceberg tables must not use field ids greater than 2147483447 (`Integer.MAX_VALU The set of metadata columns is: -|

Re: [PR] Spec: Adds Row Lineage [iceberg]

rdblue commented on code in PR #11130: URL: https://github.com/apache/iceberg/pull/11130#discussion_r1772350869 ## format/spec.md: ## @@ -298,16 +298,143 @@ Iceberg tables must not use field ids greater than 2147483447 (`Integer.MAX_VALU The set of metadata columns is: -|

Re: [PR] Spec: Adds Row Lineage [iceberg]

rdblue commented on code in PR #11130: URL: https://github.com/apache/iceberg/pull/11130#discussion_r1772348560 ## format/spec.md: ## @@ -298,16 +298,143 @@ Iceberg tables must not use field ids greater than 2147483447 (`Integer.MAX_VALU The set of metadata columns is: -|

Re: [PR] Spec: Adds Row Lineage [iceberg]

rdblue commented on code in PR #11130: URL: https://github.com/apache/iceberg/pull/11130#discussion_r1772348376 ## format/spec.md: ## @@ -298,16 +298,143 @@ Iceberg tables must not use field ids greater than 2147483447 (`Integer.MAX_VALU The set of metadata columns is: -|

Re: [PR] Spec: Adds Row Lineage [iceberg]

rdblue commented on code in PR #11130: URL: https://github.com/apache/iceberg/pull/11130#discussion_r1772347415 ## format/spec.md: ## @@ -298,16 +298,143 @@ Iceberg tables must not use field ids greater than 2147483447 (`Integer.MAX_VALU The set of metadata columns is: -|

Re: [PR] API, Core: Add scan planning api request and response models [iceberg]

rdblue commented on code in PR #11180: URL: https://github.com/apache/iceberg/pull/11180#discussion_r1772344733 ## core/src/main/java/org/apache/iceberg/rest/requests/PlanTableScanRequest.java: ## @@ -0,0 +1,67 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under on

Re: [PR] API, Core: Add scan planning api request and response models [iceberg]

rdblue commented on code in PR #11180: URL: https://github.com/apache/iceberg/pull/11180#discussion_r1772343985 ## core/src/main/java/org/apache/iceberg/rest/requests/PlanTableScanRequest.java: ## @@ -0,0 +1,67 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under on

Re: [PR] API, Core: Add scan planning api request and response models [iceberg]

rdblue commented on code in PR #11180: URL: https://github.com/apache/iceberg/pull/11180#discussion_r1772343366 ## core/src/main/java/org/apache/iceberg/rest/responses/PlanTableScanResponse.java: ## @@ -0,0 +1,54 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] API, Core: Add scan planning api request and response models [iceberg]

rdblue commented on code in PR #11180: URL: https://github.com/apache/iceberg/pull/11180#discussion_r1772342573 ## core/src/main/java/org/apache/iceberg/rest/responses/FetchPlanningResultResponse.java: ## @@ -0,0 +1,51 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] API, Core: Add scan planning api request and response models [iceberg]

rdblue commented on code in PR #11180: URL: https://github.com/apache/iceberg/pull/11180#discussion_r1772342030 ## core/src/main/java/org/apache/iceberg/rest/responses/FetchPlanningResultResponse.java: ## @@ -0,0 +1,51 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] API, Core: Add scan planning api request and response models [iceberg]

rdblue commented on code in PR #11180: URL: https://github.com/apache/iceberg/pull/11180#discussion_r1772340520 ## core/src/main/java/org/apache/iceberg/rest/requests/FetchScanTasksRequest.java: ## @@ -0,0 +1,35 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under o

Re: [PR] API, Core: Add scan planning api request and response models [iceberg]

rdblue commented on code in PR #11180: URL: https://github.com/apache/iceberg/pull/11180#discussion_r1772339858 ## core/src/main/java/org/apache/iceberg/rest/PlanStatus.java: ## @@ -0,0 +1,36 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contr

[PR] [Docs] Update AWS docs to add more AWS engines that supports iceberg [iceberg]

singhpk234 opened a new pull request, #11192: URL: https://github.com/apache/iceberg/pull/11192 ### About the change Have worked on adding support of iceberg in more AWS engines recently, Having this in our docs would certainly help in navigation and support. cc @amogh-jaha

[PR] Bump thrift from 0.20.0 to 0.21.0 [iceberg-python]

dependabot[bot] opened a new pull request, #1197: URL: https://github.com/apache/iceberg-python/pull/1197 Bumps [thrift](https://github.com/apache/thrift) from 0.20.0 to 0.21.0. Release notes Sourced from https://github.com/apache/thrift/releases";>thrift's releases. Tag to r

[PR] Bump mmh3 from 4.1.0 to 5.0.1 [iceberg-python]

dependabot[bot] opened a new pull request, #1196: URL: https://github.com/apache/iceberg-python/pull/1196 Bumps [mmh3](https://github.com/hajimes/mmh3) from 4.1.0 to 5.0.1. Release notes Sourced from https://github.com/hajimes/mmh3/releases";>mmh3's releases. v5.0.1 What's

[PR] Bump mkdocs-material from 9.5.35 to 9.5.36 [iceberg-python]

dependabot[bot] opened a new pull request, #1195: URL: https://github.com/apache/iceberg-python/pull/1195 Bumps [mkdocs-material](https://github.com/squidfunk/mkdocs-material) from 9.5.35 to 9.5.36. Release notes Sourced from https://github.com/squidfunk/mkdocs-material/releases";>

Re: [I] Fast Avro Decoder not included in Conda Deployment of pyiceberg [iceberg-python]

fcrimins commented on issue #1093: URL: https://github.com/apache/iceberg-python/issues/1093#issuecomment-2369589615 I must have missed the "Falling back to pure Python Avro decoder, missing Cython implementation" warning during installation, but I was indeed missing Cython, which was caus

Re: [PR] Spec: Support geo type [iceberg]

szehon-ho commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1772045484 ## format/spec.md: ## @@ -1312,7 +1325,7 @@ This serialization scheme is for storing single values as individual binary valu | **`struct`** | **`JSON object

Re: [PR] API: Deprecate ContentFile#path API and add location API which returns String [iceberg]

amogh-jahagirdar commented on PR #11092: URL: https://github.com/apache/iceberg/pull/11092#issuecomment-2369299394 THanks for reviewing @flyrain @nastra @aokolnychyi , I'll go ahead and merge -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [PR] API: Deprecate ContentFile#path API and add location API which returns String [iceberg]

amogh-jahagirdar merged PR #11092: URL: https://github.com/apache/iceberg/pull/11092 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@

[PR] feat: Safer PartitionSpec & SchemalessPartitionSpec [iceberg-rust]

c-thiel opened a new pull request, #645: URL: https://github.com/apache/iceberg-rust/pull/645 Fixes https://github.com/apache/iceberg-rust/issues/550 This PR is a result of the issue mentioned above, a [Slack Discussion]( https://apache-iceberg.slack.com/archives/C03LG1D563F/p17258788

Re: [PR] feat: Add NamespaceIdent.parent() [iceberg-rust]

c-thiel commented on code in PR #641: URL: https://github.com/apache/iceberg-rust/pull/641#discussion_r1772000307 ## crates/iceberg/src/catalog/mod.rs: ## @@ -133,6 +133,16 @@ impl NamespaceIdent { pub fn inner(self) -> Vec { self.0 } + +/// Get the parent

Re: [PR] Core: Add a util to compute partition stats [iceberg]

ajantha-bhat commented on code in PR #11146: URL: https://github.com/apache/iceberg/pull/11146#discussion_r1771927587 ## core/src/main/java/org/apache/iceberg/PartitionStats.java: ## @@ -0,0 +1,249 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more

Re: [PR] AWS: Introduce opt-in S3LocationProvider which is optimized for S3 performance [iceberg]

ookumuso commented on PR #2: URL: https://github.com/apache/iceberg/pull/2#issuecomment-2369065326 > > @ookumuso Overall, this looks like a great feature if this is better for S3 to repartition and distribute data, but it also seems like it would fit cleanly into the existing Object

Re: [PR] Core: Add a util to compute partition stats [iceberg]

ajantha-bhat commented on code in PR #11146: URL: https://github.com/apache/iceberg/pull/11146#discussion_r1771883618 ## core/src/main/java/org/apache/iceberg/PartitionStats.java: ## @@ -0,0 +1,249 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more

Re: [PR] Core: Add a util to compute partition stats [iceberg]

ajantha-bhat commented on code in PR #11146: URL: https://github.com/apache/iceberg/pull/11146#discussion_r1771881650 ## core/src/main/java/org/apache/iceberg/PartitionStats.java: ## @@ -0,0 +1,249 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more

Re: [PR] Core: Add a util to compute partition stats [iceberg]

ajantha-bhat commented on code in PR #11146: URL: https://github.com/apache/iceberg/pull/11146#discussion_r1771880686 ## core/src/main/java/org/apache/iceberg/PartitionStatsUtil.java: ## @@ -0,0 +1,150 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or

Re: [PR] Core: Add a util to compute partition stats [iceberg]

ajantha-bhat commented on code in PR #11146: URL: https://github.com/apache/iceberg/pull/11146#discussion_r1771880045 ## core/src/main/java/org/apache/iceberg/BaseScan.java: ## @@ -289,4 +289,21 @@ private static Schema lazyColumnProjection(TableScanContext context, Schema sche

Re: [PR] Core: Add a util to compute partition stats [iceberg]

ajantha-bhat commented on code in PR #11146: URL: https://github.com/apache/iceberg/pull/11146#discussion_r1771865504 ## core/src/main/java/org/apache/iceberg/PartitionStats.java: ## @@ -0,0 +1,249 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more

Re: [PR] Updating SparkScan to only read Apache DataSketches [iceberg]

jeesou commented on PR #11035: URL: https://github.com/apache/iceberg/pull/11035#issuecomment-2368935077 Hi @aokolnychyi could you please help review this PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

Re: [PR] API, AWS: Retry S3InputStream reads [iceberg]

amogh-jahagirdar commented on code in PR #10433: URL: https://github.com/apache/iceberg/pull/10433#discussion_r1771831100 ## aws/src/test/java/org/apache/iceberg/aws/s3/TestS3InputStream.java: ## @@ -52,18 +52,15 @@ public void before() { createBucket("bucket"); } - @

Re: [I] Support create table `PRIMARY KEY` column via Spark sql? [iceberg]

anuragmantri commented on issue #5069: URL: https://github.com/apache/iceberg/issues/5069#issuecomment-2368871238 There is a very old JIRA in Spark to add unenforced referential integrity constraints in Spark [SPARK-19842](https://issues.apache.org/jira/browse/SPARK-19842). This change is

Re: [PR] API, AWS: Retry S3InputStream reads [iceberg]

amogh-jahagirdar commented on code in PR #10433: URL: https://github.com/apache/iceberg/pull/10433#discussion_r1771783667 ## core/src/main/java/org/apache/iceberg/io/RetryableInputStream.java: ## @@ -0,0 +1,130 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under on

Re: [I] Remove python 3.8 support [iceberg-python]

kevinjqliu commented on issue #1121: URL: https://github.com/apache/iceberg-python/issues/1121#issuecomment-2368779184 @sungwy thanks! for following up, this fell off my radar -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [I] javax.net.ssl.SSLException: Connection reset on S3 w/ S3FileIO and Apache HTTP client [iceberg]

danielcweeks commented on issue #10340: URL: https://github.com/apache/iceberg/issues/10340#issuecomment-2368743193 @SandeepSinghGahir Thanks for the additional context (it really helps to have specifics like this). I think we're close to having a solution for this and @amogh-jahagirdar wi

Re: [PR] API, Core: Add scan planning api request and response models [iceberg]

rahil-c commented on PR #11180: URL: https://github.com/apache/iceberg/pull/11180#issuecomment-2368649315 > Thanks @rahil-c for kicking off the implementation! I wonder what others think but IMO this can be combined with serializer implementations that get registered in `RESTSerializers`. T

Re: [PR] Core: Add a util to compute partition stats [iceberg]

ajantha-bhat commented on code in PR #11146: URL: https://github.com/apache/iceberg/pull/11146#discussion_r1771606149 ## core/src/main/java/org/apache/iceberg/PartitionStatsUtil.java: ## @@ -0,0 +1,150 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or

Re: [PR] Bump pypa/cibuildwheel from 2.21.0 to 2.21.1 [iceberg-python]

sungwy merged PR #1181: URL: https://github.com/apache/iceberg-python/pull/1181 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@icebe

Re: [PR] Bump sqlalchemy from 2.0.34 to 2.0.35 [iceberg-python]

sungwy merged PR #1179: URL: https://github.com/apache/iceberg-python/pull/1179 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@icebe

Re: [PR] Bump mkdocs-material from 9.5.34 to 9.5.35 [iceberg-python]

sungwy merged PR #1184: URL: https://github.com/apache/iceberg-python/pull/1184 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@icebe

Re: [PR] scan: fix error when reading an empty table [iceberg-rust]

Xuanwo commented on PR #608: URL: https://github.com/apache/iceberg-rust/pull/608#issuecomment-2368247337 > We've been very selective when it comes to adding new values to `ErrorKind`. I'd personally go for `Unexpected` here - but maybe @liurenjie1024 or @Xuanwo can confirm what would be be

Re: [I] Minimum required pyarrow version [iceberg-python]

sungwy closed issue #1174: Minimum required pyarrow version URL: https://github.com/apache/iceberg-python/issues/1174 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubsc

Re: [I] Add integration tests with PostgreSQL DB backend [iceberg-python]

sungwy commented on issue #1178: URL: https://github.com/apache/iceberg-python/issues/1178#issuecomment-2368190069 Hi @akashd11 - yes I think that would be great. I think putting up a test container with postgres and parameterizing the tests in `tests/catalog/test_sql.py` will be a good wa

Re: [PR] Bump mypy-boto3-glue from 1.35.3 to 1.35.23 [iceberg-python]

sungwy merged PR #1186: URL: https://github.com/apache/iceberg-python/pull/1186 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@icebe

Re: [PR] Support python 3.12 in poetry [iceberg-python]

sungwy merged PR #1192: URL: https://github.com/apache/iceberg-python/pull/1192 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@icebe

Re: [I] Support setting a snapshot property in same commit as spark.sql [iceberg-python]

sungwy closed issue #368: Support setting a snapshot property in same commit as spark.sql URL: https://github.com/apache/iceberg-python/issues/368 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [I] Support setting a snapshot property in same commit as spark.sql [iceberg-python]

sungwy commented on issue #368: URL: https://github.com/apache/iceberg-python/issues/368#issuecomment-2368148002 Snapshot property can now be specified in PyIceberg Table APIs https://github.com/apache/iceberg-python/pull/419 -- This is an automated message from the Apache Git Servi

Re: [PR] Add Support for Dynamic Overwrite [iceberg-python]

sungwy commented on code in PR #931: URL: https://github.com/apache/iceberg-python/pull/931#discussion_r1771352661 ## pyiceberg/table/__init__.py: ## @@ -456,6 +461,89 @@ def append(self, df: pa.Table, snapshot_properties: Dict[str, str] = EMPTY_DICT) for data_

Re: [PR] API, Core: Add scan planning api request and response models [iceberg]

nastra commented on PR #11180: URL: https://github.com/apache/iceberg/pull/11180#issuecomment-2368122460 @rahil-c can you also please add respective JSON parser(s) with tests for those parser(s) as those request/response classes won't be serialized/deserialized -- This is an automated me

Re: [PR] API, Core: Add scan planning api request and response models [iceberg]

nastra commented on code in PR #11180: URL: https://github.com/apache/iceberg/pull/11180#discussion_r1771335798 ## core/src/main/java/org/apache/iceberg/rest/requests/FetchScanTasksRequest.java: ## @@ -0,0 +1,35 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under o

Re: [PR] Build: Bump org.roaringbitmap:RoaringBitmap from 1.2.1 to 1.3.0 [iceberg]

nastra merged PR #11187: URL: https://github.com/apache/iceberg/pull/11187 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Build: Bump com.google.cloud:libraries-bom from 26.44.0 to 26.47.0 [iceberg]

nastra merged PR #11185: URL: https://github.com/apache/iceberg/pull/11185 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Build: Bump nessie from 0.95.0 to 0.97.1 [iceberg]

nastra merged PR #11184: URL: https://github.com/apache/iceberg/pull/11184 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Build: Bump tez010 from 0.10.3 to 0.10.4 [iceberg]

nastra merged PR #11183: URL: https://github.com/apache/iceberg/pull/11183 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Build: Bump org.apache.httpcomponents.client5:httpclient5 from 5.3.1 to 5.4 [iceberg]

nastra merged PR #11186: URL: https://github.com/apache/iceberg/pull/11186 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [I] Convert datafusion table scan filter into iceberg table scan' filter. [iceberg-rust]

Xuanwo closed issue #585: Convert datafusion table scan filter into iceberg table scan' filter. URL: https://github.com/apache/iceberg-rust/issues/585 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] RecordBatchTransformer: Handle schema migration and column re-ordering in table scans [iceberg-rust]

Xuanwo commented on PR #602: URL: https://github.com/apache/iceberg-rust/pull/602#issuecomment-2367835715 Waiting for @liurenjie1024 to take another look. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

Re: [PR] feat: expose remove_all in FileIO [iceberg-rust]

Xuanwo merged PR #643: URL: https://github.com/apache/iceberg-rust/pull/643 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.a

[I] Spark SQL UI can't show scan metrics. [iceberg]

majian1998 opened a new issue, #11191: URL: https://github.com/apache/iceberg/issues/11191 ### Apache Iceberg version 1.6.1 (latest release) ### Query engine Spark ### Please describe the bug 🐞 I executed the following query using the Hadoop catalog: `sel

Re: [PR] Support changelog scan for table with delete files [iceberg]

pvary commented on PR #10935: URL: https://github.com/apache/iceberg/pull/10935#issuecomment-2367649336 This looks good to me. @dramaticlly: Any more comments, before we try to involve the guys who are more experienced with the core/spark parts? -- This is an automated message from the

Re: [PR] Support changelog scan for table with delete files [iceberg]

pvary commented on code in PR #10935: URL: https://github.com/apache/iceberg/pull/10935#discussion_r1771032932 ## core/src/test/java/org/apache/iceberg/TestBaseIncrementalChangelogScan.java: ## @@ -132,6 +131,175 @@ public void testFileDeletes() { assertThat(t1.existingDele

Re: [PR] Support changelog scan for table with delete files [iceberg]

pvary commented on code in PR #10935: URL: https://github.com/apache/iceberg/pull/10935#discussion_r1771031788 ## core/src/test/java/org/apache/iceberg/TestBaseIncrementalChangelogScan.java: ## @@ -132,6 +131,139 @@ public void testFileDeletes() { assertThat(t1.existingDele

Re: [PR] Core, Spark: Fix delete with filter on nested columns [iceberg]

blakewhatley82 commented on PR #7132: URL: https://github.com/apache/iceberg/pull/7132#issuecomment-2367597533 This issue is still around in spark 3.5 and would really be a big capability to have for data that is all in structured format -- This is an automated message from the Apache Gi

[PR] feat: expose remove_all in FileIO [iceberg-rust]

xxchan opened a new pull request, #643: URL: https://github.com/apache/iceberg-rust/pull/643 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-

Re: [PR] API: Add RemoveUnusedSpecs in Table [iceberg]

advancedxy commented on PR #10755: URL: https://github.com/apache/iceberg/pull/10755#issuecomment-2367497991 @amogh-jahagirdar Updated, It would be great if you can take another look at this. -- This is an automated message from the Apache Git Service. To respond to the message, please lo

Re: [PR] feat: Add NamespaceIdent.parent() [iceberg-rust]

Xuanwo commented on code in PR #641: URL: https://github.com/apache/iceberg-rust/pull/641#discussion_r1770894847 ## crates/iceberg/src/catalog/mod.rs: ## @@ -133,6 +133,16 @@ impl NamespaceIdent { pub fn inner(self) -> Vec { self.0 } + +/// Get the parent

Re: [PR] Migrate to arrow-* v53 [iceberg-rust]

Xuanwo merged PR #626: URL: https://github.com/apache/iceberg-rust/pull/626 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.a

Re: [I] Bump arrow to 53 [iceberg-rust]

Xuanwo closed issue #622: Bump arrow to 53 URL: https://github.com/apache/iceberg-rust/issues/622 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues

Re: [PR] chore(deps): Update arrow-select requirement from 52 to 53 [iceberg-rust]

dependabot[bot] commented on PR #632: URL: https://github.com/apache/iceberg-rust/pull/632#issuecomment-2367422738 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version,

Re: [PR] chore(deps): Update arrow-string requirement from 52 to 53 [iceberg-rust]

dependabot[bot] commented on PR #633: URL: https://github.com/apache/iceberg-rust/pull/633#issuecomment-2367422694 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version,

Re: [PR] chore(deps): Update arrow-select requirement from 52 to 53 [iceberg-rust]

Xuanwo closed pull request #632: chore(deps): Update arrow-select requirement from 52 to 53 URL: https://github.com/apache/iceberg-rust/pull/632 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

  1   2   >