Re: [PR] Spark 3.5: Parallelize reading files in add_files procedure [iceberg]

manuzhang commented on code in PR #9274: URL: https://github.com/apache/iceberg/pull/9274#discussion_r1437478098 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/SparkTableUtil.java: ## @@ -374,14 +376,16 @@ private static Iterator buildManifest( * @param partition

Re: [PR] Docs: Note CREATE TABLE LIKE is not supported in Spark DDL [iceberg]

manuzhang commented on PR #9358: URL: https://github.com/apache/iceberg/pull/9358#issuecomment-1870950782 @ajantha-bhat @nastra please help review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[I] Partitioned table folder creation behaviour [iceberg]

Il-Pela opened a new issue, #9388: URL: https://github.com/apache/iceberg/issues/9388 ### Query engine Spark ### Question Hi all, I'm here to ask a question about partition folder creation behaviour while executing the following Python code `df.writeTo('db.table'

Re: [PR] Spark 3.5: Parallelize reading files in add_files procedure [iceberg]

amogh-jahagirdar commented on code in PR #9274: URL: https://github.com/apache/iceberg/pull/9274#discussion_r1437745561 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/SparkTableUtil.java: ## @@ -374,14 +376,16 @@ private static Iterator buildManifest( * @param pa

Re: [PR] Spark 3.5: Parallelize reading files in add_files procedure [iceberg]

amogh-jahagirdar merged PR #9274: URL: https://github.com/apache/iceberg/pull/9274 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [PR] AWS: Add S3 Access Grants Integration [iceberg]

jackye1995 commented on code in PR #9385: URL: https://github.com/apache/iceberg/pull/9385#discussion_r1437795543 ## aws/src/main/java/org/apache/iceberg/aws/s3/S3FileIOProperties.java: ## @@ -50,6 +51,23 @@ public class S3FileIOProperties implements Serializable { */ pub

Re: [PR] AWS: Add S3 Access Grants Integration [iceberg]

jackye1995 commented on code in PR #9385: URL: https://github.com/apache/iceberg/pull/9385#discussion_r1437796852 ## aws/src/main/java/org/apache/iceberg/aws/s3/S3FileIOProperties.java: ## @@ -749,4 +795,23 @@ public void applyEndpointConfigurations(T builder) { builder

[PR] preserve field order in `Schema.select` [iceberg-python]

felixscherz opened a new pull request, #242: URL: https://github.com/apache/iceberg-python/pull/242 Closes #26 Hi 👋 , this PR intends to preserve the order of fields passed in as arguments to `Schema.select` in the output. It is implemented as an additional step after pruning

Re: [PR] Core: Add ManifestWrite benchmark [iceberg]

Fokko commented on code in PR #8637: URL: https://github.com/apache/iceberg/pull/8637#discussion_r1437872743 ## core/src/jmh/java/org/apache/iceberg/ManifestWriteBenchmark.java: ## @@ -0,0 +1,153 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more c

Re: [PR] Core: Add ManifestWrite benchmark [iceberg]

Fokko commented on code in PR #8637: URL: https://github.com/apache/iceberg/pull/8637#discussion_r1437873021 ## core/src/jmh/java/org/apache/iceberg/ManifestWriteBenchmark.java: ## @@ -0,0 +1,153 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more c

Re: [PR] Core: Add ManifestWrite benchmark [iceberg]

Fokko commented on code in PR #8637: URL: https://github.com/apache/iceberg/pull/8637#discussion_r1437873121 ## core/src/jmh/java/org/apache/iceberg/ManifestWriteBenchmark.java: ## @@ -0,0 +1,153 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more c

Re: [PR] Core: Add ManifestWrite benchmark [iceberg]

Fokko commented on code in PR #8637: URL: https://github.com/apache/iceberg/pull/8637#discussion_r1437873121 ## core/src/jmh/java/org/apache/iceberg/ManifestWriteBenchmark.java: ## @@ -0,0 +1,153 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more c

Re: [PR] AWS: Add S3 Access Grants Integration [iceberg]

adnanhemani commented on code in PR #9385: URL: https://github.com/apache/iceberg/pull/9385#discussion_r1437879000 ## aws/src/main/java/org/apache/iceberg/aws/s3/S3FileIOProperties.java: ## @@ -50,6 +51,23 @@ public class S3FileIOProperties implements Serializable { */ pu

Re: [I] Spark DataFrame write fails if input dataframe has columns in different order than iceberg schema [iceberg]

amitmittal5 commented on issue #741: URL: https://github.com/apache/iceberg/issues/741#issuecomment-1871497269 Hello, is this issue resolved? I am still getting this issue in iceberg 1.4.2 while trying to write in iceberg format to ADLS using spark-streaming. -- This is an automated m

[PR] Build: Bump coverage from 7.3.4 to 7.4.0 [iceberg-python]

dependabot[bot] opened a new pull request, #243: URL: https://github.com/apache/iceberg-python/pull/243 Bumps [coverage](https://github.com/nedbat/coveragepy) from 7.3.4 to 7.4.0. Changelog Sourced from https://github.com/nedbat/coveragepy/blob/master/CHANGES.rst";>coverage's chang

[PR] Build: Bump sqlalchemy from 2.0.23 to 2.0.24 [iceberg-python]

dependabot[bot] opened a new pull request, #244: URL: https://github.com/apache/iceberg-python/pull/244 Bumps [sqlalchemy](https://github.com/sqlalchemy/sqlalchemy) from 2.0.23 to 2.0.24. Release notes Sourced from https://github.com/sqlalchemy/sqlalchemy/releases";>sqlalchemy's r

Re: [I] Migrating iceberg table on object storage [iceberg]

github-actions[bot] commented on issue #6964: URL: https://github.com/apache/iceberg/issues/6964#issuecomment-1871636636 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] Explore to use SparkUtil to return key metadata for add-files [iceberg]

github-actions[bot] closed issue #6961: Explore to use SparkUtil to return key metadata for add-files URL: https://github.com/apache/iceberg/issues/6961 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [I] Explore to use SparkUtil to return key metadata for add-files [iceberg]

github-actions[bot] commented on issue #6961: URL: https://github.com/apache/iceberg/issues/6961#issuecomment-1871636654 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [PR] AWS: Add S3 Access Grants Integration [iceberg]

adnanhemani commented on code in PR #9385: URL: https://github.com/apache/iceberg/pull/9385#discussion_r1437959281 ## aws/src/main/java/org/apache/iceberg/aws/s3/S3FileIOProperties.java: ## @@ -749,4 +795,23 @@ public void applyEndpointConfigurations(T builder) { builde

Re: [PR] AWS: Add S3 Access Grants Integration [iceberg]

adnanhemani commented on code in PR #9385: URL: https://github.com/apache/iceberg/pull/9385#discussion_r1437959281 ## aws/src/main/java/org/apache/iceberg/aws/s3/S3FileIOProperties.java: ## @@ -749,4 +795,23 @@ public void applyEndpointConfigurations(T builder) { builde

Re: [PR] chore: Upgrade toolchain to 1.75.0 [iceberg-rust]

liurenjie1024 commented on PR #140: URL: https://github.com/apache/iceberg-rust/pull/140#issuecomment-1871785701 cc @Xuanwo @Fokko @ZENOTME PTAL -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[PR] Core, Spark: Correct the delete record count for PartitionTable [iceberg]

ConeyLiu opened a new pull request, #9389: URL: https://github.com/apache/iceberg/pull/9389 The record counts for delete-file are calculated wrong in ParitionTable. For example, we got the following strange results. https://github.com/apache/iceberg/assets/12733256/a57f150c-0567-4439-8e57

Re: [PR] Core, Spark: Correct the delete record count for PartitionTable [iceberg]

ConeyLiu commented on PR #9389: URL: https://github.com/apache/iceberg/pull/9389#issuecomment-1871801687 Hi @nastra @Fokko @ajantha-bhat @dramaticlly could you help to review this when you are free? Thanks a lot. -- This is an automated message from the Apache Git Service. To respond to t