Re: [PR] Spark 3.5: Parallelize reading files in add_files procedure [iceberg]

manuzhang commented on PR #9274: URL: https://github.com/apache/iceberg/pull/9274#issuecomment-2312956253 I will submit a PR to back-port to Spark 3.3 and 3.4 soon. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

Re: [PR] Spark 3.5: Parallelize reading files in add_files procedure [iceberg]

lurnagao-dahua commented on PR #9274: URL: https://github.com/apache/iceberg/pull/9274#issuecomment-2311648574 Hi,May I ask if there are any plans to port to Spark 3.3? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

Re: [PR] Spark 3.5: Parallelize reading files in add_files procedure [iceberg]

amogh-jahagirdar merged PR #9274: URL: https://github.com/apache/iceberg/pull/9274 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [PR] Spark 3.5: Parallelize reading files in add_files procedure [iceberg]

amogh-jahagirdar commented on code in PR #9274: URL: https://github.com/apache/iceberg/pull/9274#discussion_r1437745561 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/SparkTableUtil.java: ## @@ -374,14 +376,16 @@ private static Iterator buildManifest( * @param pa

Re: [PR] Spark 3.5: Parallelize reading files in add_files procedure [iceberg]

manuzhang commented on code in PR #9274: URL: https://github.com/apache/iceberg/pull/9274#discussion_r1437478098 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/SparkTableUtil.java: ## @@ -374,14 +376,16 @@ private static Iterator buildManifest( * @param partition

Re: [PR] Spark 3.5: Parallelize reading files in add_files procedure [iceberg]

amogh-jahagirdar commented on code in PR #9274: URL: https://github.com/apache/iceberg/pull/9274#discussion_r1437431367 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/SparkTableUtil.java: ## @@ -374,14 +376,16 @@ private static Iterator buildManifest( * @param pa

Re: [PR] Spark 3.5: Parallelize reading files in add_files procedure [iceberg]

amogh-jahagirdar commented on code in PR #9274: URL: https://github.com/apache/iceberg/pull/9274#discussion_r1437431367 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/SparkTableUtil.java: ## @@ -374,14 +376,16 @@ private static Iterator buildManifest( * @param pa

Re: [PR] Spark 3.5: Parallelize reading files in add_files procedure [iceberg]

manuzhang commented on PR #9274: URL: https://github.com/apache/iceberg/pull/9274#issuecomment-1868726591 @amogh-jahagirdar updated. Thanks for review. Merry Christmas 🎄 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] Spark 3.5: Parallelize reading files in add_files procedure [iceberg]

manuzhang commented on PR #9274: URL: https://github.com/apache/iceberg/pull/9274#issuecomment-1862098862 @amogh-jahagirdar since there are other usages of "parallelism", I updated all references to be more specific "readFilesParallelism". -- This is an automated message from the Apache G