Re: [PR] Bump Spark 3.5.4 RC1 [iceberg]

2024-12-11 Thread via GitHub
dongjoon-hyun commented on PR #11731: URL: https://github.com/apache/iceberg/pull/11731#issuecomment-2536419878 Thank you for checking Apache Iceberg side, @pan3793 . -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] Bump Spark 3.5.4 RC1 [iceberg]

2024-12-10 Thread via GitHub
pan3793 commented on PR #11731: URL: https://github.com/apache/iceberg/pull/11731#issuecomment-2533597688 FYI @LuciferYang @viirya @dongjoon-hyun @wForget CI is green now, and I think the current Spark `branch-3.5` is in good shape for Iceberg. -- This is an automated message from

Re: [PR] Bump Spark 3.5.4 RC1 [iceberg]

2024-12-10 Thread via GitHub
pan3793 commented on code in PR #11731: URL: https://github.com/apache/iceberg/pull/11731#discussion_r1879272199 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/data/vectorized/IcebergArrowColumnVector.java: ## @@ -59,6 +59,11 @@ public void close() { accessor.cl

Re: [PR] Bump Spark 3.5.4 RC1 [iceberg]

2024-12-10 Thread via GitHub
pan3793 commented on code in PR #11731: URL: https://github.com/apache/iceberg/pull/11731#discussion_r1879272199 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/data/vectorized/IcebergArrowColumnVector.java: ## @@ -59,6 +59,11 @@ public void close() { accessor.cl

Re: [PR] Bump Spark 3.5.4 RC1 [iceberg]

2024-12-10 Thread via GitHub
singhpk234 commented on code in PR #11731: URL: https://github.com/apache/iceberg/pull/11731#discussion_r1879265358 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/data/vectorized/IcebergArrowColumnVector.java: ## @@ -59,6 +59,11 @@ public void close() { accessor

Re: [PR] Bump Spark 3.5.4 RC1 [iceberg]

2024-12-10 Thread via GitHub
pan3793 commented on code in PR #11731: URL: https://github.com/apache/iceberg/pull/11731#discussion_r1879240394 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/data/vectorized/IcebergArrowColumnVector.java: ## @@ -59,6 +59,11 @@ public void close() { accessor.cl

Re: [PR] Bump Spark 3.5.4 RC1 [iceberg]

2024-12-10 Thread via GitHub
pan3793 commented on PR #11731: URL: https://github.com/apache/iceberg/pull/11731#issuecomment-2533468089 @viirya @LuciferYang thank you for the guidance, let me try -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] Bump Spark 3.5.4 RC1 [iceberg]

2024-12-10 Thread via GitHub
LuciferYang commented on PR #11731: URL: https://github.com/apache/iceberg/pull/11731#issuecomment-2532512457 > Please check for some discussion there [apache/spark#49131 (comment)](https://github.com/apache/spark/pull/49131#issuecomment-2532341673) Yes, I tested it locally and found

Re: [PR] Bump Spark 3.5.4 RC1 [iceberg]

2024-12-10 Thread via GitHub
viirya commented on PR #11731: URL: https://github.com/apache/iceberg/pull/11731#issuecomment-2532346946 Please check for some discussion there https://github.com/apache/spark/pull/49131#issuecomment-2532341673 -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] Bump Spark 3.5.4 RC1 [iceberg]

2024-12-10 Thread via GitHub
viirya commented on PR #11731: URL: https://github.com/apache/iceberg/pull/11731#issuecomment-2532312814 Does Iceberg implement ColumnVector interface which is similar to writable column vector or constant column vector? -- This is an automated message from the Apache Git Service. To resp

Re: [PR] Bump Spark 3.5.4 RC1 [iceberg]

2024-12-10 Thread via GitHub
pan3793 commented on PR #11731: URL: https://github.com/apache/iceberg/pull/11731#issuecomment-2532122346 update: the failure is related to [SPARK-50235](https://github.com/apache/spark/pull/48767), and the test passed on Spark 3.5.4 RC0 with reverting that patch -- This is an automated

Re: [PR] Bump Spark 3.5.4 RC1 [iceberg]

2024-12-09 Thread via GitHub
pan3793 commented on PR #11731: URL: https://github.com/apache/iceberg/pull/11731#issuecomment-2530152795 the failure seems related to arrow minimal reproduce command ``` ./gradlew -DsparkVersions=3.5 -DscalaVersion=2.12 -DhiveVersions= -DflinkVersions= -DkafkaVersions= \

Re: [PR] Bump Spark 3.5.4 RC1 [iceberg]

2024-12-09 Thread via GitHub
pan3793 commented on PR #11731: URL: https://github.com/apache/iceberg/pull/11731#issuecomment-2530114142 the upgrading causes a core dump in iceberg-spark-3.5's tests ``` # # A fatal error has been detected by the Java Runtime Environment: # # SIGSEGV (0xb) at pc=0x7f8529

[PR] Bump Spark 3.5.4 RC1 [iceberg]

2024-12-08 Thread via GitHub
pan3793 opened a new pull request, #11731: URL: https://github.com/apache/iceberg/pull/11731 https://lists.apache.org/thread/btdsos7b92962lh10hzswkfy3oj1v0fb -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL