This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new bc7a5c0a47b3 [SPARK-48958][BUILD] Upgrade `zstd-jni` to 1.5.6-4
bc7a5c0a47b3 is described below
commit bc7a5c0a47b32d7c81a4dab2a8a15e6965220f0e
Author: panbingkun <[email protected]>
AuthorDate: Mon Jul 22 09:01:12 2024 -0700
[SPARK-48958][BUILD] Upgrade `zstd-jni` to 1.5.6-4
### What changes were proposed in this pull request?
The pr aims to upgrade `zstd-jni` from `1.5.6-3` to `1.5.6-4`.
### Why are the changes needed?
1.v1.5.6-3 VS v1.5.6-4
https://github.com/luben/zstd-jni/compare/v1.5.6-3...v1.5.6-4
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Pass GA.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #47432 from panbingkun/SPARK-48958.
Authored-by: panbingkun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
.../ZStandardBenchmark-jdk21-results.txt | 56 +++++++++++-----------
core/benchmarks/ZStandardBenchmark-results.txt | 56 +++++++++++-----------
dev/deps/spark-deps-hadoop-3-hive-2.3 | 2 +-
pom.xml | 2 +-
4 files changed, 58 insertions(+), 58 deletions(-)
diff --git a/core/benchmarks/ZStandardBenchmark-jdk21-results.txt
b/core/benchmarks/ZStandardBenchmark-jdk21-results.txt
index 4c986ca0e4e0..66c89ac139dc 100644
--- a/core/benchmarks/ZStandardBenchmark-jdk21-results.txt
+++ b/core/benchmarks/ZStandardBenchmark-jdk21-results.txt
@@ -2,48 +2,48 @@
Benchmark ZStandardCompressionCodec
================================================================================================
-OpenJDK 64-Bit Server VM 21.0.3+9-LTS on Linux 6.5.0-1018-azure
+OpenJDK 64-Bit Server VM 21.0.4+7-LTS on Linux 6.5.0-1023-azure
AMD EPYC 7763 64-Core Processor
Benchmark ZStandardCompressionCodec: Best Time(ms) Avg
Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
--------------------------------------------------------------------------------------------------------------------------------------
-Compression 10000 times at level 1 without buffer pool 649
748 156 0.0 64921.9 1.0X
-Compression 10000 times at level 2 without buffer pool 689
689 0 0.0 68927.1 0.9X
-Compression 10000 times at level 3 without buffer pool 782
782 0 0.0 78180.6 0.8X
-Compression 10000 times at level 1 with buffer pool 580
582 2 0.0 57976.0 1.1X
-Compression 10000 times at level 2 with buffer pool 614
618 4 0.0 61395.3 1.1X
-Compression 10000 times at level 3 with buffer pool 725
734 11 0.0 72535.5 0.9X
+Compression 10000 times at level 1 without buffer pool 654
675 20 0.0 65380.3 1.0X
+Compression 10000 times at level 2 without buffer pool 714
715 1 0.0 71445.4 0.9X
+Compression 10000 times at level 3 without buffer pool 811
814 3 0.0 81142.7 0.8X
+Compression 10000 times at level 1 with buffer pool 605
606 1 0.0 60518.7 1.1X
+Compression 10000 times at level 2 with buffer pool 634
637 2 0.0 63441.4 1.0X
+Compression 10000 times at level 3 with buffer pool 743
743 0 0.0 74258.7 0.9X
-OpenJDK 64-Bit Server VM 21.0.3+9-LTS on Linux 6.5.0-1018-azure
+OpenJDK 64-Bit Server VM 21.0.4+7-LTS on Linux 6.5.0-1023-azure
AMD EPYC 7763 64-Core Processor
Benchmark ZStandardCompressionCodec: Best Time(ms)
Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------------------------
-Decompression 10000 times from level 1 without buffer pool 831
832 1 0.0 83114.9 1.0X
-Decompression 10000 times from level 2 without buffer pool 834
835 1 0.0 83372.7 1.0X
-Decompression 10000 times from level 3 without buffer pool 831
832 1 0.0 83092.3 1.0X
-Decompression 10000 times from level 1 with buffer pool 759
760 1 0.0 75870.2 1.1X
-Decompression 10000 times from level 2 with buffer pool 759
760 1 0.0 75877.3 1.1X
-Decompression 10000 times from level 3 with buffer pool 759
759 0 0.0 75874.5 1.1X
+Decompression 10000 times from level 1 without buffer pool 824
826 4 0.0 82358.5 1.0X
+Decompression 10000 times from level 2 without buffer pool 824
824 0 0.0 82394.3 1.0X
+Decompression 10000 times from level 3 without buffer pool 823
824 1 0.0 82343.3 1.0X
+Decompression 10000 times from level 1 with buffer pool 748
749 1 0.0 74792.0 1.1X
+Decompression 10000 times from level 2 with buffer pool 748
749 2 0.0 74773.6 1.1X
+Decompression 10000 times from level 3 with buffer pool 749
749 1 0.0 74868.3 1.1X
-OpenJDK 64-Bit Server VM 21.0.3+9-LTS on Linux 6.5.0-1018-azure
+OpenJDK 64-Bit Server VM 21.0.4+7-LTS on Linux 6.5.0-1023-azure
AMD EPYC 7763 64-Core Processor
Parallel Compression at level 3: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Parallel Compression with 0 workers 48 50
1 0.0 376632.9 1.0X
-Parallel Compression with 1 workers 35 37
2 0.0 272066.6 1.4X
-Parallel Compression with 2 workers 34 38
2 0.0 263055.3 1.4X
-Parallel Compression with 4 workers 37 39
2 0.0 286835.7 1.3X
-Parallel Compression with 8 workers 38 40
1 0.0 299961.3 1.3X
-Parallel Compression with 16 workers 43 45
1 0.0 335272.5 1.1X
+Parallel Compression with 0 workers 48 49
1 0.0 377356.2 1.0X
+Parallel Compression with 1 workers 36 37
2 0.0 279079.1 1.4X
+Parallel Compression with 2 workers 36 38
1 0.0 283760.8 1.3X
+Parallel Compression with 4 workers 38 40
2 0.0 298581.6 1.3X
+Parallel Compression with 8 workers 41 43
1 0.0 320669.0 1.2X
+Parallel Compression with 16 workers 46 48
1 0.0 356997.0 1.1X
-OpenJDK 64-Bit Server VM 21.0.3+9-LTS on Linux 6.5.0-1018-azure
+OpenJDK 64-Bit Server VM 21.0.4+7-LTS on Linux 6.5.0-1023-azure
AMD EPYC 7763 64-Core Processor
Parallel Compression at level 9: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Parallel Compression with 0 workers 157 158
1 0.0 1224138.2 1.0X
-Parallel Compression with 1 workers 187 188
1 0.0 1463264.4 0.8X
-Parallel Compression with 2 workers 111 115
6 0.0 863722.6 1.4X
-Parallel Compression with 4 workers 105 109
2 0.0 822422.6 1.5X
-Parallel Compression with 8 workers 110 114
2 0.0 862852.1 1.4X
-Parallel Compression with 16 workers 111 115
2 0.0 870311.3 1.4X
+Parallel Compression with 0 workers 159 161
1 0.0 1242020.8 1.0X
+Parallel Compression with 1 workers 187 188
1 0.0 1463507.3 0.8X
+Parallel Compression with 2 workers 114 118
5 0.0 888481.5 1.4X
+Parallel Compression with 4 workers 107 110
2 0.0 836926.1 1.5X
+Parallel Compression with 8 workers 110 115
3 0.0 856838.0 1.4X
+Parallel Compression with 16 workers 112 115
2 0.0 874554.2 1.4X
diff --git a/core/benchmarks/ZStandardBenchmark-results.txt
b/core/benchmarks/ZStandardBenchmark-results.txt
index 5569f27bb016..7093636c4bc2 100644
--- a/core/benchmarks/ZStandardBenchmark-results.txt
+++ b/core/benchmarks/ZStandardBenchmark-results.txt
@@ -2,48 +2,48 @@
Benchmark ZStandardCompressionCodec
================================================================================================
-OpenJDK 64-Bit Server VM 17.0.11+9-LTS on Linux 6.5.0-1018-azure
+OpenJDK 64-Bit Server VM 17.0.12+7-LTS on Linux 6.5.0-1023-azure
AMD EPYC 7763 64-Core Processor
Benchmark ZStandardCompressionCodec: Best Time(ms) Avg
Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
--------------------------------------------------------------------------------------------------------------------------------------
-Compression 10000 times at level 1 without buffer pool 656
660 3 0.0 65632.5 1.0X
-Compression 10000 times at level 2 without buffer pool 695
696 1 0.0 69509.7 0.9X
-Compression 10000 times at level 3 without buffer pool 803
807 7 0.0 80258.4 0.8X
-Compression 10000 times at level 1 with buffer pool 584
586 2 0.0 58381.4 1.1X
-Compression 10000 times at level 2 with buffer pool 615
616 1 0.0 61463.0 1.1X
-Compression 10000 times at level 3 with buffer pool 743
743 0 0.0 74310.9 0.9X
+Compression 10000 times at level 1 without buffer pool 259
260 1 0.0 25854.0 1.0X
+Compression 10000 times at level 2 without buffer pool 678
678 1 0.0 67756.2 0.4X
+Compression 10000 times at level 3 without buffer pool 775
775 1 0.0 77452.6 0.3X
+Compression 10000 times at level 1 with buffer pool 572
573 1 0.0 57218.3 0.5X
+Compression 10000 times at level 2 with buffer pool 600
602 1 0.0 60041.6 0.4X
+Compression 10000 times at level 3 with buffer pool 710
711 2 0.0 70967.2 0.4X
-OpenJDK 64-Bit Server VM 17.0.11+9-LTS on Linux 6.5.0-1018-azure
+OpenJDK 64-Bit Server VM 17.0.12+7-LTS on Linux 6.5.0-1023-azure
AMD EPYC 7763 64-Core Processor
Benchmark ZStandardCompressionCodec: Best Time(ms)
Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------------------------
-Decompression 10000 times from level 1 without buffer pool 620
621 1 0.0 61972.9 1.0X
-Decompression 10000 times from level 2 without buffer pool 622
622 1 0.0 62168.8 1.0X
-Decompression 10000 times from level 3 without buffer pool 621
622 1 0.0 62130.0 1.0X
-Decompression 10000 times from level 1 with buffer pool 549
550 0 0.0 54939.0 1.1X
-Decompression 10000 times from level 2 with buffer pool 550
550 0 0.0 54963.5 1.1X
-Decompression 10000 times from level 3 with buffer pool 549
550 1 0.0 54927.7 1.1X
+Decompression 10000 times from level 1 without buffer pool 588
588 1 0.0 58764.1 1.0X
+Decompression 10000 times from level 2 without buffer pool 589
595 10 0.0 58919.5 1.0X
+Decompression 10000 times from level 3 without buffer pool 591
597 12 0.0 59059.2 1.0X
+Decompression 10000 times from level 1 with buffer pool 520
521 1 0.0 52039.6 1.1X
+Decompression 10000 times from level 2 with buffer pool 521
522 1 0.0 52116.4 1.1X
+Decompression 10000 times from level 3 with buffer pool 521
521 0 0.0 52100.6 1.1X
-OpenJDK 64-Bit Server VM 17.0.11+9-LTS on Linux 6.5.0-1018-azure
+OpenJDK 64-Bit Server VM 17.0.12+7-LTS on Linux 6.5.0-1023-azure
AMD EPYC 7763 64-Core Processor
Parallel Compression at level 3: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Parallel Compression with 0 workers 47 48
1 0.0 365666.1 1.0X
-Parallel Compression with 1 workers 34 36
3 0.0 268562.3 1.4X
-Parallel Compression with 2 workers 32 35
2 0.0 251265.1 1.5X
-Parallel Compression with 4 workers 35 38
1 0.0 273574.1 1.3X
-Parallel Compression with 8 workers 37 40
1 0.0 288217.8 1.3X
-Parallel Compression with 16 workers 42 44
1 0.0 330318.7 1.1X
+Parallel Compression with 0 workers 46 48
1 0.0 362926.1 1.0X
+Parallel Compression with 1 workers 34 36
4 0.0 265302.5 1.4X
+Parallel Compression with 2 workers 32 36
1 0.0 252423.6 1.4X
+Parallel Compression with 4 workers 36 38
2 0.0 282974.7 1.3X
+Parallel Compression with 8 workers 38 40
1 0.0 298633.6 1.2X
+Parallel Compression with 16 workers 42 44
1 0.0 329766.4 1.1X
-OpenJDK 64-Bit Server VM 17.0.11+9-LTS on Linux 6.5.0-1018-azure
+OpenJDK 64-Bit Server VM 17.0.12+7-LTS on Linux 6.5.0-1023-azure
AMD EPYC 7763 64-Core Processor
Parallel Compression at level 9: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Parallel Compression with 0 workers 155 157
2 0.0 1214057.2 1.0X
-Parallel Compression with 1 workers 192 193
2 0.0 1499524.2 0.8X
-Parallel Compression with 2 workers 112 119
9 0.0 871848.8 1.4X
-Parallel Compression with 4 workers 106 109
2 0.0 830699.8 1.5X
-Parallel Compression with 8 workers 111 114
2 0.0 870700.3 1.4X
-Parallel Compression with 16 workers 112 114
2 0.0 873315.6 1.4X
+Parallel Compression with 0 workers 155 156
1 0.0 1212983.5 1.0X
+Parallel Compression with 1 workers 191 195
7 0.0 1492776.4 0.8X
+Parallel Compression with 2 workers 110 118
7 0.0 861496.4 1.4X
+Parallel Compression with 4 workers 105 108
2 0.0 821249.1 1.5X
+Parallel Compression with 8 workers 110 114
3 0.0 856234.4 1.4X
+Parallel Compression with 16 workers 110 113
2 0.0 855949.6 1.4X
diff --git a/dev/deps/spark-deps-hadoop-3-hive-2.3
b/dev/deps/spark-deps-hadoop-3-hive-2.3
index 0d786bf49b43..9d00f178ef9b 100644
--- a/dev/deps/spark-deps-hadoop-3-hive-2.3
+++ b/dev/deps/spark-deps-hadoop-3-hive-2.3
@@ -275,4 +275,4 @@ xz/1.9//xz-1.9.jar
zjsonpatch/0.3.0//zjsonpatch-0.3.0.jar
zookeeper-jute/3.9.2//zookeeper-jute-3.9.2.jar
zookeeper/3.9.2//zookeeper-3.9.2.jar
-zstd-jni/1.5.6-3//zstd-jni-1.5.6-3.jar
+zstd-jni/1.5.6-4//zstd-jni-1.5.6-4.jar
diff --git a/pom.xml b/pom.xml
index 88a1fff8e71c..c179d547cbcb 100644
--- a/pom.xml
+++ b/pom.xml
@@ -801,7 +801,7 @@
<dependency>
<groupId>com.github.luben</groupId>
<artifactId>zstd-jni</artifactId>
- <version>1.5.6-3</version>
+ <version>1.5.6-4</version>
</dependency>
<dependency>
<groupId>com.clearspring.analytics</groupId>
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
