[
https://issues.apache.org/jira/browse/HADOOP-19012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18029166#comment-18029166
]
Tsz-wo Sze edited comment on HADOOP-19012 at 10/14/25 8:31 PM:
---------------------------------------------------------------
szetszwo commented on PR #8011:
URL: https://github.com/apache/hadoop/pull/8011#issuecomment-3393054570
Benchmark results:
{code}
Start warming up with 2 iterations ...
Run CRC32 with 100000000 computations
galoisFieldMultiply: 1.896s (52742616.03 ops)
multiplyCrc32 : 0.898s (111358574.61 ops)
multiplyCrc32 is 111.14% faster
Run CRC32 with 100000000 computations
galoisFieldMultiply: 1.769s (56529112.49 ops)
multiplyCrc32 : 0.824s (121359223.30 ops)
multiplyCrc32 is 114.68% faster
Start benchmark with 10 iterations ...
0) Run CRC32 with 100000000 computations
galoisFieldMultiply: 1.771s (56465273.86 ops)
multiplyCrc32 : 0.816s (122549019.61 ops)
multiplyCrc32 is 117.03% faster
1) Run CRC32 with 100000000 computations
galoisFieldMultiply: 1.770s (56497175.14 ops)
multiplyCrc32 : 0.821s (121802679.66 ops)
multiplyCrc32 is 115.59% faster
2) Run CRC32 with 100000000 computations
galoisFieldMultiply: 1.775s (56338028.17 ops)
multiplyCrc32 : 0.809s (123609394.31 ops)
multiplyCrc32 is 119.41% faster
3) Run CRC32 with 100000000 computations
galoisFieldMultiply: 1.773s (56401579.24 ops)
multiplyCrc32 : 0.810s (123456790.12 ops)
multiplyCrc32 is 118.89% faster
4) Run CRC32 with 100000000 computations
galoisFieldMultiply: 1.772s (56433408.58 ops)
multiplyCrc32 : 0.820s (121951219.51 ops)
multiplyCrc32 is 116.10% faster
5) Run CRC32 with 100000000 computations
galoisFieldMultiply: 1.770s (56497175.14 ops)
multiplyCrc32 : 0.828s (120772946.86 ops)
multiplyCrc32 is 113.77% faster
6) Run CRC32 with 100000000 computations
galoisFieldMultiply: 1.772s (56433408.58 ops)
multiplyCrc32 : 0.823s (121506682.87 ops)
multiplyCrc32 is 115.31% faster
7) Run CRC32 with 100000000 computations
galoisFieldMultiply: 1.766s (56625141.56 ops)
multiplyCrc32 : 0.822s (121654501.22 ops)
multiplyCrc32 is 114.84% faster
8) Run CRC32 with 100000000 computations
galoisFieldMultiply: 1.780s (56179775.28 ops)
multiplyCrc32 : 0.824s (121359223.30 ops)
multiplyCrc32 is 116.02% faster
9) Run CRC32 with 100000000 computations
galoisFieldMultiply: 1.774s (56369785.79 ops)
multiplyCrc32 : 0.821s (121802679.66 ops)
multiplyCrc32 is 116.08% faster
Result) 10 x 100000000 computations:
galoisFieldMultiply: 17.723s (5642385.60 ops)
multiplyCrc32 : 8.194s (12204051.75 ops)
multiplyCrc32 is 116.29% faster
Process finished with exit code 0
{code}
was (Author: githubbot):
szetszwo commented on PR #8011:
URL: https://github.com/apache/hadoop/pull/8011#issuecomment-3393054570
Benchmark results:
```
Start warming up with 2 iterations ...
Run CRC32 with 100000000 computations
galoisFieldMultiply: 1.896s (52742616.03 ops)
multiplyCrc32 : 0.898s (111358574.61 ops)
multiplyCrc32 is 111.14% faster
Run CRC32 with 100000000 computations
galoisFieldMultiply: 1.769s (56529112.49 ops)
multiplyCrc32 : 0.824s (121359223.30 ops)
multiplyCrc32 is 114.68% faster
Start benchmark with 10 iterations ...
0) Run CRC32 with 100000000 computations
galoisFieldMultiply: 1.771s (56465273.86 ops)
multiplyCrc32 : 0.816s (122549019.61 ops)
multiplyCrc32 is 117.03% faster
1) Run CRC32 with 100000000 computations
galoisFieldMultiply: 1.770s (56497175.14 ops)
multiplyCrc32 : 0.821s (121802679.66 ops)
multiplyCrc32 is 115.59% faster
2) Run CRC32 with 100000000 computations
galoisFieldMultiply: 1.775s (56338028.17 ops)
multiplyCrc32 : 0.809s (123609394.31 ops)
multiplyCrc32 is 119.41% faster
3) Run CRC32 with 100000000 computations
galoisFieldMultiply: 1.773s (56401579.24 ops)
multiplyCrc32 : 0.810s (123456790.12 ops)
multiplyCrc32 is 118.89% faster
4) Run CRC32 with 100000000 computations
galoisFieldMultiply: 1.772s (56433408.58 ops)
multiplyCrc32 : 0.820s (121951219.51 ops)
multiplyCrc32 is 116.10% faster
5) Run CRC32 with 100000000 computations
galoisFieldMultiply: 1.770s (56497175.14 ops)
multiplyCrc32 : 0.828s (120772946.86 ops)
multiplyCrc32 is 113.77% faster
6) Run CRC32 with 100000000 computations
galoisFieldMultiply: 1.772s (56433408.58 ops)
multiplyCrc32 : 0.823s (121506682.87 ops)
multiplyCrc32 is 115.31% faster
7) Run CRC32 with 100000000 computations
galoisFieldMultiply: 1.766s (56625141.56 ops)
multiplyCrc32 : 0.822s (121654501.22 ops)
multiplyCrc32 is 114.84% faster
8) Run CRC32 with 100000000 computations
galoisFieldMultiply: 1.780s (56179775.28 ops)
multiplyCrc32 : 0.824s (121359223.30 ops)
multiplyCrc32 is 116.02% faster
9) Run CRC32 with 100000000 computations
galoisFieldMultiply: 1.774s (56369785.79 ops)
multiplyCrc32 : 0.821s (121802679.66 ops)
multiplyCrc32 is 116.08% faster
Result) 10 x 100000000 computations:
galoisFieldMultiply: 17.723s (5642385.60 ops)
multiplyCrc32 : 8.194s (12204051.75 ops)
multiplyCrc32 is 116.29% faster
Process finished with exit code 0
```
> Use CRC tables to speed up galoisFieldMultiply in CrcUtil
> ---------------------------------------------------------
>
> Key: HADOOP-19012
> URL: https://issues.apache.org/jira/browse/HADOOP-19012
> Project: Hadoop Common
> Issue Type: Improvement
> Components: util
> Reporter: Tsz-wo Sze
> Assignee: Tsz-wo Sze
> Priority: Major
> Labels: pull-request-available
> Fix For: 3.5.0
>
>
> CrcUtil.galoisFieldMultiply(p, q, m) supports multiplying two polynomials p,
> q modulo any modulus polynomial m over GF(2). Since the method is used for
> CRC calculations, the modulus polynomial m is restricted to either the
> GZIP_POLYNOMIAL or the CASTAGNOLI_POLYNOMIAL. We may use CRC tables in
> PureJavaCrc32/PureJavaCrc32C to speed up the computation.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
