[
https://issues.apache.org/jira/browse/LUCENE-8739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17468137#comment-17468137
]
Adrien Grand commented on LUCENE-8739:
--------------------------------------
I ran the same benchmark over the above PR with the dictionary mode.
||Codec ||Indexing time (ms) ||Disk usage (MB) || Retrieval time per 10k docs
(ms) ||
| BEST_SPEED | 35383 | 90.175 | 190.17524 |
| BEST_COMPRESSION (vanilla zlib) | 76671 | 58.682 | 1910.42106 |
| BEST_COMPRESSION (Cloudflare zlib) | 54791 | 58.601 | 1395.53593 |
| ZSTD (level=1) | 42433 | 70.527 | 240.04036 |
| ZSTD (level=3) | 53426 | 68.737 | 259.61897 |
| ZSTD (level=6) | 100697 | 66.283 | 251.91177 |
| ZSTD dict (level=1) | 50571 | 69.860 | 254.10496 |
| ZSTD dict (level=3) | 60580 | 68.690 | 266.72929 |
| ZSTD dict (level=6) | 128322 | 65.605 | 251.91177 |
Compression ratios are a bit disappointing, I wonder if this is because DEFLATE
outperforms ZSTD on this sort of data or because there is a bug in your
contribution?
> ZSTD Compressor support in Lucene
> ---------------------------------
>
> Key: LUCENE-8739
> URL: https://issues.apache.org/jira/browse/LUCENE-8739
> Project: Lucene - Core
> Issue Type: New Feature
> Components: core/codecs
> Reporter: Sean Torres
> Priority: Minor
> Labels: features
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> ZStandard has a great speed and compression ratio tradeoff.
> ZStandard is open source compression from Facebook.
> More about ZSTD
> [https://github.com/facebook/zstd]
> [https://code.facebook.com/posts/1658392934479273/smaller-and-faster-data-compression-with-zstandard/]
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]