[
https://issues.apache.org/jira/browse/LUCENE-9850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17314865#comment-17314865
]
Michael McCandless commented on LUCENE-9850:
--------------------------------------------
Impressive results! And I love the flame charts that now come builtin with
JMC/JFR in the JDK!
I would expect {{XTerm}} to show speedups since this is largely dominated by
decoding many postings blocks. But it is odd to see the {{XTermYSort}} tasks
negatively impacted: those tasks are just sorting by a {{DocValues}} field
instead of default text relevance (BM25).
Net/net I think giving the impressive improvement in compression, speedup in
raw decode of postings blocks, I think this is worth doing?
These results are {{wikimediumall}} right? If you re-run but with only the
tasks that regressed above, do they still show regression? I'm wondering if
hotspot compilation noise is contributing ...
> Explore PFOR for Doc ID delta encoding (instead of FOR)
> -------------------------------------------------------
>
> Key: LUCENE-9850
> URL: https://issues.apache.org/jira/browse/LUCENE-9850
> Project: Lucene - Core
> Issue Type: Task
> Components: core/codecs
> Affects Versions: main (9.0)
> Reporter: Greg Miller
> Priority: Minor
> Attachments: apply_exceptions.png, bulk_read_1.png, bulk_read_2.png,
> for.png, pfor.png
>
>
> It'd be interesting to explore using PFOR instead of FOR for doc ID encoding.
> Right now PFOR is used for positions, frequencies and payloads, but FOR is
> used for doc ID deltas. From a recent
> [conversation|http://mail-archives.apache.org/mod_mbox/lucene-dev/202103.mbox/%3CCAPsWd%2BOp7d_GxNosB5r%3DQMPA-v0SteHWjXUmG3gwQot4gkubWw%40mail.gmail.com%3E]
> on the dev mailing list, it sounds like this decision was made based on the
> optimization possible when expanding the deltas.
> I'd be interesting in measuring the index size reduction possible with
> switching to PFOR compared to the performance reduction we might see by no
> longer being able to apply the deltas in as optimal a way.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
