gf2121 commented on PR #12775:
URL: https://github.com/apache/lucene/pull/12775#issuecomment-1805262547
Close this in favor of https://github.com/apache/lucene/pull/12784
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use
gf2121 closed pull request #12775: Speed up BytesRefHash#sort
URL: https://github.com/apache/lucene/pull/12775
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e
gf2121 commented on PR #12775:
URL: https://github.com/apache/lucene/pull/12775#issuecomment-1802480151
I came up with https://github.com/apache/lucene/pull/12784 as another idea
to speed up `BytesRefHash#sort`, which has been shown to have performance
improvements running on Intel chips.
gf2121 commented on PR #12775:
URL: https://github.com/apache/lucene/pull/12775#issuecomment-1801756914
> it's fine if this yields a speedup on some platforms but not other
platforms
+1
> it doesn't seem to slow down flushing either?
Yes!
--
This is an automated messa
jpountz commented on PR #12775:
URL: https://github.com/apache/lucene/pull/12775#issuecomment-1801748121
FWIW I think it's fine if this yields a speedup on some platforms but not
other platforms. From your results, it doesn't seem to slow down flushing
either?
--
This is an automated mes
dweiss commented on PR #12775:
URL: https://github.com/apache/lucene/pull/12775#issuecomment-1801236300
Maybe Apple chips are tuned for sorting (they need to sort out what to do
with all these income bills, after all)? :)
And seriously - thank you for checking on different hardware. T
gf2121 commented on PR #12775:
URL: https://github.com/apache/lucene/pull/12775#issuecomment-1801199102
I checked linux x86, no obvious speedup too. Maybe the stable style somewhat
helped the arm CPU cache
```
use stable sort: false, sort 5169965 terms, took: 4900ms
use stable sort:
gf2121 commented on PR #12775:
URL: https://github.com/apache/lucene/pull/12775#issuecomment-1801136851
Something odd: I accidentally run the benchmark on another mac with `intel
chip` and the result is disappointing (no obvious improvements)
```
use stable sort: false, sort 5169965 te
gf2121 commented on PR #12775:
URL: https://github.com/apache/lucene/pull/12775#issuecomment-1801128536
Something odd: I accidentally run the benchmark on another mac with `intel
chip` and the result is disappointing (no improvements)
```
use stable sort: false, sort 5169965 terms, too
mikemccand commented on PR #12775:
URL: https://github.com/apache/lucene/pull/12775#issuecomment-1798603970
This is incredible speedup :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the speci
dweiss commented on PR #12775:
URL: https://github.com/apache/lucene/pull/12775#issuecomment-1798550955
Wow. Nice improvement!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment
gf2121 commented on PR #12775:
URL: https://github.com/apache/lucene/pull/12775#issuecomment-1798365624
For realistic data I tried to index `wikimedium10m` with ramBuffer = 1024m
and accumulating the took of all invoking of `BytesRefHash#sort`. Result shows
the took sum decreased from 27161
gf2121 commented on PR #12775:
URL: https://github.com/apache/lucene/pull/12775#issuecomment-1797986285
Update:
* Make `MergeSorter` take advantage of `setPivot` to reduce decode.
* Fix the problem that `StringSorter#getFallBackSorter` did not use the
passed in comparator which can voi
13 matches
Mail list logo
