Toke Eskildsen kirjoitti 5.9.2017 klo 13.49:
On Mon, 2017-09-04 at 11:03 -0400, Yonik Seeley wrote:
It's due to this (see comments in UnInvertedField):
I have read that. What I don't understand is the difference between 4.x
and 6.x. But as you say, Ere seems to be in the process of verifying
whether this is simply due to more segments in 6.x.
During my testing I never optimized the 4.x index, so unless it
maintains a minimal number of segments automatically, there's something
else too.
There's probably a number of ways we can speed this up somewhat:
- optimize how much memory is used to store the term index and use
the savings to store more than every 128th term
- store the terms contiguously in block(s)
I'm considering taking a shot at that. A fairly easy optimization would
be to replace the BytesRef[] indexedTermsArray with a BytesRefArray.
I'd be happy to try out any patches.. :)
--Ere
--
Ere Maijala
Kansalliskirjasto / The National Library of Finland
