Oh, I am sorry, I totally forgot to mention our solr version, it's 7.7.3.
-----Ursprüngliche Nachricht-----
Von: Sebastian Riemer [mailto:s.rie...@littera.eu]
Gesendet: Freitag, 24. Juli 2020 09:53
An: solr-user@lucene.apache.org
Betreff: solr suggester.rebuild takes forever and eventually runs out of memory
on production
Dear mailing list community,
we have troubles when starting the Suggester-Build on one of our production
servers.
1. We execute the required query with the suggest.build parameter
2. It seems solr is taking up the task to recreate the suggester index
(we see that the CPU rises significantly)
3. It takes forever to build (and seems to never finish!)
4. Sometimes the linux OOM killer strikes and usually picks the solr
process and kills it
5. During the rebuild calling the suggester results in "suggester not
built" exception
6. Restarting the solr-Service has no effect. It just continues the
rebuild
How long should it take for that task, given that our index currently holds
approximately 7,2 Mio documents in a parent/child structure?
Is it possible, to query the progress of the suggester.build task after it was
started?
How can we be sure, whether the suggester.build task is still running or
whether it is finished?
Which factors have the most significant impact on the duration of the rebuild
process, given that we use the config below? (Let me now, if you need
additional information) Can we speed up the process somehow?
Best regards,
Sebastian
Solrconfig.xml
<searchComponent name="suggest" class="solr.SuggestComponent">
<lst name="suggester">
<str
name="name">infixSuggester</str>
<str
name="lookupImpl">AnalyzingInfixLookupFactory</str>
<str
name="indexPath">infix_suggestions</str>
<!--<str
name="dictionaryImpl">DocumentDictionaryFactory</str>-->
<str
name="dictionaryImpl">DocumentDictionaryFactory</str>
<str name="field">SUGGEST</str>
<str
name="suggestAnalyzerFieldType">textSuggest</str>
<str
name="buildOnStartup">false</str>
<str
name="buildOnCommit">false</str>
</lst>
</searchComponent>
<requestHandler name="/suggest" class="solr.SearchHandler" startup="lazy">
<lst name="defaults">
<str name="suggest">true</str>
<str
name="suggest.dictionary">infixSuggester</str>
<str
name="suggest.onlyMorePopular">true</str>
<str
name="suggest.count">500</str>
<str
name="suggest.collate">true</str>
</lst>
<arr name="components">
<str>suggest</str>
</arr>
</requestHandler>
Mit freundlichen Grüßen
Sebastian Riemer, BSc
