Tim Underwood created SOLR-16945:
------------------------------------
Summary: Add multi-threaded warming support to CaffeineCache and
QuerySenderListener
Key: SOLR-16945
URL: https://issues.apache.org/jira/browse/SOLR-16945
Project: Solr
Issue Type: Improvement
Security Level: Public (Default Security Level. Issues are Public)
Components: Server
Reporter: Tim Underwood
Cache warming (via CaffeineCache) and newSearcher/firstSearcher listeners (via
QuerySenderListener) currently run queries single-threaded. It would be nice to
be able to specify running these with multiple threads to speed up the warming
process when we know there is extra CPU capacity.
For our use case we are trying to minimize the amount of time it takes for
newly indexed changes to be visible to users while also minimizing any response
time spikes caused by opening up new searchers and repopulating caches. We
currently make use of cache warming and listeners to warm new searchers,
however, this process is currently too slow when running single-threaded. We
run our servers ~50% loaded and would like to be able to make use of extra CPU
capacity to speed up the warming.
I propose adding an optional "autowarmThreads" attribute when specifying the
caches. For example:
{code:java}
<filterCache size="512"
initialSize="512"
autowarmCount="512"
autowarmThreads="4"/>{code}
The QuerySenderListener configuration would use a <threads> element (I didn't
see an obvious say to use an attribute there). For example:
{code:java}
<listener event="firstSearcher" class="solr.QuerySenderListener">
<threads>4</threads>
<arr name="queries">
<!--
<lst>
<str name="q">static firstSearcher warming in solrconfig.xml</str>
</lst>
-->
</arr>
</listener>{code}
In both cases the number of threads is optional. The default behavior is the
existing single-threaded approach. Using multiple threads is opt-in.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
