On 11/27/2018 10:32 AM, ~$alpha` wrote:
<http://lucene.472066.n3.nabble.com/file/t482390/1.png>
SOLR VERSION 6.0.0
As seen in the image, There is a spike which can be observed at every 2
hours.
i.e whenever delta import runs
1. Response time doubles
2. CPU load average doubles and if it runs to near to peal hour than it
goes even to 10 times.
Indexing puts a lot of load on a Solr server. It tends to cause
disruption to queries running on the same machine, especially if
available memory is low enough to make the OS disk cache ineffective.
If you're running in SolrCloud mode on version 6, every server does both
indexing and queries, and there's really no way to change that. Version
7 has some additional options, especially in the latest version, 7.5.0.
**My Cache Settings are**
1. <filterCache class="solr.FastLRUCache"
size="5000"
initialSize="512"
autowarmCount="128"/>
An autowarmCount of 128 means that every time you do a commit operation,
Solr is going to execute up to 128 filters (depending on how many are
actually in the cache) on the new index. If each one takes 100
milliseconds, that's nearly 13 seconds of straight filter queries being
executed on your index, which is probably going to impact performance.
If each filter takes longer than 100 milliseconds to execute, the amount
of time and system resources required to execute this warming could be
REALLY high. I notice you have the same autowarmCount on your
queryResultCache. The same thing goes for that cache -- those queries
will be re-executed on the new index every time you do a commit that
opens a new searcher.
Most of my queries are CPU centric as they involved lots of IN queries and
NOT IN Queries. Also, have if cond for scoring. Assuming my queries will
continue to be CPU centric.
What evidence do you have that the queries are CPU-centric? I'm looking
for something very specific.
Your mention of load average says to me that you're probably not running
on a Windows system. Which I think means you could gather the
screenshot described here:
https://wiki.apache.org/solr/SolrPerformanceProblems#Process_listing_on_POSIX_operating_systems
Can you gather that and make it available with a file sharing website?
It would be best if you could do it while the import is running and
performance goes down.
**Help**
What Am I doing wrong as my delta import causing so much high response?
Index size : 2GB
Servers: 4
Serving: 1Lac per hour
Also, Delta update results in 2lac record update out of 10lac as one of the
fields of solr (last login) changed frequently.
How long does it take to import those 200000 documents? What are your
autoCommit and autoSoftCommit settings? Do you know how long it is
taking to warm each of your Solr caches? You can get that information
in the admin UI, under Plugins/Stats.
Thanks,
Shawn
