-------- Forwarded Message --------
Subject: Re: [SPAM] abnormal cpu usage (solr 7.3.1)
Date: Wed, 11 Dec 2019 17:41:05 +0100
From: Danilo Tomasoni <tomas...@cosbi.eu>
To: Erick Erickson <erickerick...@gmail.com>
Thank you!.
any ideas for the cpu spikes?
On 11/12/19 17:28, Erick Erickson wrote:
This is pretty useful:
http://blog.mikemccandless.com/2011/02/visualizing-lucenes-segment-merges.html
TMP is the third visualization.
Mike is probably the best resource for all the nitty-gritty of merge
policies…
Best,
Erick
On Dec 11, 2019, at 11:11 AM, Danilo Tomasoni <tomas...@cosbi.eu> wrote:
On 11/12/19 16:53, Erick Erickson wrote:
Why are you using LogByteSIzeMergePolicy? The current default is
TieredMergePolicy, unless you have a specific reason, I’d just take
explicit merge policies out of your solrconfig and use the defaults.
Just because I found an article that explains nicely how it works
while I've not found such article for the TieredMergePolicy and I
like to know what's happening under the hood.
That aside, I doubt it’s related. You could well see these spikes
due to background merging, but that should have been present before
as well.
Every 3-5 seconds is interesting, does that coincide with commits
and/or opening searchers? If opening searchers, quite possibly
autowarming if you have that configured.
The instance isn't doing anything, nor searches nor commits, no user
interaction at all, except for web interface browsing to look for
thread usageand such.
In the log I've not found anything that suggests opening a searcher
or any other operation (browsing web interface excluded)
Danilo
On Dec 11, 2019, at 9:26 AM, Danilo Tomasoni <tomas...@cosbi.eu> wrote:
I'm sorry I forgot the pictures.
On 11/12/19 15:20, Danilo Tomasoni wrote:
Hello all,
we have a solr instance with around 41MLN documents.
Recently we stopped our forcemerge policy that ensured only 1
segmentwas present at query time, because we read here in the ML
and elsewhere that this is considered bad practice and it's also
very time consuming.
So we implemented
<mergePolicyFactory
class="org.apache.lucene.index.LogByteSizeMergePolicy">
<int name="mergeFactor">10</int>
</mergePolicyFactory>
I don't know if this is related to our problem, but recently we
noticed spikes in CPU usage (without iowait, and that's the
strange thing, because otherwise I would have thought about an
automatic merge)
every 3-5 seconds.
I attach a picture of the metrics collected from the system
hosting solr that show this behaviour, as well as the gc log.
Any idea why this can happen?
Thank you
Danilo
-- Danilo Tomasoni
Fondazione The Microsoft Research - University of Trento Centre for
Computational and Systems Biology (COSBI)
Piazza Manifattura 1, 38068 Rovereto (TN), Italy
tomas...@cosbi.eu
http://www.cosbi.eu
As for the European General Data Protection Regulation 2016/679 on
theprotection of natural persons with regard to the processing of
personal data, we inform you that all the data we possess are
object of treatment in the respect of the normative provided for by
the cited GDPR.
It is your right to be informed on which of your data are used and
how; you may ask for their correction, cancellation or you may
oppose to theiruse by written request sent by recorded delivery to
The Microsoft Research– University of Trento Centre for
Computational and Systems Biology Scarl, Piazza Manifattura 1,
38068 Rovereto (TN), Italy.
P Please don't print this e-mail unless you really need to
-- Danilo Tomasoni
Fondazione The Microsoft Research - University of Trento Centre for
Computational and Systems Biology (COSBI)
Piazza Manifattura 1, 38068 Rovereto (TN), Italy
tomas...@cosbi.eu
http://www.cosbi.eu
As for the European General Data Protection Regulation 2016/679 on
the protection of natural persons with regard to the processing of
personal data, we inform you that all the data we possess are object
of treatment in therespect of the normative provided for by the cited
GDPR.
It is your right to be informed on which of your data are used and
how; you may ask for their correction, cancellation or you may oppose
to their use by written request sent by recorded delivery to The
Microsoft Research – University of Trento Centre for Computational
and Systems BiologyScarl, Piazza Manifattura 1, 38068 Rovereto (TN),
Italy.
P Please don't print this e-mail unless you really need to
--
Danilo Tomasoni
Fondazione The Microsoft Research - University of Trento Centre for
Computational and Systems Biology (COSBI)
Piazza Manifattura 1, 38068 Rovereto (TN), Italy
tomas...@cosbi.eu
http://www.cosbi.eu
As for the European General Data Protection Regulation 2016/679 on the
protection of natural persons with regard to the processing of personal data,
we inform you that all the data we possess are object of treatment in the
respect of the normative provided for by the cited GDPR.
It is your right to be informed on which of your data are used and how; youmay
ask for their correction, cancellation or you may oppose to their use by
written request sent by recorded delivery to The Microsoft Research –
University of Trento Centre for Computational and Systems Biology Scarl, Piazza
Manifattura 1, 38068 Rovereto (TN), Italy.
P Please don't print this e-mail unless you really need to
