I'm pretty sure we found the problem.
It's related to memory.
Sometimes Windows seems to unmap index files from memory, because other
processes need it.
To force Windows to map index files again, we need to rebuild the index.
We can clearly see this behaviour with tools like RAMMap.
With servers
Also check the Zookeeper config file settings for Zookeeper's keep-alive,
in some versions that was set to 15 seconds. Still shouldn't
be tripped by a 4 second GC pause.
Best,
Erick
On Thu, Oct 15, 2015 at 7:33 AM, Lorenzo Fundaró
wrote:
> On 15 October 2015 at 15:59, Eric Torti wrote:
>
>> Hi,
On 15 October 2015 at 15:59, Eric Torti wrote:
> Hi, Lorenzo,
>
> I don't think this has a direct relation to your problem but it looks
> like you're setting -DzkClientTimeout twice. From what I know about
>
Thanks, I am aware of this double setting, but I also think the jvm is
taking the last a
Hi, Lorenzo,
I don't think this has a direct relation to your problem but it looks
like you're setting -DzkClientTimeout twice. From what I know about
setting VM arguments twice, you're
probably ending up with the last one being enforced.
Just something to be aware of I guess.
I don't think this
On 14 October 2015 at 20:35, Pushkar Raste wrote:
> You may want to start solr with following settings to enable logging GC
> details. Here are some flags you might want to enable.
>
> -Xloggc:/gc.log
> -XX:+PrintGCDetails
> -XX:+PrintGCDateStamps
> -XX:+PrintGCTimeStamps
> -XX:+PrintTenuringDist
You may want to start solr with following settings to enable logging GC
details. Here are some flags you might want to enable.
-Xloggc:/gc.log
-XX:+PrintGCDetails
-XX:+PrintGCDateStamps
-XX:+PrintGCTimeStamps
-XX:+PrintTenuringDistribution
-XX:+PrintGCApplicationStoppedTime
-XX:+PrintHeapAtGC
Onc
bq: They are definetly cached. The second time runs in no time.
That's not what I was referring to. Submitting the same query
over will certainly hit the queryResultCache and return in
almost no time.
What I meant was do things like vary the fq clause you have
where you've set cache=false. Or var
On 14 October 2015 at 18:18, Pushkar Raste wrote:
> Consider
> 1. Turning on docValues for fields you are sorting, faceting on. This will
> require to reindex your data
>
Yes. I am considering doing this.
> 2. Try using TrieInt type field you are trying to do range search on (you
> may have to
<>
"debug": { "rawquerystring": "*:*", "querystring": "*:*", "parsedquery":
"(+MatchAllDocsQuery(*:*))/no_coord", "parsedquery_toString": "+*:*", "
explain": { "Product:47047358": "\n1.0 = (MATCH) MatchAllDocsQuery, product
of:\n 1.0 = queryNorm\n", "Product:3223": "\n1.0 = (MATCH)
MatchAllDoc
Consider
1. Turning on docValues for fields you are sorting, faceting on. This will
require to reindex your data
2. Try using TrieInt type field you are trying to do range search on (you
may have to fiddle with precisoinStep) to balance index size vs performance.
3. If slowness is intermittent - tu
Hi Lorenzo,
Can you provide which solr version you are using, index size on disks &
hardware config (memory/processor on each machine.
Thanks,
Susheel
On Wed, Oct 14, 2015 at 6:03 AM, Lorenzo Fundaró <
lorenzo.fund...@dawandamail.com> wrote:
> Hello,
>
> I have following conf for filters and co
A couple of things don't particularly make sense here:
You specify edismax, q=*:* yet you specify qf=
You're searching across whatever you defined as the default
field in the request handler. What do you see if you attach
&debug=true to the query?
I think this clause is wrong:
(cents_ri: [* 3000]
Hi,
using Jetty is the recommended approach while using Tomcat is not recommend
(unless you are a Tomcat shop).
But any discussion comes back to the original question - why is it slow now?
Are you I/O-bound, are CPU-bound, how many documents are committed/deleted over
the time, do you having
THnks for the answer
A dedicated box will be a great solution but I will wait for that solution,
I have restricted sources
Is Optimze action can improve performance?
Is using default servlet engine Jetty can be harmful for the performance,
SHould I use an independant tomcat engine?
rgds,
--
Vi
It might be a good idea to
* move SOLR to a dedicated box :-)
* load your SOLR server with 20.000.000 documents (the estimated number of
documents after three years) and do performance testing & tuning
Afterwards you have some hard facts about hardware sizing and expected
performance for the ne
bq: Is it better to put the solr on dedicated machine?
Yes, absolutely. Solr _likes_ memory, and on a
machine with lots of other processes you'll keep
running into this problem.
FWIW, I've seen between 10M and 300M docs fit into
16G for the JVM. But see Uwe's excellent blog on MMapDirectory
and n
Yes performance degraded over the time, I can raise the memory but I can't
do it every time and the volume will keep growing
Is it better to put the solr on dedicated machine?
Is there any thing else that can be done to the solr instance for example
deviding the collection?
rgds,
--
View this
If you performance was fine but degraded over the time it might be
easier to check / increase the memory to have better disk caching.
Cheers,
Siegfried Goeschl
On 02.12.14 09:27, melb wrote:
Hi,
I have a solr collection with 16 millions documents and growing daily with
1 documents
recen
mailto:erickerick...@gmail.com]
Sent: Monday, March 25, 2013 10:04 PM
To: solr-user@lucene.apache.org
Subject: Re: Slow queries for common terms
take a look here:
http://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html
looking at memory consumption can be a bit tricky to interpret wi
ce when I can give it say 10GB of RAM?
>
> Thanks, this has been tremendously helpful.
>
> David
>
>
> -----Original Message-
> From: Tom Burton-West [mailto:tburt...@umich.edu]
> Sent: Saturday, March 23, 2013 1:38 AM
> To: solr-user@lucene.apache.org
> Su
Thanks, this has been tremendously helpful.
David
-Original Message-
From: Tom Burton-West [mailto:tburt...@umich.edu]
Sent: Saturday, March 23, 2013 1:38 AM
To: solr-user@lucene.apache.org
Subject: Re: Slow queries for common terms
Hi David and Jan,
I wrote the blog post, and David, you are
Hi David and Jan,
I wrote the blog post, and David, you are right, the problem we had was
with phrase queries because our positions lists are so huge. Boolean
queries don't need to read the positions lists. I think you need to
determine whether you are CPU bound or I/O bound.It is possible
proper load testing. Right now I'm just trying to get it to "work"
> for a few users. If you could elaborate a bit on your thinking I'd be quite
> grateful.
>
> David
>
>
> -Original Message-
> From: Jan Høydahl [mailto:jan....@cominvent.com]
>
to "work"
for a few users. If you could elaborate a bit on your thinking I'd be quite
grateful.
David
-Original Message-
From: Jan Høydahl [mailto:jan@cominvent.com]
Sent: Thursday, March 21, 2013 8:01 PM
To: solr-user@lucene.apache.org
Subject: Re: Slow queries for
Hi David,
I happen to know CommonTermsQuery added recently. But now sure how to use it.
http://lucene.apache.org/core/4_1_0/queries/org/apache/lucene/queries/CommonTermsQuery.html
--- On Thu, 3/21/13, David Parks wrote:
> From: David Parks
> Subject: Slow queries for common terms
> To: "sol
;category book" is getting included, which seems logical and correct.
>
>
>
> -Original Message-----
> From: Jan Høydahl [mailto:jan@cominvent.com]
> Sent: Thursday, March 21, 2013 5:43 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Slow queries for common terms
>
common phrase
"category book" is getting included, which seems logical and correct.
-Original Message-
From: Jan Høydahl [mailto:jan@cominvent.com]
Sent: Thursday, March 21, 2013 5:43 PM
To: solr-user@lucene.apache.org
Subject: Re: Slow queries for common terms
Hi,
I
Hi,
I think you can start by reading this blog
http://www.hathitrust.org/blogs/large-scale-search/slow-queries-and-common-words-part-2
and try out the approach using a dictionary of the most common words in your
index.
You don't say how many documents, avg. doc size, the IDF value of "book", h
28 matches
Mail list logo
