Hi Frederik,
Did you figure out a solution to this problem?
I'm asking because I recently ran into a similar problem, with a similar setup
(8 shards on one server).
Occasionally a query will take a very long time. Occasionally I see timeout
exceptions with the HTTP requests. E.g.
> 348914 [po
Come cache hit problems can be fixed with the Large Pages feature.
http://www.google.com/search?q=large+pages
On Wed, Sep 28, 2011 at 3:30 PM, Federico Fissore wrote:
> Frederik Kraus, il 28/09/2011 23:16, ha scritto:
>
> Yep, I'm not getting more than 50-60% CPU during those load tests.
>>
>>
Frederik Kraus, il 28/09/2011 23:16, ha scritto:
Yep, I'm not getting more than 50-60% CPU during those load tests.
I would try reducing the number of shards. A part from the memory
discussion, this really seems to me a concurrency issue: too many
threads waiting for other threads to compl
---
> From: Federico Fissore [mailto:feder...@fissore.org]
> Sent: Wednesday, September 28, 2011 2:04 PM
> To: solr-user@lucene.apache.org (mailto:solr-user@lucene.apache.org)
> Subject: Re: strange performance issue with many shards on one server
>
> Jaeger, Jay - DOT, il 28/09/201
roblem, it would show up in a
higher CPU utilization than the correspondent reported.
-Original Message-
From: Federico Fissore [mailto:feder...@fissore.org]
Sent: Wednesday, September 28, 2011 2:04 PM
To: solr-user@lucene.apache.org
Subject: Re: strange performance issue with many shards on
Jaeger, Jay - DOT, il 28/09/2011 18:40, ha scritto:
That would still show up as the CPU being busy.
i don't know how the program (top, htop, whatever) displays the value
but when the cpu has a cache miss definitely that thread sits and waits
for a number of clock cycles
with 130GB of ram (
Am Mittwoch, 28. September 2011 um 16:40 schrieb Toke Eskildsen:
> On Wed, 2011-09-28 at 12:58 +0200, Frederik Kraus wrote:
> > - 10 shards per server (needed for response times) running in a single
> > tomcat instance
>
> Have you tested that sharding actually decreases response times in your
Hi Ken,
the HttpConnectionManager was actually the first thing I looked at - and bumped
the Solr default of 20 up to 50, 100, 400, 1 (which should be more or less
unlimited ;) ). Unfortunately didn't really solve anything. I don't know if the
"static" HttpClient is a problem here as it w
That would still show up as the CPU being busy.
-Original Message-
From: Federico Fissore [mailto:feder...@fissore.org]
Sent: Wednesday, September 28, 2011 6:12 AM
To: solr-user@lucene.apache.org
Subject: Re: strange performance issue with many shards on one server
Frederik Kraus, il
Hi Frederik,
I haven't directly run into this issue with Solr, but I have experienced
similar issues in a related context.
In my case, I had a custom webapp that made SolrJ requests and then generated
some aggregated/analyzed results.
During load testing, we ran into a few different issues...
On Wed, 2011-09-28 at 12:58 +0200, Frederik Kraus wrote:
> - 10 shards per server (needed for response times) running in a single tomcat
> instance
Have you tested that sharding actually decreases response times in your
case? I see the idea in decreasing response times with sharding at the
cost o
Hmm, sorry don't know...
My ideas:
- tomcat generate this problem (for example: maxthreads, number of
connections...)
- JVM - Options, especially GC
- index locks, eventually an open issue in jira
Regards
Vadim
2011/9/28 Frederik Kraus
> I just had a look at the thread-dump, pasting 3 exampl
I just had a look at the thread-dump, pasting 3 examples here:
'pool-31-thread-8233' Id=11626, BLOCKED on
lock=org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$ConnectionPool@19dd10d9,
total cpu time=20.ms user time=20.ms
at
org.apache.commons.httpclient.MultiThreadedHt
Am Mittwoch, 28. September 2011 um 13:41 schrieb Vadim Kisselmann:
> Hi Fred,
>
> ok, it's a strange behavior with same queries.
> Another questions:
> -which solr version?
3.3 (might the NIOFSDirectory from 3.4 help?)
> -do you indexing during your load test? (because of index rebuilt)
nope
Hi Fred,
ok, it's a strange behavior with same queries.
Another questions:
-which solr version?
-do you indexing during your load test? (because of index rebuilt)
-do you replicate your index?
Regards
Vadim
2011/9/28 Frederik Kraus
> Hi Vladim,
>
> the thing is, that those exact same queries
Hi Vladim,
the thing is, that those exact same queries, that take longer during a load
test, perform just fine when executed at a slower request rate and are also
random, i.e. there is no pattern in bad/slow queries.
My first thought was some kind of contention and/or connection starvation for
Hi Fred,
analyze the queries which take longer.
We observe our queries and see the problems with q-time with queries which
are complex, with phrase queries or queries which contains numbers or
special characters.
if you don't know it:
http://www.hathitrust.org/blogs/large-scale-search/tuning-search
Frederik Kraus, il 28/09/2011 12:58, ha scritto:
Hi,
I am experiencing a strange issue doing some load tests. Our setup:
just because I've listened to JUG mates talking about that at the last
meeting, could it be that your CPUs are spending their time getting
things from RAM to CPU cache
Hi,
I am experiencing a strange issue doing some load tests. Our setup:
- 2 server with each 24 cpu cores, 130GB of RAM
- 10 shards per server (needed for response times) running in a single tomcat
instance
- each query queries all 20 shards (distributed search)
- each shard holds about 1.5
19 matches
Mail list logo
