Great news for Solr -- a third party library that I'm calling is serialized.
Silly me, I made a mistake when ruling out that library as the culprit
earlier. Solr itself scales just great as add threads. JProfiler helped me
find the problem.
Sorry for the false alarm, and thanks for the suggestio
Thank you Grant and Lance for your comments -- I've run into a separate snag
which puts this on hold for a bit, but I'll return to finish digging into
this and post my results. - Michael
On Thu, Sep 24, 2009 at 9:23 PM, Lance Norskog wrote:
> Are you on Java 5, 6 or 7? Each release sees some twea
Are you on Java 5, 6 or 7? Each release sees some tweaking of the Java
multithreading model as well as performance improvements (and bug
fixes) in the Sun HotSpot runtime.
You may be tripping over the TCP/IP multithreaded connection manager.
You might wish to create each client thread with a separ
On Sep 23, 2009, at 12:09 PM, Michael wrote:
On Wed, Sep 23, 2009 at 12:05 PM, Yonik Seeley
wrote:
On Wed, Sep 23, 2009 at 11:47 AM, Michael
wrote:
If this were IO bound, wouldn't I see the same results when
sending my 8
requests to 8 Tomcats? There's only one "disk" (well, RAM)
whethe
> I'm not sure whether you'd call my operations IO heavy -- each query has
so
> many terms (~50) that even against a 45K document index a query takes
130ms,
> but the entire index is in a ramfs.
The more terms, the more it takes to find docset intersections (belonging to
each term); something in
> 8 threads sharing something may have *some* overhead versus 8 processes,
but
> as you say, 410ms overhead points to a different problem.
- You have baseline (single-threaded load-stress script sending requests to
SOLR) (1-request-in-parallel, 8 requests to 8 Tomcats); 200ms looks
extremely high
On Wed, Sep 23, 2009 at 12:05 PM, Yonik Seeley
wrote:
> On Wed, Sep 23, 2009 at 11:47 AM, Michael wrote:
> > If this were IO bound, wouldn't I see the same results when sending my 8
> > requests to 8 Tomcats? There's only one "disk" (well, RAM) whether I'm
> > querying 8 processes or 8 threads i
On Wed, Sep 23, 2009 at 11:47 AM, Michael wrote:
> Hi Yonik,
>
> On Wed, Sep 23, 2009 at 11:42 AM, Yonik Seeley
> wrote:
>>
>> This could well be IO bound - lots of seeks and reads.
>
> If this were IO bound, wouldn't I see the same results when sending my 8
> requests to 8 Tomcats? There's only
Thanks for the suggestion, Walter! I've been using Gaze 1.0 for a while
now, but when I moved to a multicore approach (which was the impetus behind
all of this testing) Gaze failed to start and I had to comment it out of
solrconfig.xml to get Solr to start. Are you aware whether Gaze is able to
w
This sure seems like a good time to try LucidGaze for Solr. That would
give some Solr-specific profiling data.
http://www.lucidimagination.com/Downloads/LucidGaze-for-Solr
wunder
On Sep 23, 2009, at 8:47 AM, Michael wrote:
Hi Yonik,
On Wed, Sep 23, 2009 at 11:42 AM, Yonik Seeley
wrote:
Hi Yonik,
On Wed, Sep 23, 2009 at 11:42 AM, Yonik Seeley
wrote:
>
> This could well be IO bound - lots of seeks and reads.
>
If this were IO bound, wouldn't I see the same results when sending my 8
requests to 8 Tomcats? There's only one "disk" (well, RAM) whether I'm
querying 8 processes or 8
Hi Fuad,
On Wed, Sep 23, 2009 at 11:37 AM, Fuad Efendi wrote:
> > 8 queries against 1 Tomcat average 600ms per query, while 8 queries
> against
> > 8 Tomcats average 190ms per query (on a dedicated 8 CPU server w 32G
> RAM).
> > I don't see how to interpret these numbers except that Tomcat is
On Wed, Sep 23, 2009 at 11:17 AM, Michael wrote:
> I'm using a Solr 1.4 nightly from around July. Is that recent enough to
> have the improved reader implementation?
> I'm not sure whether you'd call my operations IO heavy -- each query has so
> many terms (~50) that even against a 45K document i
> 8 queries against 1 Tomcat average 600ms per query, while 8 queries
against
> 8 Tomcats average 190ms per query (on a dedicated 8 CPU server w 32G RAM).
> I don't see how to interpret these numbers except that Tomcat is not
> multithreading as well as it should :)
Hi Michael, I think it is ve
On Wed, Sep 23, 2009 at 11:26 AM, Fuad Efendi wrote:
>
> - something obviously wrong in your case, 130ms is too high. Is it
> dedicated
> server? Disk swapping? Etc.
>
It's that my queries are ridiculously complex. My users are very familiar
with boolean searching, and I'm doing a lot of process
;
>
> > -Original Message-
> > From: Michael [mailto:solrco...@gmail.com]
> > Sent: September-23-09 11:17 AM
> > To: solr-user@lucene.apache.org; yo...@lucidimagination.com
> > Subject: Re: Parallel requests to Tomcat
> >
> > I'm us
nkedin.com/in/liferay
>
>
>
> > -Original Message-
> > From: Michael [mailto:solrco...@gmail.com]
> > Sent: September-22-09 4:04 PM
> > To: solr-user@lucene.apache.org
> > Subject: Parallel requests to Tomcat
> >
> > Hi,
> > I have a Solr+To
il.com]
> Sent: September-23-09 11:17 AM
> To: solr-user@lucene.apache.org; yo...@lucidimagination.com
> Subject: Re: Parallel requests to Tomcat
>
> I'm using a Solr 1.4 nightly from around July. Is that recent enough to
> have the improved reader implementation?
> I&
I'm using a Solr 1.4 nightly from around July. Is that recent enough to
have the improved reader implementation?
I'm not sure whether you'd call my operations IO heavy -- each query has so
many terms (~50) that even against a 45K document index a query takes 130ms,
but the entire index is in a ram
rializes everything too...
Fuad Efendi
http://www.linkedin.com/in/liferay
> -Original Message-
> From: Michael [mailto:solrco...@gmail.com]
> Sent: September-22-09 4:04 PM
> To: solr-user@lucene.apache.org
> Subject: Parallel requests to Tomcat
>
> Hi,
> I have a Solr
What version of Solr are you using?
Solr1.3 and Lucene 2.4 defaulted to an index reader implementation
that had to synchronize, so search operations that are IO "heavy"
can't proceed in parallel. You shouldn't see this with 1.4
-Yonik
http://www.lucidimagination.com
On Tue, Sep 22, 2009 at 4:0
Hi,
I have a Solr+Tomcat installation on an 8 CPU Linux box, and I just tried
sending parallel requests to it and measuring response time. I would expect
that it could handle up to 8 parallel requests without significant slowdown
of any individual request.
Instead, I found that Tomcat is serializ
22 matches
Mail list logo
