Re: how to improve concurrent request performance and stress testing

2009-04-01 Thread sunnyfr
Thanks for all this help, But I guess it can't be optimal with a lot of update, my slave get back from the master 20 000docs updated every 20minutes, it's made to try to warmup have a big cache and everything go fast with that amount of update I guess ...? zqzuk wrote: > > Hi, > > try to f

Re: how to improve concurrent request performance and stress testing

2008-10-29 Thread zqzuk
Yes, I created different workers and with same or different queries. sorry it has been a while since I used it, and the link to its source code (no jars) should be here: http://code.google.com/p/httpstone/source/checkout sunnyfr wrote: > > just a question about your httpstone's configuration

Re: how to improve concurrent request performance and stress testing

2008-10-29 Thread sunnyfr
just a question about your httpstone's configuration ? I would like to know how did you simulate several word search ... ?? Did you create a lot of different workers with lof of different word search ? Thanks, zqzuk wrote: > > Hi, > > try to firstly have a look at http://wiki.apache.o

Re: how to improve concurrent request performance and stress testing

2008-10-29 Thread sunnyfr
Dont get really with httpstone, I did my ant dist it works fine, but then don't get ... when I do a java -jar I've an error: ~/httpstone-read-only/dist/lib$ java -jar httpstone.jar Failed to load Main-Class manifest attribute from httpstone.jar Any idea? Sorry I'm new in java zqzuk wrote: >

Re: how to improve concurrent request performance and stress testing

2008-10-29 Thread zqzuk
Hi, try to firstly have a look at http://wiki.apache.org/solr/SolrCaching the section on firstsearcher and warming. Search engines rely on caching, so first searches will be slow. I think to be fair testing it is necessary to warm up the search engine by sending most frequently used and/or most

Re: how to improve concurrent request performance and stress testing

2008-10-29 Thread sunnyfr
Hi, I'm trying as well to stress test solr. I would love some advice to manage it properly. I'm using solr 1.3 and tomcat55. Thanks a lot, zqzuk wrote: > > Hi, I am doing a stress testing of my solr application to see how many > concurrent requests it can handle and how long it takes. But I m

Re: how to improve concurrent request performance and stress testing

2008-02-07 Thread Chris Hostetter
: Thank you so much! I will look into firstSearcher configuration next! thanks FYI: prompted by this thread, I added some blurbs about firstSearcher, newSearcher, and FieldCache to the SolrCaching wiki ... as a new users learning about this stuff, please fele free to update that wiki with any

Re: how to improve concurrent request performance and stress testing

2008-02-07 Thread Ziqi Zhang
Thank you so much! I will look into firstSearcher configuration next! thanks -- From: "Chris Hostetter" <[EMAIL PROTECTED]> Sent: Wednesday, February 06, 2008 8:56 PM To: Subject: Re: how to improve concurrent request perfor

Re: how to improve concurrent request performance and stress testing

2008-02-06 Thread Chris Hostetter
: > Also make sure that common filters, sort fields, and facets have been : > warmed. : : I assume these are achieved by setting large cache size and large : autowarmcount number in solr configuration? specifically autowarming seeds the cahces of a new Searcher using hte keys of an old searcher

Re: how to improve concurrent request performance and stress testing

2008-02-06 Thread Yonik Seeley
On Feb 7, 2008 12:26 AM, Ziqi Zhang <[EMAIL PROTECTED]> wrote: > Thanks Otis! > > I think I now got a clearer picture of the issue and its causes, thanks. > > Could you please elaborate on "warming up" searcher prior exposure to real > requests, does this mean running through as many most often use

Re: how to improve concurrent request performance and stress testing

2008-02-06 Thread Ziqi Zhang
lso use as much cache as possible? Thanks! -- From: "Otis Gospodnetic" <[EMAIL PROTECTED]> Sent: Wednesday, February 06, 2008 1:09 PM To: Subject: Re: how to improve concurrent request performance and stress testing

Re: how to improve concurrent request performance and stress testing

2008-02-06 Thread Otis Gospodnetic
t.com/ -- Lucene - Solr - Nutch - Original Message > From: Ziqi Zhang <[EMAIL PROTECTED]> > To: solr-user@lucene.apache.org > Sent: Wednesday, February 6, 2008 7:56:46 PM > Subject: Re: how to improve concurrent request performance and stress testing > > Thanks! &

Re: how to improve concurrent request performance and stress testing

2008-02-06 Thread Otis Gospodnetic
t; Sent: Wednesday, February 6, 2008 6:37:40 PM > Subject: Re: how to improve concurrent request performance and stress testing > > Thanks > Yonik, > > > > > It > uses > a > thread > per > request, > simultaneously > (up > to > any >

Re: how to improve concurrent request performance and stress testing

2008-02-06 Thread Ziqi Zhang
! -- From: "Yonik Seeley" <[EMAIL PROTECTED]> Sent: Wednesday, February 06, 2008 7:50 AM To: Subject: Re: how to improve concurrent request performance and stress testing On Feb 6, 2008 6:37 PM, Ziqi Zhang <[EMAIL PROTECTED]> wrote: I still do not unde

Re: how to improve concurrent request performance and stress testing

2008-02-06 Thread Yonik Seeley
On Feb 6, 2008 6:37 PM, Ziqi Zhang <[EMAIL PROTECTED]> wrote: > I still do not understand why sending 100 request (of same query) from 100 > threads throws solr server to silence - is it because of the computational > cost to deal with same query in 100 separate threads? Yes... sending a large num

Re: how to improve concurrent request performance and stress testing

2008-02-06 Thread Ziqi Zhang
Thanks Yonik, It uses a thread per request, simultaneously (up to any limit configured by the app server) How can I change this setting then? I suppose it is to do with Jetty or Tomcat whichever hosts solr application, not through the solrconfig? I still do not understand why sending 100 r

Re: how to improve concurrent request performance and stress testing

2008-02-06 Thread Yonik Seeley
On Feb 6, 2008 7:53 AM, zqzuk <[EMAIL PROTECTED]> wrote: > If solr receives 10 concurrent request, does it deal with the 10 requests > simultaneously It uses a thread per request, simultaneously (up to any limit configured by the app server) > using 10 (or as many as possible) searchers There is

Re: how to improve concurrent request performance and stress testing

2008-02-06 Thread zqzuk
Hi, I see there's no response to my question, maybe its better to ask this way... If solr receives 10 concurrent request, does it deal with the 10 requests simultaneously using 10 (or as many as possible) searchers, or does it deal with each request sequentially, which implies that first request