On Wed, Aug 4, 2010 at 10:50 AM, Peter Karich <peat...@yahoo.de> wrote:
> Ophir,
>
> this sounds a bit strange:
>
> > CommonsHttpSolrServer.java, line 416 takes about 95% of the application's
> total search time
>
> Is this only for heavy load?
>
>
I think this makes sense, since the hard work is done by Solr - once the
application gets the search results from the shards, it does a bit of
manipulations on them (combine, filter, ...), but these are easy tasks.
Some other things:
> * with lucene you accessed the indices with MultiSearcher in a LAN, right?
>
No, each shard was run under a different tomcat instance, and each shard was
accessed by HTTP calls (the same way we're trying to work now with Solr)
> * did you look into the logs of the servers, is there something
> wrong/delayed?
>
Everything seems peachy... logs are clean of errors/warnings and the likes
> * did you enable gzip compression for your servers or even the binary
> writer/parser for your solr clients?
>
>
We're running our application (and Solr) under Tomcat. We do not enable
compression (the configuration remained similar to our old application's
configuration)
Tried using XMLResponseParser instead of BinaryResponseParser - hardly
affected run times.
Thanks for the ideas,
Ophir
CommonsHttpSolrServer server = ...
> server.setRequestWriter(new BinaryRequestWriter());
> server.setParser(new BinaryResponseParser());
>
> Regards,
> Peter.
>
> > [posted this yesterday in lucene-user mailing list, and got an advice to
> > post this here instead. excuse me for spamming]
> >
> > Hi,
> >
> > I'm currently involved in a project of migrating from Lucene 2.9.1 to
> Solr
> > 1.4.0.
> > During stress testing, I encountered this performance problem:
> > While actual search times in our shards (which are now running Solr) have
> > not changed, the total time it takes for a query has increased
> dramatically.
> > During this performance test, we of course do not modify the indexes.
> > Our application is sending Solr select queries concurrently to the 8
> shards,
> > using CommonsHttpSolrServer.
> > I added some timing debug messages, and found that
> > CommonsHttpSolrServer.java, line 416 takes about 95% of the application's
> > total search time:
> > int statusCode = _httpClient.executeMethod(method);
> >
> > Just to clarify: looking at access logs of the Solr shards, TTLB for a
> query
> > might be around 5 ms. (on all shards), but httpClient.executeMethod() for
> > this query can be much higher - say, 50 ms.
> > On average, if under light load queries take 12 ms. on average, under
> heavy
> > load the take around 22 ms.
> >
> > Another route we tried to pursue is add the "shards=shard1,shard2,…"
> > parameter to the query instead of doing this ourselves, but this doesn't
> > seem to work due to an NPE caused by QueryComponent.returnFields(), line
> > 553:
> > if (returnScores && sdoc.score != null) {
> >
> > where sdoc is null. I saw there is a null check on trunk, but since we're
> > currently using Solr 1.4.0's ready-made WAR file, I didn't see an easy
> way
> > around this.
> > Note: we're using a custom query component which extends QueryComponent,
> but
> > debugging this, I saw nothing wrong with the results at this point in the
> > code.
> >
> > Our previous code used HTTP in a different manner:
> > For each request, we created a new
> > sun.net.www.protocol.http.HttpURLConnection, and called its
> getInputStream()
> > method.
> > Under the same load as the new application, the old application does not
> > encounter the delays mentioned above.
> >
> > Our current code is initializing CommonsHttpSolrServer for each shard
> this
> > way:
> > MultiThreadedHttpConnectionManager httpConnectionManager = new
> > MultiThreadedHttpConnectionManager();
> > httpConnectionManager.getParams().setTcpNoDelay(true);
> > httpConnectionManager.getParams().setMaxTotalConnections(1024);
> > httpConnectionManager.getParams().setStaleCheckingEnabled(false);
> > HttpClient httpClient = new HttpClient();
> > HttpClientParams params = new HttpClientParams();
> > params.setCookiePolicy(CookiePolicy.IGNORE_COOKIES);
> > params.setAuthenticationPreemptive(false);
> > params.setContentCharset(StringConstants.UTF8);
> > httpClient.setParams(params);
> > httpClient.setHttpConnectionManager(httpConnectionManager);
> >
> > and passing the new HttpClient to the Solr Server:
> > solrServer = new CommonsHttpSolrServer(coreUrl, httpClient);
> >
> > We tried two different ways - one with a single
> > MultiThreadedHttpConnectionManager and HttpClient for all the
> SolrServer's,
> > and the other with a new MultiThreadedHttpConnectionManager and
> HttpClient
> > for each SolrServer.
> > Both tries yielded similar performance results.
> > Also tried to give setMaxTotalConnections() a much higher connections
> number
> > (1,000,000) - didn't have an effect.
> >
> > One last thing - to answer Lance's question about this being an "apples
> to
> > apples" comparison (in lucene-user thread) - yes, our main goal in this
> > project is to do things as close to the previous version as possible.
> > This way we can monitor that behavior (both quality and performance)
> remains
> > similar, release this version, and then move forward to improve things.
> > Of course, there are some changes, but I believe we are indeed measuring
> the
> > complete flow on both apps, and that both apps are returning the same
> fields
> > via HTTP.
> >
> > Would love to hear what you think about this. TIA,
> > Ophir
> >
> >
>
>
> --
> http://karussell.wordpress.com/
>
>
