So I need someone with better knowledge to chime in here with an opinion on whether autowarming would help since the whole faceting thing is something I'm not very comfortable with...
<hint, hint, hint> Erick On Mon, Nov 2, 2009 at 2:21 PM, bharath venkatesh < bharathv6.proj...@gmail.com> wrote: > @Israel: yes I got that point which yonik mentioned .. but is qtime the > total time taken by solr server for that request or is it part of time > taken by the solr for that request ( is there any thing that a solr server > does for that particulcar request which is not included in that qtime > bracket ) ? I am sorry for dragging in to this qtime. I just want to be > sure, as we observed many times there is huge mismatch between qtime and > time measured at the client for the response ( does this imply it is due to > internal network issue ) > > @Erick: yes, many times query is slow first time its executed is there any > solution to improve upon this factor .. for querying we use > DisMaxRequestHandler , queries are quite long with many faceting parameters > . > > > On Mon, Nov 2, 2009 at 10:46 PM, Israel Ekpo <israele...@gmail.com> wrote: > > > On Mon, Nov 2, 2009 at 9:52 AM, bharath venkatesh < > > bharathv6.proj...@gmail.com> wrote: > > > > > Thanks for the quick response > > > @yonik > > > > > > >How much of a latency compared to normal, and what version of Solr are > > > you using? > > > > > > latency is usually around 2-4 secs (some times it goes more than that > > > ) which happens to only 15-20% of the request other 80-85% of > > > request are very fast it is in milli secs ( around 200,000 requests > > > happens every day ) > > > > > > @Israel we are not using java client .. we r using python at the > > > client with response formatted in json > > > > > > @yonikn @Israel does qtime measure the total time taken at the solr > > > server ? I am already measuring the time to get the response at > > > client end . I would want a means to know how much time the solr > > > server is taking to respond (process ) once it gets the request . so > > > that I could identify whether it is a solr server issue or internal > > > network issue > > > > > > > It is the time spent at the Solr server. > > > > I think Yonik already answered this part in his response to your thread : > > > > This is what he said : > > > > QTime is the time spent in generating the in-memory representation for > > the response before the response writer starts streaming it back in > > whatever format was requested. The stored fields of returned > > documents are also loaded at this point (to enable handling of huge > > response lists w/o storing all in memory). > > > > > > > > > > @Israel we are using rhel server 5 on both client and server .. we > > > have 6 solr sever . one is acting as master . both client and solr > > > sever are on the same network . those servers are dedicated solr > > > server except 2 severs which have DB and memcahce running .. we have > > > adjusted the load accordingly > > > > > > > > > > > > > > > > > > > > > > > > On 11/2/09, Israel Ekpo <israele...@gmail.com> wrote: > > > > On Mon, Nov 2, 2009 at 8:41 AM, Yonik Seeley > > > > <yo...@lucidimagination.com>wrote: > > > > > > > >> On Mon, Nov 2, 2009 at 8:13 AM, bharath venkatesh > > > >> <bharathv6.proj...@gmail.com> wrote: > > > >> > We are using solr for many of ur products it is doing quite > well > > > >> > . But since no of hits are becoming high we are experiencing > > latency > > > >> > in certain requests ,about 15% of our requests are suffering a > > latency > > > >> > > > >> How much of a latency compared to normal, and what version of Solr > are > > > >> you using? > > > >> > > > >> > . We are trying to identify the problem . It may be due to > > network > > > >> > issue or solr server is taking time to process the request . > > other > > > >> > than qtime which is returned along with the response is there any > > > >> > other way to track solr servers performance ? > > > >> > how is qtime calculated > > > >> > , is it the total time from when solr server got the request till > it > > > >> > gave the response ? > > > >> > > > >> QTime is the time spent in generating the in-memory representation > for > > > >> the response before the response writer starts streaming it back in > > > >> whatever format was requested. The stored fields of returned > > > >> documents are also loaded at this point (to enable handling of huge > > > >> response lists w/o storing all in memory). > > > >> > > > >> There are normally servlet container logs that can be configured to > > > >> spit out the real total request time. > > > >> > > > >> > can we do some extra logging to track solr servers > > > >> > performance . ideally I would want to pass some log id along with > > the > > > >> > request (query ) to solr server and solr server must log the > > > >> > response time along with that log id . > > > >> > > > >> Yep - Solr isn't bothered by params it doesn't know about, so just > put > > > >> logid=xxxxxxx and it should also be logged with the other request > > > >> params. > > > >> > > > >> -Yonik > > > >> http://www.lucidimagination.com > > > >> > > > > > > > > > > > > > > > > If you are not using Java then you may have to track the elapsed time > > > > manually. > > > > > > > > If you are using the SolrJ Java client you may have the following > > > options: > > > > > > > > There is a method called getElapsedTime() in > > > > org.apache.solr.client.solrj.response.SolrResponseBase which is > > available > > > to > > > > all the subclasses > > > > > > > > I have not used it personally but I think this should return the time > > > spent > > > > on the client side for that request. > > > > > > > > The QTime is not the time on the client side but the time spent > > > internally > > > > at the Solr server to process the request. > > > > > > > > > > > > > > http://lucene.apache.org/solr//api/solrj/org/apache/solr/client/solrj/response/SolrResponseBase.html > > > > > > > > > > > > > > http://lucene.apache.org/solr//api/solrj/org/apache/solr/client/solrj/response/QueryResponse.html > > > > > > > > Most likely it could be as a result of an internal network issue > > between > > > the > > > > two servers or the Solr server is competing with other applications > for > > > > resources. > > > > > > > > What operating system is the Solr server running on? Is you client > > > > application connection to a Solr server on the same network or over > the > > > > internet? Are there other applications like database servers etc > > running > > > on > > > > the same machine? If so, then the DB server (or any other > application) > > > and > > > > the Solr server could be competing for resources like CPU, memory > etc. > > > > > > > > If you are using Tomcat, you can take a look in > > > > $CATALINA_HOME/logs/catalina.out, there are timestamps there that can > > > also > > > > guide you. > > > > > > > > -- > > > > "Good Enough" is not good enough. > > > > To give anything less than your best is to sacrifice the gift. > > > > Quality First. Measure Twice. Cut Once. > > > > > > > > > > > > > > > -- > > "Good Enough" is not good enough. > > To give anything less than your best is to sacrifice the gift. > > Quality First. Measure Twice. Cut Once. > > >
