On Mon, Sep 28, 2009 at 7:51 AM, Rahul R <rahul.s...@gmail.com> wrote: > Yonik, > I understand that the network can be a bottle-neck but I am pretty sure that > it is not. I am operating on a 100 MBPS intranet....... How do I ensure that > stored fields are cached by the OS ? Only the Solr caches within the JVM are > under my control...... The result set has around 10K documents of which I am > retrieving only 10......I am displaying a max of only 3 fields per document > in my result set. Can the reading time for these stored fields be so long ?
It could be a seek per document if the index is too big to fit in the OS cache - but that still wouldn't be as slow as you report. Something is fishy here. How big are your documents? What is the total size of the index? What's the amout of RAM on your box? How big is the JVM heap (and how much free memory is left on your system)? Can you show what this slow query looks like (the whole request)? > I have totally around 1 million documents in my index........ Any thoughts > on why the FacetComponent does not take any time while the QueryComponent > takes around 2.4s. It could be a field that has very few unique values and faceting just completes quickly. Make sure you're actually getting faceting data back (that it's correctly turned on). -Yonik http://www.lucidimagination.com > I am doing a faceted and keyword query ie I have both 'q' > and 'fq' params in my query.... Thank you for your response. > > Regards > Rahul > > On Mon, Sep 28, 2009 at 1:20 AM, Yonik Seeley <yo...@lucidimagination.com> > wrote: >> >> The response times in a Solr request don't include the time to read >> stored fields (since the response is streamed) and doesn't include the >> time to transfer/read the response (which can be increased by a >> slow/congested network link, or a slow client that doesn't read the >> response immediately). >> >> How many documents are you retrieving? Reading stored fields for >> documents can be slow if they aren't cached by the OS since it's often >> a disk seek per document read for a large index. >> >> -Yonik >> http://www.lucidimagination.com >> >> >> >> On Sun, Sep 27, 2009 at 3:41 PM, Rahul R <rahul.s...@gmail.com> wrote: >> > Hello, >> > I am trying to measure why some of my queries take a long time. I am >> > using >> > EmbeddedSolrServer and with logging statements before and >> > after the EmbeddedSolrServer.query(SolrQuery) function, I have found the >> > time to be around 16s. I added the debugQuery=true and the timing >> > component >> > for this reads as following: >> > >> > * >> > >> > timing:{time=2438.0,prepare={time=0.0,org.apache.solr.handler.component.QueryComponent={time=0.0},org.apache.solr.handler.component.FacetComponent={time=0.0},org.apache.solr.handler.component.MoreLikeThisComponent={time=0.0},org.apache.solr.handler.component.HighlightComponent={time=0.0},org.apache.solr.handler.component.DebugComponent={time=0.0}},process={time=2438.0,org.apache.solr.handler.component.QueryComponent={time=2438.0},org.apache.solr.handler.component.FacetComponent={time=0.0},org.apache.solr.handler.component.MoreLikeThisComponent={time=0.0},org.apache.solr.handler.component.HighlightComponent={time=0.0},org.apache.solr.handler.component.DebugComponent={time=0.0}}} >> > * >> > >> > As you can see, this shows only 2.4s being used by the query. I can't >> > seem >> > to figure out where the rest of the time is being spent. This is within >> > my >> > office intranet and I don't think the request-response time over the >> > wire >> > will cause significant overhead. So my question : is the timing >> > information >> > presented here comprehensive or are there more time consuming operations >> > that are not represented here ? I guess GC pause times could be one >> > answer >> > (I hope not !).... Also, the above result was for a faceted query. I >> > can't >> > understand why the FacetComponent would be zero. Any thoughts ? >> > >> > Rahul >> > > >
