Sorry for the delayed response ** *How big are your documents?* I have totally 1 million documents. I have totally 1950 fields in the index. Every document would probably have values for around 20 - 50 fields. *What is the total size of the index?* 1 GB
*What's the amout of RAM on your box? How big is the JVM heap (and how much free memory is left on your system)?* I have 4 GB RAM. I am using Weblogic 10, 32 Bit. Since it is a windows box, I am able to allocate only 1 GB to the JVM. No other applications are running on the system. So the entire 4GB is at the disposal of the application. I am simulating load using a load tool (15 users) *Can you show what this slow query looks like (the whole request)?* q=*%3A*&rows=0&facet=true&facet.mincount=1&facet.limit=2&f.S9942.facet.limit=100&facet.field=S9942&facet.field=S6878&facet.field=S9156&facet.field=S0369&facet.field=S9926&facet.field=S1421&facet.field=S8990&facet.field=S6881&facet.field=S3552&debugQuery=true q=*%3A*&fq=S9942%3A%22TEXAS+INSTRUMENTS%22&rows=0&facet=true&facet.mincount=1&facet.limit=2&facet.field=S9942&facet.field=S6878&facet.field=S9156&facet.field=S0369&facet.field=S9926&facet.field=S1421&facet.field=S8990&facet.field=S6881&facet.field=S3552&debugQuery=true Other information Solr 1.3, JDK 1.5.0_14 regards Rahul On Mon, Sep 28, 2009 at 6:48 PM, Yonik Seeley <yo...@lucidimagination.com>wrote: > On Mon, Sep 28, 2009 at 7:51 AM, Rahul R <rahul.s...@gmail.com> wrote: > > Yonik, > > I understand that the network can be a bottle-neck but I am pretty sure > that > > it is not. I am operating on a 100 MBPS intranet....... How do I ensure > that > > stored fields are cached by the OS ? Only the Solr caches within the JVM > are > > under my control...... The result set has around 10K documents of which I > am > > retrieving only 10......I am displaying a max of only 3 fields per > document > > in my result set. Can the reading time for these stored fields be so long > ? > > It could be a seek per document if the index is too big to fit in the > OS cache - but that still wouldn't be as slow as you report. > Something is fishy here. > > How big are your documents? > What is the total size of the index? > What's the amout of RAM on your box? > How big is the JVM heap (and how much free memory is left on your system)? > Can you show what this slow query looks like (the whole request)? > > > I have totally around 1 million documents in my index........ Any > thoughts > > on why the FacetComponent does not take any time while the QueryComponent > > takes around 2.4s. > > It could be a field that has very few unique values and faceting just > completes quickly. > Make sure you're actually getting faceting data back (that it's > correctly turned on). > > -Yonik > http://www.lucidimagination.com > > > I am doing a faceted and keyword query ie I have both 'q' > > and 'fq' params in my query.... Thank you for your response. > > > > Regards > > Rahul > > > > On Mon, Sep 28, 2009 at 1:20 AM, Yonik Seeley < > yo...@lucidimagination.com> > > wrote: > >> > >> The response times in a Solr request don't include the time to read > >> stored fields (since the response is streamed) and doesn't include the > >> time to transfer/read the response (which can be increased by a > >> slow/congested network link, or a slow client that doesn't read the > >> response immediately). > >> > >> How many documents are you retrieving? Reading stored fields for > >> documents can be slow if they aren't cached by the OS since it's often > >> a disk seek per document read for a large index. > >> > >> -Yonik > >> http://www.lucidimagination.com > >> > >> > >> > >> On Sun, Sep 27, 2009 at 3:41 PM, Rahul R <rahul.s...@gmail.com> wrote: > >> > Hello, > >> > I am trying to measure why some of my queries take a long time. I am > >> > using > >> > EmbeddedSolrServer and with logging statements before and > >> > after the EmbeddedSolrServer.query(SolrQuery) function, I have found > the > >> > time to be around 16s. I added the debugQuery=true and the timing > >> > component > >> > for this reads as following: > >> > > >> > * > >> > > >> > > timing:{time=2438.0,prepare={time=0.0,org.apache.solr.handler.component.QueryComponent={time=0.0},org.apache.solr.handler.component.FacetComponent={time=0.0},org.apache.solr.handler.component.MoreLikeThisComponent={time=0.0},org.apache.solr.handler.component.HighlightComponent={time=0.0},org.apache.solr.handler.component.DebugComponent={time=0.0}},process={time=2438.0,org.apache.solr.handler.component.QueryComponent={time=2438.0},org.apache.solr.handler.component.FacetComponent={time=0.0},org.apache.solr.handler.component.MoreLikeThisComponent={time=0.0},org.apache.solr.handler.component.HighlightComponent={time=0.0},org.apache.solr.handler.component.DebugComponent={time=0.0}}} > >> > * > >> > > >> > As you can see, this shows only 2.4s being used by the query. I can't > >> > seem > >> > to figure out where the rest of the time is being spent. This is > within > >> > my > >> > office intranet and I don't think the request-response time over the > >> > wire > >> > will cause significant overhead. So my question : is the timing > >> > information > >> > presented here comprehensive or are there more time consuming > operations > >> > that are not represented here ? I guess GC pause times could be one > >> > answer > >> > (I hope not !).... Also, the above result was for a faceted query. I > >> > can't > >> > understand why the FacetComponent would be zero. Any thoughts ? > >> > > >> > Rahul > >> > > > > > >
