Re: Measuring timing with debugQuery=true

2009-09-29 Thread Yonik Seeley
It's harder debugging stuff like this with custom code (you say that you're using EmbeddedSolrServer) and different servlet containers. Perahps try putting your config files and index into the example jetty server, and then do a single request from curl or your web browser to see if the times are

Re: Measuring timing with debugQuery=true

2009-09-29 Thread Rahul R
I just want to clarify here that I understand my memory allocation might be less given the load on the system. The response times were only slightly better when we ran the test on a Solaris box with 12CPU, 24G RAM and with 3.2 GB allocated for the JVM. I know that I have a performance problem.

Re: Measuring timing with debugQuery=true

2009-09-29 Thread Rahul R
Sorry for the delayed response ** *How big are your documents?* I have totally 1 million documents. I have totally 1950 fields in the index. Every document would probably have values for around 20 - 50 fields. *What is the total size of the index?* 1 GB *What's the amout of RAM on your box? How bi

Re: Measuring timing with debugQuery=true

2009-09-28 Thread Yonik Seeley
On Mon, Sep 28, 2009 at 7:51 AM, Rahul R wrote: > Yonik, > I understand that the network can be a bottle-neck but I am pretty sure that > it is not. I am operating on a 100 MBPS intranet... How do I ensure that > stored fields are cached by the OS ? Only the Solr caches within the JVM are > un

Re: Measuring timing with debugQuery=true

2009-09-28 Thread Rahul R
Yonik, I understand that the network can be a bottle-neck but I am pretty sure that it is not. I am operating on a 100 MBPS intranet... How do I ensure that stored fields are cached by the OS ? Only the Solr caches within the JVM are under my control.. The result set has around 10K document

Re: Measuring timing with debugQuery=true

2009-09-27 Thread Yonik Seeley
The response times in a Solr request don't include the time to read stored fields (since the response is streamed) and doesn't include the time to transfer/read the response (which can be increased by a slow/congested network link, or a slow client that doesn't read the response immediately). How