I'm just writing to close the loop on this issue.
I moved my servlet to a beefier server with lots of RAM. I also cleaned up
the data to make the index somewhat smaller. And, I turned off all the
caches since my application doesn't benefit very much from caching. My
application is now quite zippy,
When your index is all cached by OS you won't see disk IO. Smaller heap,
smaller caches, more RAM.
Otis
--
Performance Monitoring - http://sematext.com/spm
On Dec 15, 2012 1:11 PM, "S L" wrote:
> My virtual machine has 6GB of RAM. Tomcat is currently configured to use
> 4GB
> of it. The size of
On Sat, Dec 15, 2012 at 1:11 PM, S L wrote:
> My virtual machine has 6GB of RAM. Tomcat is currently configured to use 4GB
> of it. The size of the index is 5.4GB for 3 million records which averages
> out to 1.8KB per record. I can look at trimming the data, having fewer
> records in the index to
p.s. Regarding streaming of the dat, my Java servlet uses solrj and iterates
through the results. Right now I'm focused on getting rid of the delay that
cause some queries to take 6 or 8 seconds to complete so I'm not even
looking at the performance of the streaming.
--
View this message in con
My virtual machine has 6GB of RAM. Tomcat is currently configured to use 4GB
of it. The size of the index is 5.4GB for 3 million records which averages
out to 1.8KB per record. I can look at trimming the data, having fewer
records in the index to make it smaller, or getting more memory for the VM.
On Sat, Dec 15, 2012 at 12:04 PM, S L wrote:
> Thanks everyone for the responses.
>
> I did some more queries and watched disk activity with iostat. Sure enough,
> during some of the slow queries the disk was pegged at 100% (or more.)
>
> The requirement for the app I'm building is to be able to r
I just did the experiment of retrieving only the metaDataUrl field. I still
sometimes get slow retrieval times. One query took 2.6 seconds of real time
to retrieve 80k of data. There were 500 results. QTime was 229. So, I do
need to track down where the extra 2+ seconds is going.
--
View this me
Thanks everyone for the responses.
I did some more queries and watched disk activity with iostat. Sure enough,
during some of the slow queries the disk was pegged at 100% (or more.)
The requirement for the app I'm building is to be able to retrieve 500
results in ideally one second. The index has
On Fri, Dec 14, 2012 at 3:43 PM, S L wrote:
> Does anyone have an idea why a query that takes solr just half a second (500
> ms) to execute would take 3 seconds to transfer the data?
Normally this is due to slow reading of the stored fields (i.e. slow disk IO).
For scalability, we don't read all
: 500
:
: I'm guessing the delay is from Lucene and not the network but I could be
: wrong. 90% of my queries are 8 to 10 times faster than this.
http://wiki.apache.org/solr/SolrTerminology
QTime: The elapsed time (in milliseconds) between the arrival of the
request (when the ?SolrQueryRequest
Hi,
It's the network or disk. Monitor both when running the query for the first
time. Try the query multiple times. If it's faster the second time around
it's not the network. If it is slow the second time, it is likely the
network. Try fewer rows.
Otis
--
SOLR Performance Monitoring - http://sem
11 matches
Mail list logo
