On rereading my original post it does sound weird. Let me try again and
thanks for bearing with me.
I want to know how long solr will take to process a unique query taking
full advantage of OS i/o buffers. I think executing a set of unique
queries from a cold start should measure that if I knew that I was
making maximal use of OS i/o buffering. That is, that the only variable
in my test was the amount of memory devoted to buffering index segments.
Is that basically correct?
As I scale up shard size, I just want to know that response time is
staying within bounds as I add documents. When response time falls out
of bounds, I'll add another shard but I want to know when to do that.
So I guess I'm asking how to make sure that solr is making best use of
OS i/o buffers on a dedicated server.
Phil
Otis Gospodnetic wrote:
Hi,
I think you are describing some "weird" unrealistic scenarios.
There is typically no need to test "just solr" without relying on disk caches.
Not using disk buffers will only work in trivial scenarios, but if you really want to
test it, run something that hogs memory while running solr perf tests on the same server.
You often can't prewarm caches at the very beginning because you can't predict
queries (not always true), so yes, initially caches will be empty, but then
they will get filled and then you will (want to) use them. I don't think there
is a way to clear Solr caches.
I can't give more advice at this time, I don't fully understand what you are
trying to test...
Otis
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch
----- Original Message ----
From: Phillip Farber <[EMAIL PROTECTED]>
To: solr-user@lucene.apache.org
Sent: Wednesday, August 20, 2008 1:34:20 PM
Subject: Testing query response time
I would like to test query response time for a set of queries. I'm not
interested in capacity Q/sec, just response time. My queries will be
against an index of OCR'd books so in the real world every query is
probably unique and impossible to predict so I don't see a way to
prewarm any of the caches. I'm not sorting. I'm not faceting. I'm
querying on a few fields like title, author, subject and date in a range.
Regarding initial conditions, it seems that there's no useful state into
which I can put the caches. Would the best approach be to run the
queries from a cold solr startup?
What about OS disk caches? I can see two arguments. One, just to test
solr the disk caches should be empty. On the other hand, realistically,
the disk caches would be full so that argues for executing enough
queries to load those and then redo the query set (with empty solr caches).
Speaking of empty solr caches, is there a way to flush those while solr
is running?
What other system states do I need to control for to get a handle on
response time?
Thanks and regards,
Phil
------------------------------------------
Phillip Farber - http://www.umdl.umich.edu
