Tom, AFAIK Lucene performance is very much dependent on file system cache size, in case of large index. So if you see lots of IO, this probably means that your system doesn't have enough memory to hold large file system cache, suitable for your index size. In this case you don't need to give more memory to java processes, but instead you need to free as much memory as you can for the OS.
On Mon, Nov 23, 2009 at 11:53 AM, Tomasz Kępski <tom...@kepski.pl> wrote: > Hi, > > Otis Gospodnetic pisze: > > Tom, >> >> It looks like the machine might simply be running too many things. >> > > If the load is around 1 when Solr is not running, and this is a dual-core > server, it shows its already relatively busy (cca 50% idle). > > The server is running the Postgresql and Apache/PHP as well, but without > solr the server condition is more than good (load usually less than 1, > sometimes , even dring rush hours we observed 1m load avg 0,68). > > It is double dual core so load 1 means 25% am I right (4 cores)? > > > Your caches are not small, so I am guessing you either have to have a >> relatively big heap, or your heap is not large enough and it's the GC that's >> causing high CPU load. >> > > The java starts with Xmx3584m. Should that be fine for such cache settings? > By the way I'm wondering if we need such caches. I did check query frequency > for last 10 days (~70000 unique users) and most frequent phrase appears ~150 > times, and only 11 queries exists more than 100 times. I did not count if > user used the same query but goes to next page. > > Is this worthy to keep quite big cache in this cas? > > > If you are seeing Solr causing lots of IO, that's a sign the box doesn't >> have enough memory for all those servers running comfortably on it. >> > > We do have some free memory to use. Server has 8G RAM and mostly uses up to > 6G, I haven't seen the swap used yet. I would try to give more RAM for java > and use smaller cache to see if it would work. > > Tom > > > -- Andrew Klochkov Senior Software Engineer, Grid Dynamics
