On 7/17/2010 3:28 AM, marship wrote:
Hi. Peter and All.
I merged my indexes today. Now each index stores 10M document. Now I only have
10 solr cores.
And I used
java -Xmx1g -jar -server start.jar
to start the jetty server.
How big are the indexes on each of those cores? You can easily get this
info from a URL like this (assuming the bundled Jetty and its standard
port):
http://hostname:8983/solr/corename/admin/replication/index.jsp
If your server only has 4GB of RAM, low memory is almost guaranteed to
be the true problem. With low ram levels, the disk cache is nearly
useless, and high disk I/O is the symptom.
My system runs as virtual machines. I've got six static indexes each a
little over 12GB in size (7 million rows) and an incremental index that
gets to about 700MB (300,000 rows). I've only got one active index core
per virtual machine, except when doing a full reindex, which is rare.
Each static VM is allocated 2 CPUs and 9GB of memory, each incremental
has 2 CPUs and 3GB of memory. As I'm not using VMware, the memory is not
oversubscribed. There is a slight oversubscription of CPUs, but I've
never seen a CPU load problem. I've got dedicated VMs for load balancing
and for the brokers.
With a max heap of 1.5GB, that leaves over 7GB of RAM to act as disk
cache for a 12GB index. My statistics show that each of my two broker
cores has 185000 queries under its belt, with an average query time of
about 185 milliseconds. If I had enough memory to fit the entire 12GB
index into RAM, I'm sure my query times would be MUCH smaller.
Here's a screenshot of the status page that aggregates my Solr statistics:
http://www.flickr.com/photos/52107...@n05/4801491979/sizes/l/
