Not sure if this was mentioned yet, but if you are doing slave/master replication you'll need 2x the RAM at replication time. Just something to keep in mind.
-mike On Mon, Jan 10, 2011 at 5:01 PM, Toke Eskildsen <t...@statsbiblioteket.dk>wrote: > On Mon, 2011-01-10 at 21:43 +0100, Paul wrote: > > > I see from your other messages that these indexes all live on the same > machine. > > > You're almost certainly I/O bound, because you don't have enough memory > for the > > > OS to cache your index files. With 100GB of total index size, you'll > get best > > > results with between 64GB and 128GB of total RAM. > > > > Is that a general rule of thumb? That it is best to have about the > > same amount of RAM as the size of your index? > > I does not seems like there is a clear current consensus on hardware to > handle IO problems. I am firmly in the SSD camp, but as you can see from > the current thread, other people recommend RAM and/or extra machines. > > I can say that our tests with RAM and spinning disks showed us that a > lot of RAM certainly helps a lot, but also that it takes a considerable > amount of time to warm the index before the performance is satisfactory. > It might be helped with disk cache tricks, such as copying the whole > index to /dev/null before opening it in Solr. > > > So, with a 5GB index, I should have between 4GB and 8GB of RAM > > dedicated to solr? > > Not as -Xmx, but free for disk cache, yes. If you follow the RAM ~= > index size recommendation. > >
