On 8/10/2014 11:07 PM, anand.mahajan wrote: > Thank you for your suggestions. With the autoCommit (every 10 mins) and > softCommit (every 10 secs) frequencies reduced things work much better now. > The CPU usages has gone down considerably too (by about 60%) and the > read/write throughput is showing considerable improvements too. > > There are a certain shards that are giving poor response times - these have > over 10M listings - I guess this is due to the fact that these are starving > for RAM? Would it help if I split these up in smaller shards, but with the > existing set of hardware? (I cannot allocate more machines to the cloud as > yet)
Memory requirements are actually likely to go *up* a little bit with more shards on the same hardware, not down. The ideal RAM setup is to have enough RAM in the machine to equal or exceed the sum of your max Solr heap and the size of all the index data on that machine. This allows the operating system to load the entire index into the disk cache. Disks are *slow*, if the OS can pull the data it needs out of RAM (the operating system disk cache), that becomes *very* fast. If you have enough RAM to load at least two thirds of the index size, performance is likely to also be very good, but 100% of the index is better. If at all possible, put more RAM in the machine. Thanks, Shawn
