On 12/18/2016 7:09 PM, Lasitha Wattaladeniya wrote: > @eric : thanks for the lengthy reply. So let's say I increase the > autosoftcommit time out to may be 100 ms. In that case do I have to > wait much that time from client side before calling search ?. What's > the correct way of achieving this?
Some of the following is covered by the links you've already received. Some of it may be new information. Before you can see a change you've just made, you will need to wait for the commit to be fired (in this case, the autoSoftCommit time) plus however long it actually takes to complete the commit and open a new searcher. Opening the searcher is the expensive part. What I typically recommend that people do is have the autoSoftCommit time as long as they can stand, with 60-300 seconds as a "typical" value. That's a setting of 60000 to 300000. What you are trying to achieve is much faster, and much more difficult. 100 milliseconds will typically be far too small a value unless your index is extremely small or your hardware is incredibly fast and has a lot of memory. With a value of 100, you'll want each of those soft commits (which do open a new searcher) to take FAR less than 100 milliseconds to complete. This kind of speed can be difficult to achieve, especially if the index is large. To have any hope of fast commit times, you will need to set autowarmCount on all Solr caches to zero. If you are indexing frequently enough, you might even want to completely disable Solr's internal caches, because they may be providing no benefit. You will want to have enough extra memory that your operating system can cache the vast majority (or even maybe all) of your index. https://wiki.apache.org/solr/SolrPerformanceProblems Some other info that's helpful for understanding why plenty of *spare* memory (not allocated by programs) is necessary for good performance: https://en.wikipedia.org/wiki/Page_cache http://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html The reason in a nutshell: Disks are EXTREMELY slow. Memory is very fast. Thanks, Shawn
