Re: Recent configuration change to our site causes frequent index corruption

2018-07-30 Thread Shawn Heisey
On 7/27/2018 11:02 AM, cyndefromva wrote: > I'm just curious why are there still so many 503 errors being generated > (Error - Rsolr::Error::Http - 503 Service Unavailable - retrying ...) > > Is it related to all the "Error opening new searcher. exceeded limit of > maxWarmingSearchers=2, try again

Re: Recent configuration change to our site causes frequent index corruption

2018-07-27 Thread Erick Erickson
bq: Error opening new searcher. exceeded limit of maxWarmingSearchers=2 did you make sure that your indexing client isn't issuing commits all the time? The other possible culprit (although I'd be very surprised) is if you have your filterCache and queryResultCache autowarm settings set extremely h

Re: Recent configuration change to our site causes frequent index corruption

2018-07-27 Thread cyndefromva
That makes sense, the ulimit was too small and I've updated it. I'm just curious why are there still so many 503 errors being generated (Error - Rsolr::Error::Http - 503 Service Unavailable - retrying ...) Is it related to all the "Error opening new searcher. exceeded limit of maxWarmingSearcher

Re: Recent configuration change to our site causes frequent index corruption

2018-07-27 Thread Pure Host - Wolfgang Freudenberger
Hi, You have to increase the openfile limit for your SOLR user - you can check it with uname -a. It should show something about 1024. To increase it, you have to raise the systemlimit in /etc/security/limits.conf. Add the following lines: * hard nofile 102400 * soft nofile 102400 root hard no

Re: Recent configuration change to our site causes frequent index corruption

2018-07-27 Thread Shawn Heisey
On 7/26/2018 1:32 PM, cyndefromva wrote: At the point it starts failing I see a java exception: "java.io-IOException: Too many open files" in the solr log file and a SolrException (Error open new searcher) is returned to the user. The operating system where Solr is running needs its open file l

Re: Recent configuration change to our site causes frequent index corruption

2018-07-26 Thread Erick Erickson
And as for your performance warning Overlapping onDeckSearchers, almost certainly some external process (probably the indexing client) is issuing the commits. Best, Erick. On Thu, Jul 26, 2018 at 1:43 PM, Markus Jelsma wrote: > Hello, > > Is your maximum number of open files 1024? If so, increas

RE: Recent configuration change to our site causes frequent index corruption

2018-07-26 Thread Markus Jelsma
Hello, Is your maximum number of open files 1024? If so, increase it to a more regular 65536. Some operating systems ship with 1024 for reasons i don't understand. Whenever installing Solr anywhere for the past ten years, we have had to check this each and every time, and still have to! Regard