Thanks Yonik! I recognize your name from Googling for Solr issues. Your help is greatly appreciated.
I will attempt to tune maxMergeDocs as a first step. On Fri, Aug 20, 2010 at 12:39 PM, Yonik Seeley <yo...@lucidimagination.com>wrote: > This sounds like perhaps a major merge was triggered. > > You could do a nightly optimize - which will take just as long, but > you control when it happens. > The other option is to prevent too big of segments being greated (at > the expense of search speed) with options such as maxMergeDocs. > > -Yonik > http://www.lucidimagination.com > > On Fri, Aug 20, 2010 at 2:03 PM, Devin Foley <devinfo...@gmail.com> wrote: > > Hi All, > > > > I'm currently using Solr to run a search engine that is pulling in new > data > > 24 hours a day. Currently, I'm indexing about 5 million documents per > day, > > each document being around 2k in size. Every few days, the server locks > up, > > and calls to /update stop working. During this lock, CPU, memory and > disk > > utilization on the server shoot up. After 30-60 minutes, the server > starts > > accepting updates again, without intervention on my part. Also, during > the > > lock, I am still able to /select. I just can't /update. > > > > I viewed the thread dump in the admin panel during the last lockup, and > > found a lot of these: > > > > 'btpool0-868' Id=7582, WAITING on > > > lock=java.util.concurrent.locks.reentrantreadwritelock$nonfairs...@6648711e > , > > total cpu time=0.0000ms user time=0.0000ms > > ... (java.util stuff) > > > org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:211) > > > > I'm using Solr 1.4.0, SolrJ 1.4.1 (StreamingUpdateSolrServer), and Java > > 1.6.0 64bit on CentOS. > > > > My questions... > > > > 1) Has anybody else experienced this? > > 2) Has this been addressed in Solr 1.4.0? > > 3) How can I stop this from happening? > > > > Thanks! > > Devin > > >
