Chris: OK, one of those stack traces does have the problem I referenced in the other thread. Are you sending updates to the server with SolrJ? And are you using CloudSolrServer? If you are, I'm surprised...
There are the important lines: 1. - java.util.concurrent.Semaphore.acquire() @bci=5, line=317 (Compiled frame) 2. - org.apache.solr.util.AdjustableSemaphore.acquire() @bci=4, line=61 (Compiled frame) 3. - org.apache.solr.update.SolrCmdDistributor.submit(org.apache.solr. update.SolrCmdDistributor$Request) @bci=22, line=418 (Compiled frame) 4. - org.apache.solr.update.SolrCmdDistributor.submit(org.apache.solr. client.solrj.request.UpdateRequest, On Wed, Oct 16, 2013 at 2:04 PM, Chris Geeringh <geeri...@gmail.com> wrote: > Here's another jstack http://pastebin.com/8JiQc3rb > > > On 16 October 2013 11:53, Chris Geeringh <geeri...@gmail.com> wrote: > > > Hi Erick, here is a paste from other thread (debugging update request) > > with my input as I am seeing errors too: > > > > I ran an import last night, and this morning my cloud wouldn't accept > > updates. I'm running the latest 4.6 snapshot. I was importing with latest > > solrj snapshot, and using java bin transport with CloudSolrServer. > > > > The cluster had indexed ~1.3 million docs before no further updates were > > accepted, querying still working. > > > > I'll run jstack shortly and provide the results. > > > > Here is my jstack output... Lots of blocked threads. > > > > http://pastebin.com/1ktjBYbf > > > > > > > > On 16 October 2013 11:46, Erick Erickson <erickerick...@gmail.com> > wrote: > > > >> Run jstack on the solr process (standard with Java) and > >> look for the word "semaphore". You should see your > >> servers blocked on this in the Solr code. That'll pretty > >> much nail it. > >> > >> There's an open JIRA to fix the underlying cause, see: > >> SOLR-5232, but that's currently slated for 4.6 which > >> won't be cut for a while. > >> > >> Also, there's a patch that will fix this as a side effect, > >> assuming you're using SolrJ, see. This is available in 4.5 > >> SOLR-4816 > >> > >> Best, > >> Erick > >> > >> > >> > >> > >> On Tue, Oct 15, 2013 at 1:33 PM, michael.boom <my_sky...@yahoo.com> > >> wrote: > >> > >> > Here's some of the Solr's last words (log content before it stoped > >> > accepting > >> > updates), maybe someone can help me interpret that. > >> > http://pastebin.com/mv7fH62H > >> > > >> > > >> > > >> > -- > >> > View this message in context: > >> > > >> > http://lucene.472066.n3.nabble.com/Concurent-indexing-tp4095409p4095642.html > >> > Sent from the Solr - User mailing list archive at Nabble.com. > >> > > >> > > > > >
