I ran some more tests. I added an explicit commit after each deleteByQuery()
call and removed the add/reindex step. This hung up immediately and completed
(or timed out?) after 20 minutes. The hangs occur almost exactly 20 minutes
apart. Could this be a Tomcat issue?
I ran jconsole but didn't see any extraordinary memory or CPU usage. The
delays appear on the first delete attempt immediately after start up so I
suspect it's not GC related.
I also tried adding documents without deleting. This worked with no
significant delays on the commit. The delete/commit combo appears to be the
source of the problem.
Any tips on how to debug this are appreciated!
Thanks,Mike
> From: mgal...@hotmail.com
> To: solr-user@lucene.apache.org
> Subject: Delays when deleting by query
> Date: Tue, 6 Dec 2011 08:25:28 -0500
>
> Hello,
>
> We're encountering delays of 10+ minutes when trying to delete from our Solr
> 3.4 instance. We have 335k documents indexed and interface using SolrJ. Our
> schema basically consists of a parent object with multiple child objects.
> Every object is indexed as a separate document with the child documents
> referencing parents via a 'parentId' field. When any part of a parent object
> is updated solrServer.deleteByQuery() is called to delete the parent and all
> the child documents, then solrServer.add() is called to reindex them. We
> currently rely on autocommit, with maxDocs set to 100 and maxTime set to 30s.
> Deletes work fine on another Solr test instance with 22k documents.
>
> Any thoughts? Is this sort of delay common when deleting against this many
> documents?
>
> Thanks,
> Mike
>
