Re: Slow deleteById request

[Renaud Delbru]

Hi Yonik,

We are not sending a commit with a delete. It happens when using the 
following command:
curl http://mydomain.net:8080/index/update -s -H 'Content-type:text/xml; 
charset=utf-8' -d "<delete><id>http://example.org/</id></delete>"
or using the SolrJ deleteById method (that does not execute a commit as 
far as I know).

The strange things is that it is not always reproduced. Ten or so delete 
requests will be executed fast (in few ms), then a batch of few delete 
requests will take 10, 20 or even 30 seconds.

By looking more precisely at the log, it seems that, in fact,  the 
delete request triggers the opening of a new searcher, with its 
auto-warming. On a large index (our case), I heard that it can take 
quite some time. Anyway, I am without a precise explanation about this 
problem.
This is not a big issue in our case, since it occurs for few requests 
and since other concurrent requests will be handled by the other searcher.

--
Renaud Delbru


Yonik Seeley wrote:
That's very strange... are you sending a commit with the delete perhaps?
If so, the whole request would block until a new searcher is registered.

-Yonik

On Tue, Jul 1, 2008 at 8:54 AM, Renaud Delbru <[EMAIL PROTECTED]> wrote:
  
Hi,

We experience very slow delete, taking more than 10 seconds. A delete is
executed using deleteById (from Solrj or from curl), at the same time
documents are being added.
By looking at the log (below), it seems that a delete by ID request is only
executed during the next commit (done automatically every 1000 added
documents), and that the process (Solrj or curl) executing the deleteById
request is blocked until the commit is performed.

Is it a normal behavior or a misconfiguration of our Solr server ?

Thanks in advance for insights.

[11:32:02.840]autowarming result for [EMAIL PROTECTED] main
[11:32:02.840]
 
queryResultCache{lookups=0,hits=0,hitratio=0.00,inserts=512,evictions=0,size=512,cumulative_lookups=238825,cumulative_hits=202879,cumulative_hitratio=0.84,cumulative_inserts=36255,c
umulative_evictions=4289}
[11:32:02.840]autowarming [EMAIL PROTECTED] main from [EMAIL PROTECTED] main
[11:32:02.840]
 
documentCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,cumulative_lookups=2395306,cumulative_hits=1705483,cumulative_hitratio=0.71,cumulative_inserts=689823,cumulative_evictions=411577}
[11:32:02.840]autowarming result for [EMAIL PROTECTED] main
[11:32:02.840]
 
documentCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,cumulative_lookups=2395306,cumulative_hits=1705483,cumulative_hitratio=0.71,cumulative_inserts=689823,cumulative_evictions=411577}
[11:32:02.840]Registered new searcher [EMAIL PROTECTED] main
[11:32:02.840]{delete=[http://example.org/]} 0 14212
[11:32:02.840]webapp=/index path=/update params={wt=xml&version=2.2}
status=0 QTime=14212
[11:32:02.840]DirectUpdateHandler2 deleting and removing dups for 217 ids
[11:32:02.840]Closing Writer DirectUpdateHandler2
[11:32:02.842]Closing [EMAIL PROTECTED] main
[11:32:02.842]
 
filterCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
[11:32:02.842]
 
queryResultCache{lookups=0,hits=0,hitratio=0.00,inserts=512,evictions=0,size=512,cumulative_lookups=238825,cumulative_hits=202879,cumulative_hitratio=0.84,cumulative_inserts=36255,cumulative_evictions=4289}
[11:32:02.842]
 
documentCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,cumulative_lookups=2395306,cumulative_hits=1705483,cumulative_hitratio=0.71,cumulative_inserts=689823,cumulative_evictions=411577}
[11:32:02.894]Opening [EMAIL PROTECTED] DirectUpdateHandler2
[11:32:03.566]DirectUpdateHandler2 docs deleted=0
[11:32:03.566]Closing [EMAIL PROTECTED] DirectUpdateHandler2

--
Renaud Delbru