Hey Yonik,
I have realized that the problem is happening not because of the snapshots.
I mean, I do an index and optimize it. Modifiy the database and to
delta-import and optimize again... After that I do an lsof of the tomcat
user and I see the old index still holded by tomcat...
...
ava 12814 tomcat 88r REG 8,2 149279785 2902977
/home/smack/solr/data/index/_56n.frq (deleted)
java 12814 tomcat 89r REG 8,2 91797827 2902978
/home/smack/solr/data/index/_56n.prx (deleted)
java 12814 tomcat 90r REG 8,2 4196563 2902979
/home/smack/solr/data/index/_56n.nrm (deleted)
...
I am using not a plugin, just implemented a transformer to modify the data
but this should'nt afect.
At least I see that actually the problem seems to be that an indexSearcher
or indexWriter is not beig closed...
I have checked the log and it is closing an indexWriter and registering a
new searcher but can't see that the older one is closed:
Mar 11 17:02:37 127.0.0.1 solr: 3926476 [Thread-47] DEBUG
org.apache.solr.update.SolrIndexWriter - Closing Writer DirectUpdateHandler2
Mar 11 17:02:38 127.0.0.1 solr: 3927659 [Thread-47] INFO
org.apache.solr.core.SolrCore - TrovitDeletionPolicy.onCommit: commits:
Mar 11 17:02:38 127.0.0.1 solr: 3927660 [Thread-47] INFO
org.apache.solr.core.SolrCore - last commit = 1234970421620
Mar 11 17:02:38 127.0.0.1 solr: 3927924 [Thread-47] INFO
org.apache.solr.search.SolrIndexSearcher - Opening searc...@a9f827 main
Mar 11 17:02:38 127.0.0.1 solr: 3927924 [Thread-47] INFO
org.apache.solr.update.UpdateHandler - end_commit_flush
Mar 11 17:02:38 127.0.0.1 solr: 3927925 [pool-11-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher - autowarming searc...@a9f827 main
from searc...@1e0bf98 main
fieldValueCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=3,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0,item_fa_region={field=fa_region,memSize=5599982,tindexSize=56,time=48,phase1=44,nTerms=50,bigTerms=3,termInstances=488689,uses=3},item_fa_city_area={field=fa_city_area,memSize=5599654,tindexSize=50,time=469,phase1=465,nTerms=18,bigTerms=0,termInstances=352,uses=3},item_fa_city={field=fa_city,memSize=5633866,tindexSize=1494,time=94,phase1=91,nTerms=5767,bigTerms=0,termInstances=685467,uses=3}}
Mar 11 17:02:38 127.0.0.1 solr: 3927925 [pool-11-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher - autowarming result for
searc...@a9f827 main
fieldValueCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
Mar 11 17:02:38 127.0.0.1 solr: 3927925 [pool-11-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher - autowarming searc...@a9f827 main
from searc...@1e0bf98 main
filterCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=20,warmupTime=279,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
Mar 11 17:02:38 127.0.0.1 solr: 3928161 [pool-11-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher - autowarming result for
searc...@a9f827 main
filterCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=20,warmupTime=236,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
Mar 11 17:02:38 127.0.0.1 solr: 3928161 [pool-11-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher - autowarming searc...@a9f827 main
from searc...@1e0bf98 main
queryResultCache{lookups=0,hits=0,hitratio=0.00,inserts=5,evictions=0,size=5,warmupTime=10,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
Mar 11 17:02:38 127.0.0.1 solr: 3928171 [pool-11-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher - autowarming result for
searc...@a9f827 main
queryResultCache{lookups=0,hits=0,hitratio=0.00,inserts=5,evictions=0,size=5,warmupTime=10,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
Mar 11 17:02:38 127.0.0.1 solr: 3928171 [pool-11-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher - autowarming searc...@a9f827 main
from searc...@1e0bf98 main
documentCache{lookups=0,hits=0,hitratio=0.00,inserts=5,evictions=0,size=5,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
Mar 11 17:02:38 127.0.0.1 solr: 3928171 [pool-11-thread-1] INFO
org.apache.solr.search.SolrIndexSearcher - autowarming result for
searc...@a9f827 main
documentCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
Mar 11 17:02:38 127.0.0.1 solr: 3928171 [pool-11-thread-1] INFO
org.apache.solr.core.SolrCore - QuerySenderListener sending requests to
searc...@a9f827 main
Mar 11 17:02:38 127.0.0.1 solr: 3928260 [pool-11-thread-1] INFO
org.apache.solr.core.SolrCore - UnInverted multi-valued field
{field=fa_city,memSize=5633866,tindexSize=1494,time=85,phase1=81,nTerms=5767,bigTerms=0,termInstances=685467,uses=0}
Mar 11 17:02:39 127.0.0.1 solr: 3928330 [pool-11-thread-1] INFO
org.apache.solr.core.SolrCore - UnInverted multi-valued field
{field=fa_city_area,memSize=5599654,tindexSize=50,time=70,phase1=66,nTerms=18,bigTerms=0,termInstances=352,uses=0}
Mar 11 17:02:39 127.0.0.1 solr: 3928377 [pool-11-thread-1] INFO
org.apache.solr.core.SolrCore - UnInverted multi-valued field
{field=fa_region,memSize=5599982,tindexSize=56,time=47,phase1=44,nTerms=50,bigTerms=3,termInstances=488689,uses=0}
Mar 11 17:02:39 127.0.0.1 solr: 3928378 [pool-11-thread-1] INFO
org.apache.solr.core.SolrCore - [search_homes_es] webapp=null path=null
params={start=0&q=solr&rows=10} hits=0 status=0 QTime=207
Mar 11 17:02:39 127.0.0.1 solr: 3928426 [pool-11-thread-1] INFO
org.apache.solr.core.SolrCore - [search_homes_es] webapp=null path=null
params={start=0&q=rocks&rows=10} hits=5 status=0 QTime=48
Mar 11 17:02:39 127.0.0.1 solr: 3928437 [pool-11-thread-1] INFO
org.apache.solr.core.SolrCore - [search_homes_es] webapp=null path=null
params={q=static+newSearcher+warming+query+from+solrconfig.xml} hits=0
status=0 QTime=10
Mar 11 17:02:39 127.0.0.1 solr: 3928437 [pool-11-thread-1] INFO
org.apache.solr.core.SolrCore - QuerySenderListener done.
Mar 11 17:02:39 127.0.0.1 solr: 3928439 [pool-11-thread-1] INFO
org.apache.solr.core.SolrCore - [search_homes_es] Registered new searcher
searc...@a9f827 main
Definitely looks like the problem is around here... Do you know where in the
source code should I try to close that indexSeracher?
Thanks for answering as I am really stuck with this problem...
Yonik Seeley-2 wrote:
>
> On Wed, Mar 11, 2009 at 5:17 AM, Marc Sturlese <marc.sturl...@gmail.com>
> wrote:
>> I have noticed that once snapshots are deleted, tomcat keeps holding
>> references to them.Due to this disk space will never set free until
>> Tomcat
>> is restarted, I have realized that doing a lsof | grep 'tomcat', the
>> result
>> look like:
>
> That suggests that the index files aren't being closed for some reason.
> After the new index searcher is registered, do you see the close of
> the old index reader in the logs?
> Do you have any Solr modifications or plugins?
>
> -Yonik
> http://www.lucidimagination.com
>
>
>> java 22015 tomcat 614r REG 253,0 1149569723 1093456605
>> /var/local/solr/data/index/_1fb.fdt (deleted)
>> java 22015 tomcat 615r REG 253,0 12724500 1093456606
>> /var/local/solr/data/index/_1fb.fdx (deleted)
>> java 22015 tomcat 616r REG 253,0 175953343 1093456607
>> /var/local/solr/data/index/_1fb.tis (deleted)
>> java 22015 tomcat 617r REG 253,0 1989522 1102344114
>> /var/local/solr/sdata/index/_1fb.tii (deleted)
>> java 22015 tomcat 618r REG 253,0 178646437 1102344140
>> /var/local/solr/data/index/_1fb.frq (deleted)
>> java 22015 tomcat 619r REG 253,0 108460405 1102344154
>> /var/local/solr/sdata/index/_1fb.prx (deleted)
>>
>> ...
>>
>> How can I make tomcat free the snapshots. Or even better... why is it
>> happening?
>> Thanks in advance
>> --
>> View this message in context:
>> http://www.nabble.com/Tomcat-holding-deleted-snapshots-until-it%27s-restarted-tp22451252p22451252.html
>> Sent from the Solr - User mailing list archive at Nabble.com.
>>
>>
>
>
--
View this message in context:
http://www.nabble.com/Tomcat-holding-deleted-snapshots-until-it%27s-restarted-tp22451252p22458806.html
Sent from the Solr - User mailing list archive at Nabble.com.
