Hmmmm, this isn't right. You've pretty much eliminated the obvious things. What does lsof show? I'm assuming it shows the files are being held open by your Solr instance, but it's worth checking.
I'm not getting the same behavior, admittedly on a Windows box. The only other thing I can think of is that you have a query that's somehow never ending, but that's grasping at straws. Do your log files show anything interesting? Best Erick@NotMuchHelpIKnow On Wed, Apr 20, 2011 at 8:37 AM, Bernd Fehling <bernd.fehl...@uni-bielefeld.de> wrote: > Hi Erik, > > Am 20.04.2011 13:56, schrieb Erick Erickson: >> >> Does this persist? In other words, if you just watch it for >> some time, does the disk usage go back to normal? > > Only after restarting the whole solr the disk usage goes back to normal. > >> >> Because it's typical that your index size will temporarily >> spike after the operations you describe as new searchers >> are warmed up. During that interval, both the old and new >> searchers are open. > > Temporarily yes, but still after a couple of hours after optimize > or replication? > >> >> Look particularly at your warmup time in the Solr admin page, >> that should give you an indication of how long it takes your >> warmup to happen and give you a clue about when you should >> expect the index sizes to drop again. > > We have newSearcher and firstSearcher (both with 2 simple queries) and > <useColdSearcher>false</useColdSearcher> > <maxWarmingSearchers>2</maxWarmingSearchers> > The QTime is less than 500 (0.5 second). > > warmupTime=0 for all autowarming Searcher > >> >> How often do you optimize on the master and replicate on the >> slave? Because you may be getting into the runaway warmup >> problem where a new searcher is opened before the last one >> is autowarmed and spiraling out of control. > > We commit new content about every hour and do an optimze once a day. > So replication is also once a day after optimize finished and > system has settled down. > No commit during optimize and replication. > > > Any further hints? > > >> >> Hope that helps >> Erick >> >> On Wed, Apr 20, 2011 at 2:36 AM, Bernd Fehling >> <bernd.fehl...@uni-bielefeld.de> wrote: >>> >>> Hello list, >>> >>> we have the problem that old searchers often are not closing >>> after optimize (on master) or replication (on slaves) and >>> therefore have huge index volumes. >>> Only solution so far is to stop and start solr which cleans >>> up everything successfully, but this can only be a workaround. >>> >>> Is the parameter "waitSearcher=false" an option to solve this? >>> >>> Any hints what to check or to debug? >>> >>> We use Apache Solr 3.1.0 on Linux. >>> >>> Regards >>> Bernd >>> >
