Re: throttle segment merging

2012-10-30 Thread Otis Gospodnetic
Hi Radim, To address your comment about JIRA and search - perhaps this is better - and it finds you: http://search-lucene.com/?q=throttle+merge&fc_project=Solr Otis -- Search Analytics - http://sematext.com/search-analytics/index.html Performance Monitoring - http://sematext.com/spm/index.html

Re: throttle segment merging

2012-10-29 Thread Radim Kolar
Dne 29.10.2012 12:18, Michael McCandless napsal(a): With Lucene 4.0, FSDirectory now supports merge bytes/sec throttling (FSDirectory.setMaxMergeWriteMBPerSec): it rate limits that max bytes/sec load on the IO system due to merging. Not sure if it's been exposed in Solr / ElasticSearch yet ... i

Re: throttle segment merging

2012-10-29 Thread Michael McCandless
With Lucene 4.0, FSDirectory now supports merge bytes/sec throttling (FSDirectory.setMaxMergeWriteMBPerSec): it rate limits that max bytes/sec load on the IO system due to merging. Not sure if it's been exposed in Solr / ElasticSearch yet ... Mike McCandless http://blog.mikemccandless.com On Mo

Re: throttle segment merging

2012-10-29 Thread Tomás Fernández Löbbe
> > Is there way to set-up logging to output something when segment merging >>> runs? >>> >>> I think segment merging is logged when you enable infoStream logging >> (you >> should see it commented in the solrconfig.xml) >> > no, segment merging is not logged at info level. it needs customized lo

Re: throttle segment merging

2012-10-29 Thread Radim Kolar
is there JIRA ticket dedicated to throttling segment merge? i could not find any, but jira search kinda sucks. It should be ported from ES because its not much code.

Re: throttle segment merging

2012-10-29 Thread Radim Kolar
Dne 29.10.2012 0:09, Lance Norskog napsal(a): 1) Do you use compound files (CFS)? This adds a lot of overhead to merging. i do not know. whats solr configuration statement for turning them on/off? 2) Does ES use the same merge policy code as Solr? ES rate limiting: http://www.elasticsearch.or

Re: throttle segment merging

2012-10-28 Thread Lance Norskog
. - Original Message - | From: "Radim Kolar" | To: solr-user@lucene.apache.org | Sent: Saturday, October 27, 2012 7:44:46 PM | Subject: Re: throttle segment merging | | Dne 26.10.2012 3:47, Tomás Fernández Löbbe napsal(a): | >> Is there way to set-up logging to

Re: throttle segment merging

2012-10-27 Thread Radim Kolar
Dne 26.10.2012 3:47, Tomás Fernández Löbbe napsal(a): Is there way to set-up logging to output something when segment merging runs? I think segment merging is logged when you enable infoStream logging (you should see it commented in the solrconfig.xml) no, segment merging is not logged at info

Re: throttle segment merging

2012-10-25 Thread Tomás Fernández Löbbe
> > Is there way to set-up logging to output something when segment merging > runs? > I think segment merging is logged when you enable infoStream logging (you should see it commented in the solrconfig.xml) > > Can be segment merges throttled? > You can change when and how segments are merged wit