With Lucene 4.0, FSDirectory now supports merge bytes/sec throttling (FSDirectory.setMaxMergeWriteMBPerSec): it rate limits that max bytes/sec load on the IO system due to merging.
Not sure if it's been exposed in Solr / ElasticSearch yet ... Mike McCandless http://blog.mikemccandless.com On Mon, Oct 29, 2012 at 7:07 AM, Tomás Fernández Löbbe <tomasflo...@gmail.com> wrote: >> >> Is there way to set-up logging to output something when segment merging >>>> runs? >>>> >>>> I think segment merging is logged when you enable infoStream logging >>> (you >>> should see it commented in the solrconfig.xml) >>> >> no, segment merging is not logged at info level. it needs customized log >> config. > > > INFO level is not the same as infoStream. See solrconfig, there is a > commented section that talks about it, and if you uncomment it it will > generate a file with low level Lucene logging. This file will include > segments information, including merging. > >> >> >> >>> Can be segment merges throttled? >>>> >>> > You can change when and how segments are merged with the merge policy, >> maybe it's enough for you changing the initial settings (mergeFactor for >> example)? >> >> I am now researching elasticsearch, it can do it, its lucene 3.6 based >> > > > I don't know if this is what you are looking for, but the TieredMergePolicy > (default) allows you to set maximum number of segments to be merged at once > and maximum size of segments to be created during normal merging. > Other option is, as you said, create a Jira for a new merge policy. > > Tomás
