> Set the first two to the same number, and the third to a minumum of three > times what you set the other two. > When I built a Solr setup, I increased maxMergeAtOnce and segmentsPerTier to > 35, and maxMergeAtOnceExplicit to 105. This made merging happen a lot less > frequently.
Good to know the chief recipes. > On the Solr side, atomic updates will be slightly slower than indexing the > whole document provided to Solr. This makes sense. > If Solr can access the previous document faster than you can get the > document from your source system, then atomic updates might be faster. The documents are stored within parquet files without any processing needed. In this case, the atomic update is not likely to faster things. Thanks On Wed, Oct 23, 2019 at 07:49:44AM -0600, Shawn Heisey wrote: > On 10/22/2019 1:12 PM, Nicolas Paris wrote: > > > We, at Auto-Suggest, also do atomic updates daily and specifically > > > changing merge factor gave us a boost of ~4x > > > > Interesting. What kind of change exactly on the merge factor side ? > > The mergeFactor setting is deprecated. Instead, use maxMergeAtOnce, > segmentsPerTier, and a setting that is not mentioned in the ref guide -- > maxMergeAtOnceExplicit. > > Set the first two to the same number, and the third to a minumum of three > times what you set the other two. > > The default setting for maxMergeAtOnce and segmentsPerTier is 10, with 30 > for maxMergeAtOnceExplicit. When you're trying to increase indexing speed > and you think segment merging is interfering, you want to increase these > values to something larger. Note that increasing these values will increase > the number of files that your Solr install keeps open. > > https://lucene.apache.org/solr/guide/8_1/indexconfig-in-solrconfig.html#mergepolicyfactory > > When I built a Solr setup, I increased maxMergeAtOnce and segmentsPerTier to > 35, and maxMergeAtOnceExplicit to 105. This made merging happen a lot less > frequently. > > > Would you say atomical update is faster than regular replacement of > > documents ? (considering my first thought on this below) > > On the Solr side, atomic updates will be slightly slower than indexing the > whole document provided to Solr. When an atomic update is done, Solr will > find the existing document, then combine what's in that document with the > changes you specify using the atomic update, and then index the whole > combined document as a new document that replaces with original. > > Whether or not atomic updates are faster or slower in practice than indexing > the whole document will depend on how your source systems work, and that is > not something we can know. If Solr can access the previous document faster > than you can get the document from your source system, then atomic updates > might be faster. > > Thanks, > Shawn > -- nicolas
