Hi Experts,
We are using solr 8.4 (none cloud). When ingesting data with multiple processes
to one core in a solr node, we are hitting some throttling: the max ingestion
rate achieved is about 47K docs per second with 17 posting processes; each doc
is about 250 bytes; the CPU utilization rate is only 20% and I/O about 6%. When
increasing the posting processes, the posting will start failing. With solr
6.6, such issue does not happen: increasing posting processes will increase
CPU/IO utilization rates to be close to 100% then start failing.
Below are some relevant configurations specified in solrconfig.xml:
<indexConfig> <maxIndexingThreads>16</maxIndexingThreads>
<ramBufferSizeMB>1024</ramBufferSizeMB> <mergePolicyFactory
class="org.apache.solr.index.TieredMergePolicyFactory"> <int
name="maxMergeAtOnce">10</int> <int name="segmentsPerTier">10</int>
<double name="maxMergedSegmentMB">4096</double> <double
name="noCFSRatio">0.1</double> <double
name="maxCFSSegmentSizeMB">4096</double> </mergePolicyFactory>
<lockType>${solr.lock.type:native}</lockType>
<infoStream>true</infoStream></indexConfig>
<updateHandler class="solr.DirectUpdateHandler2"> <updateLog> <str
name="dir">${solr.ulog.dir:}</str> <int
name="numVersionBuckets">${solr.ulog.numVersionBuckets:65536}</int>
</updateLog>
<autoCommit> <maxTime>${solr.autoCommit.maxTime:120000}</maxTime>
<openSearcher>false</openSearcher> </autoCommit>
<autoSoftCommit> <maxTime>${solr.autoSoftCommit.maxTime:5000}</maxTime>
</autoSoftCommit></updateHandler>
It seems maxIndexingThreads is no longer supported in solr 8? Any idea to break
the solr throttling? Thanks.
Shushuai
