Hi,
What are your commit configs? Maybe you are committing too frequently.
Thanks,
Emir
> On 21 Sep 2017, at 06:19, saiks <karlapudisam...@gmail.com> wrote:
>
> Hi,
>
> Environment:
> - Solr is running in non-cloud mode on 6.4.2, Sun Java8, Linux
> 4.4.0-31-generic x86_64
> - Ingesting into a single core
> - SoftCommit = 5 seconds, HardCommit = 10 seconds
> - System has 16 Cpus and 32 Gb of memory (Solr is given 20 Gb of JVM heap)
> - text = StandardTokenizer, id = solr.StrField/docValues, hostname =
> solr.StrField/docValues, app = solr.StrField/docValues, epoch =
> solr.TrieLongField/docValues
>
> I am using jmeter to ingest to Solr core using UpdateRequestHandle
> ("/update/json") and sending in a batch of 1000 messages(same message) in a
> single json array.
>
> Sample message
> [{"text":"May 11 10:18:22 scrooge Web-Requests: May 11 10:18:22
> @IunAIir1----7k-- EVENT_WR-Y-attack-600 SG_child[823]: [event.error]
> Possible attack - 5 blocked requests within 120 seconds",
> "id":"id1",
> "hostname": "xxxxxxxxxx.com",
> "app": "yyyy",
> "epoch": 1483667347941
> },
> ....]
>
> Jmeter is configured to run 10 threads in parallel repeating the request
> 1000 times, which should ingest 10,000,000 messages in total.
> Jmeter post url:
> "/solr/mycore/update/json?overwrite=false&wt=json&commit=false"
>
> Jmeter summary:
> summary = 5000 in 00:03:07 = 26.7/s Avg: 370 Min: 27 Max: 1734
> Err: 0 (0.00%)
>
> I am only able to ingest 26000 messages per second, looking at system
> resources only one or two cpus are at 25-30% and the rest are sitting idle
> and also Solr heap is flat at 3Gb with no iowait on the devices.
> Increasing parallelism in Jmeter to ingest using 20 threads did not increase
> ingested messages per second, but increased the latency by 2x for each
> request.
>
> I don't understand why Solr is not able to use all the cpus on the host if I
> increase Jmeter parallelism from 10 -> 20 -> 40. What can I do to achieve
> performance gain and make Solr utilize system resources to their maximum.
>
> Please help.
>
> Thank you
>
>
>
>
>
>
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
