Hi,
I’m trying to find the upper thresholds of ingestion and I have tried the
following. In each of the experiments, I’m ingesting random documents with 5
fields.
Number of Cores Number of documents ingested per second per core
1 89000
3 33000
5 18000
As you can see, the number of documents being ingested per core is not scaling
horizontally as I'm adding more cores. Rather the total number of documents
getting ingested for Solr JVM is being topped around 90k documents per second.
>From the iostats and top commands, I do not see any bottlenecks with the iops
>or cpu respectively, CPU usaeg is around 65% and a sample of iostats is below:
avg-cpu: %user %nice %system %iowait %steal %idle
55.32 0.00 2.33 1.64 0.00 40.71
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sda5 2523.00 45812.00 298312.00 45812 298312
Can someone please guide me as to how I can debug this further and root-cause
the bottleneck for not being able to increase the ingestion horizontally.
Thanks,
Shashank
